How Do I Deal With The Uploaded File Data Manually?

I am trying to save the uploaded file manually for learning, without the help of request.files. I am using Flask and it's giving me a str type of the raw body data, by print(type(

Solution 1:

Finally I figured it out by myself.

1. The reason why I am getting str from request.data is because I was using python2.7 flask packages. Such a pain in the ass. I would definitely use virtualenv next time, even for testing. But I still don't understand why I can use python3 syntax in the server code while the package is related to python2.7.

2. So actually I am getting raw byte data like b'raw binary data' from request.data. And data from other functions e.g. like request.form['firstname'] have already been decoded.

So now the question boilds down to how I can rebuild the file, provided the binary data.

---

Af first, I prepare 2 small files for testing.

file1: 1.txt

content: 1234567

file2: test.png this small image ------>

content (use open('test.png', 'rb').read()):

b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\t\x00\x00\x00\x08\x08\x02\x00\x00\x00\xa4\xafB\xe2\x00\x00\x00\x03sBIT\x08\x08\x08\xdb\xe1O\xe0\x00\x00\x00\x10tEXtSoftware\x00Shutterc\x82\xd0\t\x00\x00\x00\x15IDAT\x08\xd7c\xd4\xe5Tb\xc0\x01\x98\x18p\x83\xa1"\x07\x00T;\x00h\xb9\x9335\x00\x00\x00\x00IEND\xaeB`\x82'

So the request.data I see on the server is:

b'-----------------------------16866548741414816351605255076\r\nContent-Disposition: form-data; name="myfile"; filename="1.txt"\r\nContent-Type: text/plain\r\n\r\n1234567\r\n-----------------------------16866548741414816351605255076\r\nContent-Disposition: form-data; name="myfile2"; filename="test.png"\r\nContent-Type: image/png\r\n\r\n\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\t\x00\x00\x00\x08\x08\x02\x00\x00\x00\xa4\xafB\xe2\x00\x00\x00\x03sBIT\x08\x08\x08\xdb\xe1O\xe0\x00\x00\x00\x10tEXtSoftware\x00Shutterc\x82\xd0\t\x00\x00\x00\x15IDAT\x08\xd7c\xd4\xe5Tb\xc0\x01\x98\x18p\x83\xa1"\x07\x00T;\x00h\xb9\x9335\x00\x00\x00\x00IEND\xaeB`\x82\r\n-----------------------------16866548741414816351605255076--\r\n'

Format it a little bit:

(The data can't be used directly because I added extra new lines for displaying.)

b'-----------------------------16866548741414816351605255076\r\n  
Content-Disposition: form-data; name="myfile"; filename="1.txt"\r\n
Content-Type: text/plain\r\n\r\n
1234567\r\n
  -----------------------------16866548741414816351605255076\r\n 
Content-Disposition: form-data; name="myfile2"; filename="test.png"\r\n
Content-Type: image/png\r\n\r\n
\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\t\x00\x00\x00\x08\x08\x02\x00\x00\x00\xa4\xafB\xe2\x00\x00\x00\x03sBIT\x08\x08\x08\xdb\xe1O\xe0\x00\x00\x00\x10tEXtSoftware\x00Shutterc\x82\xd0\t\x00\x00\x00\x15IDAT\x08\xd7c\xd4\xe5Tb\xc0\x01\x98\x18p\x83\xa1"\x07\x00T;\x00h\xb9\x9335\x00\x00\x00\x00IEND\xaeB`\x82\r\n
-----------------------------16866548741414816351605255076--\r\n'

---

Let raw_data = binary data above

1. files_data_array = raw_data.split(b'-----------------------------16866548741414816351605255076\r\n)

Then I get an array with each files in different index.

Here files_data_array[1] contains the first file meta info and data. files_data_array[2] contains the second file meta info and data. And so on if you have more files.

[b'', b'Content-Disposition: form-data; name="myfile"; filename="1.txt"\r\nContent-Type: text/plain\r\n\r\n1234567\r\n', b'Content-Disposition: form-data; name="myfile2"; filename="test.png"\r\nContent-Type: image/png\r\n\r\n\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\t\x00\x00\x00\x08\x08\x02\x00\x00\x00\xa4\xafB\xe2\x00\x00\x00\x03sBIT\x08\x08\x08\xdb\xe1O\xe0\x00\x00\x00\x10tEXtSoftware\x00Shutterc\x82\xd0\t\x00\x00\x00\x15IDAT\x08\xd7c\xd4\xe5Tb\xc0\x01\x98\x18p\x83\xa1"\x07\x00T;\x00h\xb9\x9335\x00\x00\x00\x00IEND\xaeB`\x82\r\n-----------------------------16866548741414816351605255076--\r\n']

2. file2_data = files_data_array[2]

b'Content-Disposition: form-data; name="myfile2"; filename="test.png"\r\nContent-Type: image/png\r\n\r\n\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\t\x00\x00\x00\x08\x08\x02\x00\x00\x00\xa4\xafB\xe2\x00\x00\x00\x03sBIT\x08\x08\x08\xdb\xe1O\xe0\x00\x00\x00\x10tEXtSoftware\x00Shutterc\x82\xd0\t\x00\x00\x00\x15IDAT\x08\xd7c\xd4\xe5Tb\xc0\x01\x98\x18p\x83\xa1"\x07\x00T;\x00h\xb9\x9335\x00\x00\x00\x00IEND\xaeB`\x82\r\n-----------------------------16866548741414816351605255076--\r\n'

Then split the meta by file2_meta_info = file2_data.split(b'\r\n\r\n', maxsplit=1)[0]. Notice that here I am splitting binary data, and in case there are b'\r\n\r\n' in the file data, settting maxsplit is necessary.

Now I get file2_meta_info as b'Content-Disposition: form-data; name="myfile"; filename="1.txt"', I can decode it and get whatever meta info I want.

Now turn to the file body data itself, file2_body_data = file2_data.split(b'\r\n\r\n', maxsplit=1)[1]

I get

b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\t\x00\x00\x00\x08\x08\x02\x00\x00\x00\xa4\xafB\xe2\x00\x00\x00\x03sBIT\x08\x08\x08\xdb\xe1O\xe0\x00\x00\x00\x10tEXtSoftware\x00Shutterc\x82\xd0\t\x00\x00\x00\x15IDAT\x08\xd7c\xd4\xe5Tb\xc0\x01\x98\x18p\x83\xa1"\x07\x00T;\x00h\xb9\x9335\x00\x00\x00\x00IEND\xaeB`\x82\r\n-----------------------------16866548741414816351605255076--\r\n'

I still need to cut some bytes, compared to the content of test.png showed at the beginning

real_file2_body_data = file2_body_data.rsplit(b'\r\n', maxsplit=2)[0]'

Finally I can rebuid the file with :

f = open('test2.png', 'wb')
f.write(real_file2_body_data)
f.close()

Done!