How to Read File Without Upload Flask
A common feature in web applications is to let users upload files to the server. The HTTP protocol documents the machinery for a client to upload a file in RFC 1867, and our favorite spider web framework Flask fully supports information technology, just there are many implementation details that fall exterior of the formal specification that are unclear for many developers. Things such as where to store uploaded files, how to use them afterward, or how to protect the server against malicious file uploads generate a lot of defoliation and doubt.
In this commodity I'thousand going to bear witness y'all how to implement a robust file upload characteristic for your Flask server that is uniform with the standard file upload back up in your spider web browser besides as the cool JavaScript-based upload widgets:
A Basic File Upload Course
From a high-level perspective, a client uploading a file is treated the same equally any other course data submission. In other words, you have to define an HTML course with a file field in it.
Here is a simple HTML folio with a form that accepts a file:
<!doctype html> <html> <head> <title>File Upload</title> </head> <torso> <h1>File Upload</h1> <form method="POST" activity="" enctype="multipart/form-information"> <p><input type="file" name="file"></p> <p><input blazon="submit" value="Submit"></p> </course> </torso> </html> 
Every bit you probably know, the method attribute of the <form> element can exist Get or POST. With GET, the data is submitted in the query cord of the request URL, while with Mail information technology goes in the request body. When files are beingness included in the form, you lot must use POST, equally it would be impossible to submit file data in the query string.
The enctype aspect in the <form> element is normally not included with forms that don't have files. This attribute defines how the browser should format the data before it is submitted to the server. The HTML specification defines iii possible values for it:
-
application/ten-world wide web-form-urlencoded: This is the default, and the best format for any forms except those that contain file fields. -
multipart/form-data: This format is required when at least one of the fields in the form is a file field. -
text/plainly: This format has no applied employ, so you lot should ignore it.
The bodily file field is the standard <input> element that we use for most other class fields, with the blazon set to file. In the instance above I haven't included any additional attributes, but the file field supports two that are sometimes useful:
-
multiplecan exist used to allow multiple files to be uploaded in a unmarried file field. Instance:
<input blazon="file" name="file" multiple> -
acceptcan exist used to filter the allowed file types that can be selected, either by file extension or past media type. Examples:
<input type="file" name="doc_file" take=".physician,.docx"> <input type="file" proper name="image_file" have="image/*"> Accepting File Submissions with Flask
For regular forms, Flask provides admission to submitted form fields in the request.course lexicon. File fields, nevertheless, are included in the asking.files dictionary. The request.form and asking.files dictionaries are really "multi-dicts", a specialized dictionary implementation that supports indistinguishable keys. This is necessary because forms can include multiple fields with the aforementioned name, as is often the case with groups of bank check boxes. This besides happens with file fields that allow multiple files.
Ignoring important aspects such equally validation and security for the moment, the brusk Flask application shown below accepts a file uploaded with the course shown in the previous section, and writes the submitted file to the current directory:
from flask import Flask, render_template, request, redirect, url_for app = Flask(__name__) @app.route('/') def index(): return render_template('index.html') @app.route('/', methods=['POST']) def upload_file(): uploaded_file = request.files['file'] if uploaded_file.filename != '': uploaded_file.salve(uploaded_file.filename) return redirect(url_for('index')) The upload_file() office is busy with @app.road and then that information technology is invoked when the browser sends a POST request. Note how the same root URL is dissever between two view functions, with index() set to take the Go requests and upload_file() the Post ones.
The uploaded_file variable holds the submitted file object. This is an instance of class FileStorage, which Flask imports from Werkzeug.
The filename attribute in the FileStorage provides the filename submitted by the customer. If the user submits the form without selecting a file in the file field, then the filename is going to be an empty string, then it is important to always check the filename to determine if a file is bachelor or not.
When Flask receives a file submission information technology does not automatically write it to disk. This is really a good thing, because it gives the awarding the opportunity to review and validate the file submission, as you will run across later. The actual file data can be accessed from the stream attribute. If the awarding just wants to save the file to disk, then it can phone call the save() method, passing the desired path as an argument. If the file's save() method is not called, then the file is discarded.
Want to exam file uploads with this awarding? Make a directory for your application and write the code to a higher place every bit app.py. Then create a templates subdirectory, and write the HTML page from the previous section as templates/alphabetize.html. Create a virtual environment and install Flask on it, then run the application with flask run. Every fourth dimension you lot submit a file, the server will write a copy of information technology in the current directory.
Before I movement on to the topic of security, I'm going to talk over a few variations on the code shown higher up that yous may find useful. As I mentioned before, the file upload field can be configured to have multiple files. If you use request.files['file'] as above y'all will get only one of the submitted files, simply with the getlist() method you tin access all of them in a for-loop:
for uploaded_file in request.files.getlist('file'): if uploaded_file.filename != '': uploaded_file.salvage(uploaded_file.filename) Many people code their form treatment routes in Flask using a single view role for both the GET and POST requests. A version of the example application using a single view function could exist coded equally follows:
@app.road('/', methods=['Become', 'POST']) def index(): if request.method == 'POST': uploaded_file = asking.files['file'] if uploaded_file.filename != '': uploaded_file.relieve(uploaded_file.filename) return redirect(url_for('index')) return render_template('index.html') Finally, if you employ the Flask-WTF extension to handle your forms, y'all can use the FileField object for your file uploads. The form used in the examples you've seen so far can be written using Flask-WTF as follows:
from flask_wtf import FlaskForm from flask_wtf.file import FileField from wtforms import SubmitField course MyForm(FlaskForm): file = FileField('File') submit = SubmitField('Submit') Note that the FileField object comes from the flask_wtf package, unlike most other field classes, which are imported directly from the wtforms package. Flask-WTF provides two validators for file fields, FileRequired, which performs a cheque similar to the empty string cheque, and FileAllowed, which ensures the file extension is included in an allowed extensions list.
When you utilise a Flask-WTF course, the data attribute of the file field object points to the FileStorage example, then saving a file to deejay works in the same mode as in the examples to a higher place.
Securing file uploads
The file upload instance presented in the previous section is an extremely simplistic implementation that is non very robust. Ane of the most important rules in spider web development is that data submitted past clients should never be trusted, and for that reason when working with regular forms, an extension such as Flask-WTF performs strict validation of all fields before the grade is accepted and the information incorporated into the application. For forms that include file fields in that location needs to be validation as well, because without file validation the server leaves the door open up to attacks. For case:
- An attacker can upload a file that is so big that the disk infinite in the server is completely filled, causing the server to malfunction.
- An attacker can arts and crafts an upload asking that uses a filename such equally ../../../.bashrc or similar, with the attempt to trick the server into rewriting system configuration files.
- An assaulter tin upload files with viruses or other types of malware in a place where the application, for example, expects images.
Limiting the size of uploaded files
To prevent clients from uploading very big files, you can apply a configuration option provided by Flask. The MAX_CONTENT_LENGTH pick controls the maximum size a request body tin have. While this isn't an option that is specific to file uploads, setting a maximum asking body size finer makes Flask discard any incoming requests that are larger than the immune amount with a 413 status lawmaking.
Let'south modify the app.py case from the previous department to but have requests that are up to 1MB in size:
app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 If yous attempt to upload a file that is larger than 1MB, the awarding will now refuse it.
Validating filenames
We can't really trust that the filenames provided by the client are valid and safe to use, so filenames coming with uploaded files accept to exist validated.
A very unproblematic validation to perform is to make sure that the file extension is ane that the application is willing to accept, which is like to what the FileAllowed validator does when using Flask-WTF. Let's say the application accepts images, and so it tin can configure the list of approved file extensions:
app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] For every uploaded file, the awarding tin can make sure that the file extension is i of the allowed ones:
filename = uploaded_file.filename if filename != '': file_ext = os.path.splitext(filename)[ane] if file_ext not in current_app.config['UPLOAD_EXTENSIONS']: abort(400) With this logic, whatsoever filenames that do non have one of the approved file extensions is going to be responded with a 400 error.
In addition to the file extension, it is also important to validate the filename, and any path given with it. If your awarding does not intendance well-nigh the filename provided past the client, the nigh secure way to handle the upload is to ignore the client provided filename and generate your own filename instead, that yous pass to the save() method. An example utilize example where this technique works well is with avatar image uploads. Each user'south avatar can be saved with the user id as filename, so the filename provided past the client can be discarded. If your application uses Flask-Login, you could implement the following save() phone call:
uploaded_file.relieve(os.path.bring together('static/avatars', current_user.get_id())) In other cases information technology may be better to preserve the filenames provided by the client, so the filename must be sanitized first. For those cases Werkzeug provides the secure_filename() function. Let's run across how this function works by running a few tests in a Python session:
>>> from werkzeug.utils import secure_filename >>> secure_filename('foo.jpg') 'foo.jpg' >>> secure_filename('/some/path/foo.jpg') 'some_path_foo.jpg' >>> secure_filename('../../../.bashrc') 'bashrc' As yous run across in the examples, no matter how complicated or malicious the filename is, the secure_filename() function reduces it to a flat filename.
Allow's incorporate secure_filename() into the instance upload server, and also add together a configuration variable that defines a dedicated location for file uploads. Here is the complete app.py source file with secure filenames:
import os from flask import Flask, render_template, asking, redirect, url_for, abort from werkzeug.utils import secure_filename app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads' @app.road('/') def index(): return render_template('index.html') @app.route('/', methods=['POST']) def upload_files(): uploaded_file = request.files['file'] filename = secure_filename(uploaded_file.filename) if filename != '': file_ext = os.path.splitext(filename)[one] if file_ext non in app.config['UPLOAD_EXTENSIONS']: abort(400) uploaded_file.save(os.path.join(app.config['UPLOAD_PATH'], filename)) render redirect(url_for('alphabetize')) Validating file contents
The third layer of validation that I'1000 going to talk over is the virtually circuitous. If your application accepts uploads of a certain file type, it should ideally perform some form of content validation and turn down whatsoever files that are of a different type.
How you accomplish content validation largely depends on the file types your application accepts. For the example application in this article I'm using images, so I can use the imghdr parcel from the Python standard library to validate that the header of the file is, in fact, an image.
Permit'southward write a validate_image() function that performs content validation on images:
import imghdr def validate_image(stream): header = stream.read(512) stream.seek(0) format = imghdr.what(None, header) if not format: return None return '.' + (format if format != 'jpeg' else 'jpg') This role takes a byte stream every bit an argument. It starts past reading 512 bytes from the stream, so resetting the stream pointer back, because afterwards when the save() part is chosen we want information technology to see the unabridged stream. The beginning 512 bytes of the paradigm data are going to be sufficient to identify the format of the image.
The imghdr.what() role can await at a file stored on disk if the first argument is the filename, or else it can look at information stored in memory if the starting time argument is None and the data is passed in the 2d statement. The FileStorage object gives united states of america a stream, so the most convenient option is to read a safe amount of data from it and laissez passer information technology equally a byte sequence in the second argument.
The return value of imghdr.what() is the detected prototype format. The function supports a diverseness of formats, amongst them the popular jpeg, png and gif. If not known image format is detected, so the return value is None. If a format is detected, the name of the format is returned. The most convenient is to return the format every bit a file extension, because the application tin can then ensure that the detected extension matches the file extension, so the validate_image() function converts the detected format into a file extension. This is as simple as adding a dot as prefix for all epitome formats except jpeg, which normally uses the .jpg extension, and then this case is treated as an exception.
Hither is the complete app.py, with all the features from the previous sections plus content validation:
import imghdr import bone from flask import Flask, render_template, request, redirect, url_for, abort from werkzeug.utils import secure_filename app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads' def validate_image(stream): header = stream.read(512) stream.seek(0) format = imghdr.what(None, header) if non format: return None return '.' + (format if format != 'jpeg' else 'jpg') @app.route('/') def index(): return render_template('index.html') @app.road('/', methods=['POST']) def upload_files(): uploaded_file = request.files['file'] filename = secure_filename(uploaded_file.filename) if filename != '': file_ext = os.path.splitext(filename)[1] if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \ file_ext != validate_image(uploaded_file.stream): abort(400) uploaded_file.save(bone.path.bring together(app.config['UPLOAD_PATH'], filename)) render redirect(url_for('alphabetize')) The merely change in the view function to comprise this last validation logic is here:
if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \ file_ext != validate_image(uploaded_file.stream): arrest(400) This expanded check first makes sure that the file extension is in the allowed listing, and and then ensures that the detected file extension from looking at the data stream is the aforementioned as the file extension.
Before you test this version of the awarding create a directory named uploads (or the path that y'all defined in the UPLOAD_PATH configuration variable, if dissimilar) so that files can be saved there.
Using Uploaded Files
Y'all now know how to handle file uploads. For some applications this is all that is needed, as the files are used for some internal process. But for a big number of applications, in particular those with social features such as avatars, the files that are uploaded past users have to be integrated with the application. Using the example of avatars, once a user uploads their avatar image, any mention of the username requires the uploaded image to appear to the side.
I separate file uploads into two large groups, depending on whether the files uploaded past users are intended for public use, or they are private to each user. The avatar images discussed several times in this article are conspicuously in the first group, as these avatars are intended to be publicly shared with other users. On the other side, an application that performs editing operations on uploaded images would probably exist in the second group, because you'd want each user to but take admission to their own images.
Consuming public uploads
When images are of a public nature, the easiest mode to make the images bachelor for use by the awarding is to put the upload directory inside the awarding's static folder. For example, an avatars subdirectory can exist created inside static, and and then avatar images can exist saved in that location using the user id as name.
Referencing these uploads stored in a subdirectory of the static binder is washed in the same way equally regular static files of the application, using the url_for() role. I previously suggested using the user id equally a filename, when saving an uploaded avatar image. This was the way the images were saved:
uploaded_file.save(os.path.join('static/avatars', current_user.get_id())) With this implementation, given a user_id, the URL for the user's avatar can be generated as follows:
url_for('static', filename='avatars/' + str(user_id)) Alternatively, the uploads can be saved to a directory outside of the static binder, and and so a new route can be added to serve them. In the example app.py application file uploads are saved to the location set in the UPLOAD_PATH configuration variable. To serve these files from that location, nosotros tin can implement the following route:
from flask import send_from_directory @app.route('/uploads/<filename>') def upload(filename): return send_from_directory(app.config['UPLOAD_PATH'], filename) Ane reward that this solution has over storing uploads inside the static folder is that here you lot tin can implement boosted restrictions earlier these files are returned, either directly with Python logic inside the torso of the function, or with decorators. For example, if you desire to just provide access to the uploads to logged in users, you can add Flask-Login'southward @login_required decorator to this road, or any other hallmark or role checking machinery that you use for your normal routes.
Let's use this implementation idea to show uploaded files in our example application. Here is a new complete version of app.py:
import imghdr import os from flask import Flask, render_template, request, redirect, url_for, abort, \ send_from_directory from werkzeug.utils import secure_filename app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads' def validate_image(stream): header = stream.read(512) # 512 bytes should be enough for a header check stream.seek(0) # reset stream pointer format = imghdr.what(None, header) if non format: return None render '.' + (format if format != 'jpeg' else 'jpg') @app.route('/') def index(): files = os.listdir(app.config['UPLOAD_PATH']) return render_template('alphabetize.html', files=files) @app.route('/', methods=['POST']) def upload_files(): uploaded_file = asking.files['file'] filename = secure_filename(uploaded_file.filename) if filename != '': file_ext = os.path.splitext(filename)[1] if file_ext non in app.config['UPLOAD_EXTENSIONS'] or \ file_ext != validate_image(uploaded_file.stream): abort(400) uploaded_file.save(bone.path.join(app.config['UPLOAD_PATH'], filename)) return redirect(url_for('index')) @app.route('/uploads/<filename>') def upload(filename): return send_from_directory(app.config['UPLOAD_PATH'], filename) In addition to the new upload() part, the index() view part gets the list of files in the upload location using bone.listdir() and sends it downward to the template for rendering. The alphabetize.html template updated to show uploads is shown below:
<!doctype html> <html> <head> <championship>File Upload</title> </caput> <torso> <h1>File Upload</h1> <form method="Postal service" action="" enctype="multipart/form-data"> <p><input type="file" proper name="file"></p> <p><input blazon="submit" value="Submit"></p> </class> <hr> {% for file in files %} <img src="{{ url_for('upload', filename=file) }}" style="width: 64px"> {% endfor %} </body> </html> With these changes, every fourth dimension yous upload an epitome, a thumbnail is added at the bottom of the page:
Consuming private uploads
When users upload private files to the application, additional checks demand to exist in place to preclude sharing files from one user with unauthorized parties. The solution for these cases crave variations of the upload() view part shown above, with additional access checks.
A mutual requirement is to only share uploaded files with their possessor. A convenient way to shop uploads when this requirement is present is to use a separate directory for each user. For example, uploads for a given user tin exist saved to the uploads/<user_id> directory, and and so the uploads() part can be modified to only serve uploads from the user'southward ain upload directory, making it impossible for one user to run across files from another. Below you lot can encounter a possible implementation of this technique, again bold Flask-Login is used:
@app.route('/uploads/<filename>') @login_required def upload(filename): return send_from_directory(bone.path.join( app.config['UPLOAD_PATH'], current_user.get_id()), filename) Showing upload progress
Up until at present we have relied on the native file upload widget provided by the web browser to initiate our file uploads. I'1000 sure we tin all agree that this widget is not very appealing. Not only that, but the lack of an upload progress display makes information technology unusable for uploads of big files, as the user receives no feedback during the entire upload process. While the scope of this article is to cover the server side, I thought it would be useful to give y'all a few ideas on how to implement a mod JavaScript-based file upload widget that displays upload progress.
The practiced news is that on the server there aren't any big changes needed, the upload machinery works in the same mode regardless of what method you use in the browser to initiate the upload. To evidence you lot an example implementation I'grand going to supplant the HTML course in alphabetize.html with 1 that is compatible with dropzone.js, a pop file upload client.
Here is a new version of templates/index.html that loads the dropzone CSS and JavaScript files from a CDN, and implements an upload class according to the dropzone documentation:
<html> <caput> <title>File Upload</title> <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/dropzone/5.seven.i/min/dropzone.min.css"> </head> <trunk> <h1>File Upload</h1> <form action="{{ url_for('upload_files') }}" class="dropzone"> </form> <script src="https://cdnjs.cloudflare.com/ajax/libs/dropzone/5.vii.ane/min/dropzone.min.js"></script> </body> </html> The one interesting thing that I've plant when implementing dropzone is that it requires the activity attribute in the <form> element to be prepare, even though normal forms take an empty action to indicate that the submission goes to the same URL.
Start the server with this new version of the template, and this is what you'll get:
That's basically it! Y'all can now drop files and they'll be uploaded to the server with a progress bar and a final indication of success or failure.
If the file upload fails, either due to the file being too large or invalid, dropzone wants to display an mistake message. Because our server is currently returning the standard Flask error pages for the 413 and 400 errors, y'all will see some HTML gibberish in the error popup. To right this we can update the server to return its error responses as text.
The 413 error for the file too big status is generated past Flask when the request payload is bigger than the size set in the configuration. To override the default error page we accept to use the app.errorhandler decorator:
@app.errorhandler(413) def too_large(e): render "File is too large", 413 The second error status is generated by the application when any of the validation checks fails. In this case the error was generated with a abort(400) call. Instead of that the response tin can be generated directly:
if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \ file_ext != validate_image(uploaded_file.stream): return "Invalid paradigm", 400 The concluding modify that I'm going to make isn't actually necessary, but information technology saves a scrap of bandwidth. For a successful upload the server returned a redirect() back to the primary route. This acquired the upload grade to be displayed again, and as well to refresh the list of upload thumbnails at the lesser of the page. None of that is necessary now considering the uploads are done as background requests by dropzone, and then we can eliminate that redirect and switch to an empty response with a code 204.
Here is the complete and updated version of app.py designed to work with dropzone.js:
import imghdr import os from flask import Flask, render_template, request, redirect, url_for, abort, \ send_from_directory from werkzeug.utils import secure_filename app = Flask(__name__) app.config['MAX_CONTENT_LENGTH'] = 2 * 1024 * 1024 app.config['UPLOAD_EXTENSIONS'] = ['.jpg', '.png', '.gif'] app.config['UPLOAD_PATH'] = 'uploads' def validate_image(stream): header = stream.read(512) stream.seek(0) format = imghdr.what(None, header) if not format: return None render '.' + (format if format != 'jpeg' else 'jpg') @app.errorhandler(413) def too_large(due east): return "File is as well large", 413 @app.road('/') def index(): files = os.listdir(app.config['UPLOAD_PATH']) render render_template('index.html', files=files) @app.route('/', methods=['POST']) def upload_files(): uploaded_file = request.files['file'] filename = secure_filename(uploaded_file.filename) if filename != '': file_ext = os.path.splitext(filename)[1] if file_ext not in app.config['UPLOAD_EXTENSIONS'] or \ file_ext != validate_image(uploaded_file.stream): return "Invalid image", 400 uploaded_file.save(bone.path.bring together(app.config['UPLOAD_PATH'], filename)) return '', 204 @app.road('/uploads/<filename>') def upload(filename): return send_from_directory(app.config['UPLOAD_PATH'], filename) Restart the application with this update and at present errors volition have a proper message:
The dropzone.js library is very flexible and has many options for customization, and then I encourage you to visit their documentation to learn how to adapt it to your needs. You tin can besides await for other JavaScript file upload libraries, every bit they all follow the HTTP standard, which means that your Flask server is going to piece of work well with all of them.
Determination
This was a long overdue topic for me, I can't believe I have never written anything on file uploads! I'd love you hear what you lot recall about this topic, and if you think there are aspects of this characteristic that I haven't covered in this commodity. Feel free to permit me know beneath in the comments!
Source: https://blog.miguelgrinberg.com/post/handling-file-uploads-with-flask
0 Response to "How to Read File Without Upload Flask"
Post a Comment