Running Django Better with Gunicorn, Supervisor, and Nginx

Running Django via runserver

I believe that those who have used Django for development are no strangers to the command python manage.py runserver. This command utilizes a web server built into Django, enabling us to easily run Django locally. It suffices for local testing but is unfit for production environments, not even for testing environments. The main issues are as follows:

1. Poor Performance
   It is single - process and single - threaded, so it can only handle one request at a time. As the number of requests increases, the CPU and memory usage of the server will keep rising, ultimately leading to performance degradation.

2. Limited Functionality
   It only supports basic HTTP requests and lacks support for features such as HTTPS, load balancing, and static file serving. In a production environment, these features are essential.

Running Django via Gunicorn

Before officially using Gunicorn, let’s first talk about its advantages:
(The following content is from GPT4)

Gunicorn (Green Unicorn) is a Python WSGI HTTP server. Many large - scale websites and high - performance applications choose it to host Python web applications. Here are some of the main advantages of Gunicorn:

1. Easy to Use: Gunicorn is very easy to install and configure. You just need to install it via pip and can start a WSGI application with just a few commands. This allows developers to focus more on their application code without having to worry too much about deployment and operation and maintenance issues.

2. Strong Compatibility: Gunicorn fully supports the WSGI specification, so theoretically, it is compatible with all Python web frameworks that follow this specification, including but not limited to Django, Flask, and Pyramid.

3. Pre - forking Model: Gunicorn uses the Unix fork system call to create child processes (workers), and each worker can handle requests independently. With the pre - forking model, the parent process can kill and rebuild child processes when necessary, providing a protection mechanism against memory leaks.

4. Load Management: Gunicorn offers various types of workers, including synchronous workers and asynchronous workers. Synchronous workers are suitable for CPU - intensive tasks, while asynchronous workers are suitable for IO - intensive or network - intensive tasks. This means you can choose the most suitable worker type according to the characteristics of your application.

5. Error Isolation: Since each request is processed by its own worker process, if a request causes a crash or an uncaught exception, it will only affect the worker handling that request, and other workers or requests will not be affected.

6. Robustness and Reliability: Gunicorn can automatically manage worker processes. If a worker exits abnormally, the Gunicorn master process will automatically restart a new worker process to replace it, ensuring the continuous availability of the service.

7. Flexible Configuration: Gunicorn provides a large number of configuration options. You can control aspects such as log levels, output locations, the number of workers, request timeout times, and SSL settings. At the same time, Gunicorn supports reading configurations from Python files, environment variables, and command - line arguments, meeting the needs of different scenarios.

8. Hot Reload: Gunicorn supports seamless restarts, meaning you can upgrade code or configurations without interrupting the service.

9. Community Support and Documentation: Last but not least, Gunicorn has an active developer community and detailed official documentation, which can provide a great deal of help and guidance to developers using Gunicorn.

Before using Gunicorn to run Django, you need to ensure that Django and Gunicorn are correctly installed. Suppose your Django project is named myproject and it is located under /path/to/myproject/.

1. Install Gunicorn
   If you haven’t installed Gunicorn yet, you can use pip to install it:

   pip install gunicorn

2. Run Gunicorn
   Now you should be able to use Gunicorn to start your Django application. The basic command format of Gunicorn is gunicorn [OPTIONS] APP_MODULE, where APP_MODULE is a Python import path pointing to the module containing the WSGI application object.

   In a standard Django project, this object (usually called application) is defined in the wsgi.py file. So if your project is named myproject, then APP_MODULE is myproject.wsgi.

   To start the Django project with Gunicorn, you can execute the following command:

   cd /path/to/myproject/
   gunicorn myproject.wsgi

   This will start your Django application on a Gunicorn server listening on localhost:8000.

   Note: This is just to demonstrate a simple way to run gunicorn. It is not recommended to use this way in a formal environment.

3. Configure Gunicorn

   Gunicorn provides many configurable options that you can adjust according to your needs. A common way is to create a Gunicorn configuration file, which can make your configuration more structured and easier to manage. The Gunicorn configuration file is usually a Python script in which some global variables are defined.

   Suppose we create the following configuration file at /path/to/myproject/gunicorn_config.py:

     
   # gunicorn_config.py
   import multiprocessing
   
   # Bind IP and port number
   bind = "0.0.0.0:8080"
   
   # Use gevent mode, sync mode can also be used, and sync mode is the default
   worker_class = 'gevent'
   
   # Number of processes to start
   workers = multiprocessing.cpu_count() * 2 + 1
   
   # Number of requests to handle concurrently
   threads = 2
   
   # Maximum number of pending connections
   backlog = 2048
   
   # Working mode coroutine
   worker_connections = 1000
   
   # Reload the program automatically after reloading or modifying the configuration
   reload = True
   
   # Access log file
   accesslog = "/var/log/gunicorn/access.log"
   
   # Error log file
   errorlog = "/var/log/gunicorn/error.log"

   In the above configuration, we set multiple parameters such as the binding address, working mode, and log location. This is just a basic configuration example, and you can modify or expand it according to your actual needs.

   Then, you can use the -c or --config command - line option to specify the path of the configuration file and run the Django application as follows:

   cd /path/to/myproject/
   gunicorn myproject.wsgi -c gunicorn_config.py

   This command tells Gunicorn to load the gunicorn_config.py file and apply the configurations defined in it.

Using Nginx as a Reverse Proxy

Gunicorn is much better than Django’s runserver. However, in practice, Gunicorn is generally not directly exposed to the outside world. Instead, an additional layer of reverse proxy is added, and the most commonly used one is Nginx.

Using Nginx as a reverse proxy has the following main advantages:

1. Static File Handling: Nginx is very good at handling static content (such as CSS, JavaScript files, or images), while Python WSGI servers are usually not suitable for directly serving static files, which may cause performance problems. By delegating the static file serving task to Nginx, you can free up Gunicorn’s resources to handle dynamic content.

2. Load Balancing: If you have multiple backend servers or multiple worker processes, Nginx can effectively distribute incoming requests to various backend servers to achieve load balancing. It also supports various load - balancing strategies and health checks.

3. Request Buffering: Nginx can provide a layer of protection for the backend because it intercepts and processes all client connections. This means that the backend server only needs to handle complete requests without having to worry about network problems or slow connections. In addition, if the backend application crashes or restarts, Nginx can still continue to serve users during this period (for example, by returning a friendly error page).

4. SSL Termination: If your website requires SSL encryption, Nginx can handle all HTTPS handshake processes and communicate with the backend server in an unencrypted manner, thus reducing the burden on the backend server.

5. HTTP/2 Support: Nginx supports the HTTP/2 protocol, while most WSGI servers, including Gunicorn, currently have no plans to directly support HTTP/2. By enabling HTTP/2 in Nginx, your users can enjoy faster loading speeds and lower latency.

6. Access Control and Security Protection: Nginx provides a series of security - related functions, such as IP whitelists/blacklists, rate - limiting, and protection against DDOS attacks.

7. gzip Compression: Nginx can compress response data using gzip, thereby reducing network bandwidth consumption and improving page loading speed.

The following are the detailed steps to use Nginx as a reverse proxy for Gunicorn:

1. Install Nginx

   On Ubuntu/Debian, you can install Nginx via apt - get:

   sudo apt-get install nginx

2. Configure Nginx

   Configure Nginx so that it can correctly forward requests to Gunicorn. The Nginx settings file is usually located at /etc/nginx/sites - available/default.

   Here is a basic configuration example:

   server {
       listen 80;
       server_name yourdomain.com;
   
       location /static/ {
           alias /path/to/myproject/static/;  # This points to the static file directory in Django
       }
   
       location / {
           proxy_set_header Host $host;
           proxy_set_header X - Real - IP $remote_addr;
           proxy_set_header X - Forwarded - For $proxy_add_x_forwarded_for;
           proxy_set_header X - Forwarded - Proto $scheme;
           
           proxy_pass http://localhost:8000; # Replace with the address and port that gunicorn is listening on
       }
   }

3. Start Nginx

   After completing the configuration, you can start Nginx:

   sudo service nginx start

The above is the basic process. Of course, this is just the simplest configuration. If you need more advanced features (such as HTTPS, load balancing, or caching), more configurations are required. It is recommended to refer to the official Nginx documentation for more detailed information.

Using Supervisor to Manage Gunicorn and Nginx

I didn’t come across Supervisor until my second job. After getting to know it, I really like this tool. Its biggest advantage is that it can manage a certain process. Especially if a process encounters problems and dies, Supervisor will automatically try to restart the process, which is very important for the online environment.

Supervisor is a process management tool written in Python. It can be easily used to start, restart (automatically), and shut down processes on UNIX - like systems (not supported on Windows).

The following are the steps to use Supervisor to manage Gunicorn and Nginx:

1. Install Supervisor

   On Ubuntu/Debian, you can install Supervisor via apt - get:

   sudo apt-get install supervisor

2. Create Supervisor Configuration Files

   You need to create a configuration file for each program to be managed by Supervisor. These files are usually located in the /etc/supervisor/conf.d/ directory and end with .conf.

   As assumed above, if the Django project path is /path/to/myproject/ and the Gunicorn configuration file is named gunicorn_config.py, then we need to create a file named myproject_gunicorn.conf for Gunicorn:

   # /etc/supervisor/conf.d/myproject_gunicorn.conf
   
   [program:myproject_gunicorn]
   command=/usr/local/bin/gunicorn myproject.wsgi:application -c /path/to/myproject/gunicorn_config.py
   directory=/path/to/myproject/
   user=yourusername
   autostart=true
   autorestart=true
   redirect_stderr=true

   Similarly, we also need to create a file named nginx.conf for Nginx:

   # /etc/supervisor/conf.d/nginx.conf
   
   [program:nginx]
   command=/usr/sbin/nginx -g "daemon off;"
   autostart=true
   autorestart=true
   redirect_stderr=true

3. Run Supervisor

   After installing Supervisor and creating the relevant configuration files, you can let Supervisor start working. First, you need to read all new or modified configuration files:

   sudo supervisorctl reread

   Then, you can update the status of the Supervisor service to start running the newly added programs:

   sudo supervisorctl update

   Or, if you want to start a specific program, such as myproject_gunicorn (that is, the above - mentioned Django), you can do this:

   sudo supervisorctl start myproject_gunicorn