Django, NGINX & Gunicorn

Posted: April 18, 2020

This post was ported from my old blog

In the previous post, I explained the setup of my CentOS VPS, here I will explain how to host a Django website, using NGINX as a webserver and Gunicorn as a reverse proxy.

Project Folder & Virtualenv

We are about to create the Django project we want to host, so from now on keep in mind that:

Every reference to mysite or mysite.com will have to be replaced with your own project name/domain.
I like to keep all my websites in ~/webapps, in separate subfolders, your preference may vary.

Project Folder

Let's create our project folder.

1cd ~/webapps 
2mkdir mysite 
3cd mysite

Virtualenv

Now we will create a Python Virtualenv, this will be necessary if you plan to serve multiple Django websites, where every website should have its own dependencies.

1python3 -m venv venv      # (This will only work with Python 3) 
2source venv/bin/activate  # Activate the virtualenv

At this point, your terminal shell should look like this

1(venv) user@Hostname:~/webapps/mysite$

That (venv) means we are now inside the Python Virtualenv, so every pip package installed will be installed inside this single environment, and not system-wide.

Installing Dependencies

For the sake of simplicity, I will be installing only 2 packages that are essential for this guide.

1pip install --upgrade pip    # First let's update pip 
2pip install django           # Install Django inside the virtualenv 
3pip install gunicorn         # Install Gunicorn (needed later)

Note for Django 3.0

If you want to use the new Django 3.0 that came out, and you use SQLite as database, you will need SQLite 3.8, but CentOS 7 ships only with SQLite 3.7.
To update it, you can install it from Fedora packages like this:

1wget https://kojipkgs.fedoraproject.org//packages/sqlite/3.8.11/1.fc21/x86_64/sqlite-3.8.11-1.fc21.x86_64.rpm 
2
3sudo yum install sqlite-3.8.11-1.fc21.x86_64.rpm

And voilá, you now have SQLite 3.8!

Exit the Virtualenv

To exit the Virtualenv, simply type this in the shell.

1deactivate

New Django Project

Alright, time to init our Django Project.
If you already have a project you want to host you can skip this step, just make sure to install all the needed dependencies inside the virtualenv.

So, let's go.

The following steps should run inside the virtualenv, so if you don't see that (venv) on your terminal go ahead and run:

1source venv/bin/activate

Now that we are inside the virtualenv, we can initialize our Django Project.

1django-admin startproject mysite . 
2# The dot at the end means we are initializing the project in the curren

After that command, you should have all the project files created.
We can now do some other commands to setup Django:

1python manage.py migrate # Makes initial migrations to the Database 
2
3# And create an Admin user: 
4python manage.py createsuperuser 
5# Then fill the required data with username, email and password

After that we are done with python, so you can exit the virtualenv (with deactivate).

Configuring Gunicorn

To host Python applications (Django) with NGINX we first need Gunicorn, a Python HTTP Server. This will serve our webapp, then NGINX will point to this web server.

First of all, we need to create the Gunicorn socket and service, and we do so by creating 2 files that will contain a Systemd process for the Python web server, and one for the socket.

Gunicorn Socket

Create and open the mysite.com.socket file under /etc/systemd/system/

1# (remember to change "mysite.com" with your site domain) 
2sudo nano /etc/systemd/system/mysite.com.socket

Now inside it paste this code:

/etc/systemd/system/mysite.com.socket

1# Gunicorn socket file 
2# - Adjust folder paths if different 
3
4[Unit] 
5Description=mysite gunicorn socket 
6
7[Socket] 
8ListenStream=/run/mysite.com.sock 
9
10[Install] 
11WantedBy=sockets.target

Gunicorn Service

This service will connect to the previously created socket.
Create and open the mysite.com.service file under /etc/systemd/system/

1# (remember to change "mysite.com" with your site domain) 
2sudo nano /etc/systemd/system/mysite.com.service

Now inside it paste this code:

/etc/systemd/system/mysite.com.service

1# Gunicorn service file 
2# - Replace "mysite" with your project name 
3# - Replace "simone" with your username 
4# - Adjust folder paths if different 
5
6[Unit] 
7Description=mysite gunicorn daemon 
8Requires=mysite.com.socket 
9After=network.target 
10
11[Service] 
12User=simone 
13Group=nginx 
14WorkingDirectory=/home/simone/webapps/mysite ExecStart=/home/simone/webapps/mysite/venv/bin/gunicorn \
15    -w 4 \	
16    --error-logfile /home/simone/webapps/mysite/gunicorn.log \	
17    --bind unix:/run/mysite.com.sock \	
18    mysite.wsgi:application 
19    
20[Install] 
21WantedBy=multi-user.target

This will be our Python web server, let me explain a couple of things about it:

What the hell is happening here?
- WorkingDirectory: It's the root folder of your project.
- ExecStart: The first path here tells Systemd to use the Gunicorn we installed inside our project's virtualenv (at the "Installing Dependencies" step).
Then -w specifies how many workers to spawn (usually (2 x $num_cores) + 1 ).
Next, the --error-logfile tells Gunicorn where to save the logfile, in this example will be inside our project folder, put it where you want.
Finally, --bind will point to the .sock file created by mysite.com.socket.

Putting aside the boring stuff, we now have the Python web server, we just need to start it, and run these 6 commands:

1sudo systemctl start mysite.com.socket 
2sudo systemctl enable mysite.com.socket 
3sudo systemctl start mysite.com.service 
4sudo systemctl enable mysite.com.service 
5sudo systemctl daemon-reload 
6sudo systemctl restart mysite.com.service

This will start the web server and enable it to autostart if your VPS/Server is rebooted.

Setup NGINX - Round 1

NGINX will be our proxy, it takes all the requests for mysite.com and tells them to go to the Gunicorn web server we set up earlier.

I called this step "Round 1" because we actually need to edit this config twice to have our nice HTTPS there:

First we setup a basic NGINX config that serves the website as HTTP
Then we request a Let's Encrypt Certificate, and it will check if our website is actually running online, that's why we need it online as HTTP.
Then, after we obtained our certificate, we can edit the NGINX config to add the "SSL" part.

I used a configuration generated with Digitalocean's NGINX configurator, that consists in some files that will contain some general settings, a main nginx config file, and a file containing your website configuration.
To make things more clear, this is the final folder structure we will have

1/etc/nginx/	
2  ┣╸conf.d/	
3  ┃	    ┗╸mysite.com.nginx.conf	
4  ┃	
5  ┣╸nginxconfig.io/	
6  ┃	    ┣╸general.conf	
7  ┃	    ┣╸letsencrypt.conf	
8  ┃	    ┣╸proxy.conf	
9  ┃	    ┗╸security.conf	
10  ┃	
11  ┣╸dhparam.pem	
12  ┗╸nginx.conf

Let's go from top to bottom, for every file you will find a codeblock with the path to that file at the bottom:

/etc/nginx/conf.d/mysite.com.nginx.conf

1server {
2	listen 80;
3	listen [::]:80;
4
5	server_name mysite.com;
6
7	include nginxconfig.io/letsencrypt.conf;
8
9	# reverse proxy
10	location / {
11		proxy_pass http://unix:/run/mysite.com.sock;
12		include nginxconfig.io/proxy.conf;
13	}
14}
15

/etc/nginx/nginxconfig.io/general.conf

1# favicon.ico
2location = /favicon.ico {
3	log_not_found off;
4	access_log off;
5}
6
7# robots.txt
8location = /robots.txt {
9	log_not_found off;
10	access_log off;
11}
12
13# assets, media
14location ~* \.(?:css(\.map)?|js(\.map)?|jpe?g|png|gif|ico|cur|heic|webp|tiff?|mp3|m4a|aac|ogg|midi?|wav|mp4|mov|webm|mpe?g|avi|ogv|flv|wmv)$ {
15	expires 7d;
16	access_log off;
17}
18
19# svg, fonts
20location ~* \.(?:svgz?|ttf|ttc|otf|eot|woff2?)$ {
21	add_header Access-Control-Allow-Origin "*";
22	expires 7d;
23	access_log off;
24}
25
26# gzip
27gzip on;
28gzip_vary on;
29gzip_proxied any;
30gzip_comp_level 6;
31gzip_types text/plain text/css text/xml application/json application/javascript application/rss+xml application/atom+xml image/svg+xml;
32
33# brotli (commented because was giving errors, but somehow brotli still works)
34# brotli on;
35# brotli_comp_level 6;
36# brotli_types text/plain text/css text/xml application/json application/javascript # application/rss+xml application/atom+xml image/svg+xml;
37

/etc/nginx/nginxconfig.io/letsencrypt.conf

1# ACME-challenge
2location ^~ /.well-known/acme-challenge/ {
3	root /etc/letsencrypt;
4}

/etc/nginx/nginxconfig.io/proxy.conf

1proxy_http_version	1.1;
2proxy_cache_bypass	$http_upgrade;
3
4proxy_set_header Upgrade			$http_upgrade;
5proxy_set_header Connection 		"upgrade";
6proxy_set_header Host				$host;
7proxy_set_header X-Real-IP			$remote_addr;
8proxy_set_header X-Forwarded-For	$proxy_add_x_forwarded_for;
9proxy_set_header X-Forwarded-Proto	$scheme;
10proxy_set_header X-Forwarded-Host	$host;
11proxy_set_header X-Forwarded-Port	$server_port;
12

/etc/nginx/nginxconfig.io/security.conf

1# security headers
2add_header X-Frame-Options "SAMEORIGIN" always;
3add_header X-XSS-Protection "1; mode=block" always;
4add_header X-Content-Type-Options "nosniff" always;
5add_header Referrer-Policy "no-referrer-when-downgrade" always;
6add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always;
7add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
8
9# . files
10location ~ /\.(?!well-known) {
11	deny all;
12}
13

1user nginx;
2pid /var/run/nginx.pid;
3worker_processes auto;
4worker_rlimit_nofile 65535;
5
6events {
7	multi_accept on;
8	worker_connections 65535;
9}
10
11http {
12	charset utf-8;
13	sendfile on;
14	tcp_nopush on;
15	tcp_nodelay on;
16	server_tokens off;
17	log_not_found off;
18	types_hash_max_size 2048;
19	client_max_body_size 16M;
20
21	# MIME
22	include mime.types;
23	default_type application/octet-stream;
24
25	# logging
26	access_log /var/log/nginx/access.log;
27	error_log /var/log/nginx/error.log warn;
28
29	# SSL
30	ssl_session_timeout 1d;
31	ssl_session_cache shared:SSL:10m;
32	ssl_session_tickets off;
33
34	# Diffie-Hellman parameter for DHE ciphersuites
35	ssl_dhparam /etc/nginx/dhparam.pem;
36
37	# Mozilla Intermediate configuration
38	ssl_protocols TLSv1.2 TLSv1.3;
39	ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
40
41	# OCSP Stapling
42	ssl_stapling on;
43	ssl_stapling_verify on;
44	resolver 1.1.1.1 1.0.0.1 8.8.8.8 8.8.4.4 208.67.222.222 208.67.220.220 valid=60s;
45	resolver_timeout 2s;
46
47	# load configs
48	include /etc/nginx/conf.d/*.conf;
49}
50

Ok we miss just one more file, the dhparam.pem, you will need to generate it with openssl (code below).
Keep in mind that this file will take some time to generate, normally 3-4 minutes, so just be patient till it's done, this will be unique for your server.

1sudo openssl dhparam -out /etc/nginx/dhparam.pem 2048

Once done, we have everything we need, so just to start (and enable) NGINX:

1sudo systemctl start nginx 
2sudo systemctl enable nginx

Remember that after every edit you make to the NGINX config file, you need to reload NGINX for it to see the updated content, you can do it like this:

1# Checks if the new NGINX config has errors 
2sudo nginx -t 
3
4# Reload NGINX 
5sudo systemctl reload nginx

Let's Encrypt for SSL (HTTPS)

Time to get that green lock!
To get an SSL Certificate, we first install certbot, a utility made to request free Let's Encrypt Certificates.

1sudo dnf install certbot python3-certbot-nginx

Before requesting a certificate, make sure your website is accessible from the internet! Certbot needs to view it to verify it's an existing website!

Then, to request a certificate:

1# For more information about this command, go here: 
2# https://certbot.eff.org/lets-encrypt/centosrhel7-nginx 
3
4sudo certbot certonly --nginx

This will generate some files under /etc/letsencrypt/live/mysite.com, we will need them later for the final NGINX config.

SSL Certificates, obviously, have an expiry. But fear not! We can renew it automatically as much as we want, and that's done by using crontab with python3-certbot-nginx (installed previously with certbot).

You use it like this:

1sudo echo "0 0,12 * * * root python -c 'import random; import time; time.sleep(random.random() * 3600)' && certbot renew -q" | sudo tee -a /etc/crontab > /dev/null

With this, crontab will try to renew every certificate we have, every day, all on its own, no action required!

Setup NGINX - Round 2

Almost there!
Now that we have our SSL Certificate, this will be the final NGINX config file.

/etc/nginx/conf.d/mysite.com.nginx.conf

1# In this file, make sure to replace every path with your own paths.
2# For example, every /home/simone with /home/yourusername
3# and mysite.com with your own website name
4
5server {
6	listen 443 ssl http2;
7	listen [::]:443 ssl http2;
8
9	server_name mysite.com www.mysite.com;
10	root /home/simone/webapps/mysite/public;
11
12	# SSL
13	ssl_certificate /etc/letsencrypt/live/mysite.com/fullchain.pem;
14	ssl_certificate_key /etc/letsencrypt/live/mysite.com/privkey.pem;
15	ssl_trusted_certificate /etc/letsencrypt/live/mysite.com/chain.pem;
16
17	# security
18	include nginxconfig.io/security.conf;
19
20	# logging
21	access_log /var/log/nginx/mysite.com.access.log;
22	error_log /var/log/nginx/mysite.com.error.log warn;
23
24	# reverse proxy
25	location / {
26		proxy_pass http://unix:/run/mysite.com.sock;
27		include nginxconfig.io/proxy.conf;
28	}
29
30	# additional config
31	include nginxconfig.io/general.conf;
32}
33
34# HTTP redirect
35server {
36	listen 80;
37	listen [::]:80;
38
39	server_name mysite.com www.mysite.com;
40
41	include nginxconfig.io/letsencrypt.conf;
42
43	location / {
44		return 301 https://www.mysite.com$request_uri;
45	}
46}
47

That's a lot of stuff, I know.

For now, reload NGINX and you should have your website accessible over HTTPS!

1sudo nginx -t 
2sudo systemctl reload nginx

Errors management

"Something's wrong, I can feel it!"

I, for sure, know the pain when you do all this stuff and nothing works, so let me give you a quick list of things to check if "something's wrong":

If you kept all the logs paths I wrote in this guide, you have 2 error logs files you can look for:
NGINX error log is under /var/log/nginx/mysite.com.error.log, then Gunicorn error log is inside your project folder, so /home/yourusername/webapps/mysite/gunicorn.log.
Another thing that can go wrong is system services, the ones we started with systemctl. To see if one of them is running (and check if there are errors) just type sudo systectl status mysite.com.service or sudo systectl status mysite.com.socket (obviously "mysite" will be whatever you called it).
Still something not working? Maybe you have problems with permissions, so check you didn't miss one of the first steps where we did chmod 710 /home/yourusername, or maybe in the last guide, you didn't add your user to the nginx group.

Wrapping things up

I'll try to write a super short list of "key points" of this guide, to wrap things up.

Create a Python Virtualenv that contains all our python dependencies.
Initialize a new Django project.
Configure and enable a Gunicorn Systemd service and socket, this is the Python Web Server.
Create a basic NGINX config (with other needed configs) to serve our website as HTTP. This will forward requests to our Gunicorn web server.
Generate a Let's Encrypt Certificate for our website.
Complete our NGINX config with the newly obtained SSL Certificate.

🚀

Dotfiles repository with submodule

March 26, 2023

CentOS Server Setup

April 16, 2020