Django, Gunicorn, Nginx, & HTTPS

Django, gunicorn, nginx, and https logosI can find many examples of using Django, Gunicorn, and Nginx for an application, and also ways to implement HTTPS on Nginx, but not all together.  This post will document my final configurations and explain my decisions.

First, the code:

nginx.conf:

worker_processes  auto;

events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;

    ## Rewrite http to https

    server {
        listen 80 default_server;
        listen [::]:80 default_server;
        server_name _;
        return 301 https://$host$request_uri;
    }

    ## Use https

    server {
        add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; ";

        listen              443 ssl;
        server_name         your.domain.xxx;
        ssl_certificate     <path-to-certificate>;
        ssl_certificate_key <path-to-private-key>;
        ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers         HIGH:!aNULL:!MD5;

        location = /favicon.ico { access_log off; log_not_found off; }


        location / {
            proxy_set_header Host               $host;
            proxy_set_header X-Real-IP          $remote_addr;
            proxy_set_header X-Forwarded-For    $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Host   $host:443;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-Port   443;
            proxy_set_header X-Forwarded-Proto  https;
            proxy_read_timeout 300s; 
            proxy_pass http://localhost:8000;
        }
    }
}

gunicorn.py:

##bind = ":8000"
workers = 3
timeout = 300
proc_name = "dupe"
user = "<your user>"
group = "<your group>"
raw_env = []
capture_output = True
loglevel = "debug"
errorlog = "<path-and-name-for-log-file>"

gunicorn.service:

[Unit]
Description=gunicorn daemon
After=network.target

[Service]
Environment=DJANGO_DEBUG=False
Environment=DJANGO_SECRET_KEY=<your SECRET_KEY>
Environment=DJANGO_ALLOWED_HOSTS=[*]
Environment=DJANGO_STATIC_ROOT=<application.path>/staticfiles
Environment=DJANGO_SETTINGS_MODULE=<project>.settings.devl
User=<your user>
Group=<your group>
WorkingDirectory=<application.path>
ExecStart=<virtualenv.path>/bin/gunicorn -c <application.path>/gunicorn.py  --bind 0.0.0.0:8000 <project>.wsgi:application

[Install]
WantedBy=multi-user.target

Notes:

nginx.conf holds all of the interesting stuff

The first server section grabs any http traffic coming in on port 80 and redirects it to the https URL on port 443.

Handling https traffic was trickier. We need to pass all of the header settings from the incoming request on to gunicorn/Django at localhost. All of the proxy-set attribute values in the location section came from a little bit of trial and error.

The setting for proxy-read-timeout is admittedly high – five minutes! This application has one longer running transaction that was exceeding the 60 second default. This value needs to match the timeout setting in the gunicorn settings. I’ll probably lower it to a more respectable two minutes before hitting production.

Finally, astute readers will notice that there is no code to handle requests for static resources. This is because I am using Whitenoise to route those calls through Django. This works only because my app just has eight users and Django can handle the load. Anything that is public facing should use Nginx to serve static files directly (or look into using a CDN, etc.)

Not much in gunicorn.py

I’m using 3 workers – plenty for my expected load. The timeout is set to 300 seconds, which is the same found in nginx.conf above. These should match.

The bind setting is commented and moved to gunicorn.service so that I can use this file for more than one instance of the application. A second service file can specify a unique port for its copy of gunicorn.

gunicorn.service – Instance specific settings

Since my app is stored in a git repo (on a company server, not github), I don’t want to include any security information like the SECRET_KEY in a versioned file. Therefore, this guy is not included there – although an example file is.

The Environment variables set here are read by Django at startup.

The ExecStart line starts by running the version of gunicorn found in the virtualenv for this instance. Setting up a virtualenv is beyond this scope of this post, but I hope that if you’ve gotten this far you know all about them.

Note that the bind parameter includes the port number used for this instance of gunicorn as mentioned above.

Questions? Comments?

I hope this helps someone setting up a Django app. Please drop a note in the comments with your experience. I’d also invite questions, as well as comments on how I can improve my configuration. Thanks all!
 

Installing Oracle Client on Ubuntu 11.10

Oracle Logo(Another post written for personal documentation)

Get Software

Download these three packages from Oracle for the proper operating system (32 bit for me):

  • Instant Client Basic-Lite
  • Instant Client SDK
  • Instand Client SQLPlus

Unzip and copy to /opt/oracle/11_2/instantclient

Set LD_LIBRARY_PATH

Create /etc/ld.so.conf.d/oracle_instantclient.conf:

#Oracle client LD_LIBRARY_PATH setting
/opt/oracle/11_2/instantclient

Update cache:

sudo ldconfig -v

Symbolic Link to Library

ln -s libclntsh.so.10.1 libclntsh.so

Set ORACLE_HOME

export ORACLE_HOME=/opt/oracle/11_2/instantclient

Install Python Library

The library is called cx-oracle. Use your favorite installation method. (Don’t forget about your virtual environment!)

That’s It

Hope I remembered everything.

UPDATE

Found another post that outlines the procedure maybe a little better than I did, and includes notes on setting up tnsnames.ora. See Install Oracle Instant Client 11.1 and cx_Oracle 4.4 on Ubuntu 8.04 on Technoblog.

Changing Hosts in Dreamhost

Dreamhost logoLast night, Dreamhost moved my account to a new host. Not a big deal, and it’s certainly understandable, but it did break my one running Django app – the Gentryart Gallery. Here’s what I did to fix it.

First, as suggested in the Dreamhost Server Moves Page, I recompiled Python to create a 64bit version. While I was at it, I upgraded from v2.5.2 to v2.6.5. This great page by Ryan Kanno outlined the steps. I just changed the version number. I also deleted my old installation of Python so there wouldn’t be any chance of confusion there.

Now, if I would have used the same python version and kept everything in the same locations, I would have been done. Since I didn’t, I had a little more to do.

Next, I needed to install all of the python packages I use. Here’s the list:

Following the famous Jeff Croft post, I updated the admin_media shortcut to point to the new Django files.

And the gallery is back in business!

Reporting in Django

The Django application I’m developing has a reporting need: paper (PDF) output of reports, both 8 1/2 X 11 and special forms (labels, name tags, etc.). So, I’ve begun evaluating my reporting options.

Unfortunately, I haven’t found very many methods to examine, probably because of my requirements:

Python based
Since I’m using Python/Django for the rest of the work, it would make sense to use Python for this component as well. Other folks that may someday support the application will appreciate having a single skill set to maintain.
Contained in the Application
I’ve found at least one system that runs as a separate service on a server, but that is more complexity than I would like to introduce into a client’s technology stack. This application will appeal to small offices that likely don’t have strong tech support. Adding a separate service (in this example, Java/Tomcat) on top of Django, MySQL and Apache is more than many can do, and I certainly don’t want to get into the server admin business on their behalf.
Open Source
Another issue that the clients won’t want to deal with is licensing to third parties. I’m using all open source tools to build and run the application, and it wouldn’t make sense to require a potential customer to license something before they could use it. Plus, would I have to become a reseller for the licensed software?
Integrated with Django
This is total ‘wish list’ stuff, but wouldn’t it be neat to have an interface similar to the Django templating system where I could bring in data from a view into a report ‘template’?

Looking through Google searches, I’ve found that the terms “reports” and “reporting” can mean different things to different people. Since I’m looking to generate paper, I have been looking at Geraldo, which uses the ReportLab library to generate PDF output. This application looks promising, but is still a work in progress. I’m finding that flexible formatting with variable length text is a difficult, and maybe impossible, thing to do.

I’d like to hear from others on this subject. Have you found a good package for reporting? Are you developing one? Please leave your thoughts in the comments.

Accessing the Value of a Form Field in a Template

(This is a ‘Note to Self’ post)

I’ve been trying for half the day to get radio buttons in a custom template to work. The buttons are generated in a for loop that builds a table row for each choice (iterating over a queryset called cls), including this code for the row’s radio button:

input type="radio" name="reg_choice_1" value="{{ cls.id }}"
 {% ifequal srform.reg_choice_1 cls.id %}CHECKED{% endifequal %}

The goal is to have the ifequal test tell me if the row’s cls.id matches the value set in the form. However, I didn’t know how to access that value. After many Google searches, perusal of the Django documentation, and even a failed attempt to follow the template rendering code, I was stumped. As a wild guess, I tried the .data attribute, and it worked! Here’s the updated (and functional) code:

input type="radio" name="reg_choice_1" value="{{ cls.id }}" 
{% ifequal srform.reg_choice_1.data cls.id %}CHECKED{% endifequal %}

I hope that I’ve put enough keywords in this post so that future stumped Djangonauts can find some help.

As always, leave your comments below.