Fun with Pagination – Back and Ahead by 5

Yesterday I wanted to beef up a ListView-based page with additional navigation controls.  Specifically, I decided to add buttons to jump back 5 and ahead 5 pages in the list.  With a little bit of class extension, this was easy to do.

Source:

from django.core.paginator import Page, Paginator

class PageFive(Page):class PageFive(Page):

    ''' Include info for next 5 and previous 5 pages '''

    def has_next_five(self): 
        return self.number < self.paginator.num_pages - 5 

    def has_previous_five(self): 
        return self.number > 6

    def next_five_page_number(self): 
        return self.paginator.validate_number(self.number + 5)

    def previous_five_page_number(self): 
        return self.paginator.validate_number(self.number - 5)


class PaginatorFive(Paginator):

 ''' Uses the PageFive class to report info for next and previous 5 pages
     Set pageinator_class in ListView to use '''

    def _get_page(self, *args, **kwargs): 
        """ 
        Return an instance of a single page using the PageFive object 
        """ 

        return PageFive(*args, **kwargs)

First step was to extend the Pageclass to add additional methods.  Mimicking the existing methods for next and previous pages, these four new functions return information on the previous five and next five pages of the list.

To use my new PageFiveclass, I also extended the Paginator class.  Lucky for me, the authors have included the _get_page()method as a hook to modify the Pageclass used.  Simple to override.

Next, I have to tell ListViewto call my redefined paginator.  The paginator_classvariable is used to make that change:

class MyListView(ListView):
    model = MyModel
    paginate_by = 10
    paginator_class = PaginatorFive

Finally, I can use the new methods of the PageFiveclass in the template:

<!-- PaginatorFive navigation buttons -->
 {% if is_paginated %}
   <ul class="pagination pagination-centered">
     {% if page_obj.has_previous %}
       <li><a href="?page=1">First</a></li>
       {% if page_obj.has_previous_five %}
         <li><a href="?page={{ page_obj.previous_five_page_number }}">Back 5</a></li>
       {% endif %}
       <li><a href="?page={{ page_obj.previous_page_number }}">Prev</a></li>
    {% endif %}

    <li class="active" ><a href="?page={{page_obj.number}}">{{page_obj.number}} of{{ page_obj.paginator.num_pages }}</a></li>

    {% if page_obj.has_next %}
       <li><a href="?page={{ page_obj.next_page_number }}">Next</a></li>
       {% if page_obj.has_next_five %}
         <li><a href="?page={{ page_obj.next_five_page_number }}">Ahead 5</a></li>
       {% endif %}
       <li><a href="?page={{ page_obj.paginator.num_pages }}">Last</a></li>
     {% endif %}
   </ul>
 {% endif %}

This was a fun little exercise.

 

(Sorry for the word wrap in the code.  Copy to your editor for a better view)

Django, Gunicorn, Nginx, & HTTPS

Django, gunicorn, nginx, and https logosI can find many examples of using Django, Gunicorn, and Nginx for an application, and also ways to implement HTTPS on Nginx, but not all together.  This post will document my final configurations and explain my decisions.

First, the code:

nginx.conf:

worker_processes  auto;

events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;

    ## Rewrite http to https

    server {
        listen 80 default_server;
        listen [::]:80 default_server;
        server_name _;
        return 301 https://$host$request_uri;
    }

    ## Use https

    server {
        add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; ";

        listen              443 ssl;
        server_name         your.domain.xxx;
        ssl_certificate     <path-to-certificate>;
        ssl_certificate_key <path-to-private-key>;
        ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers         HIGH:!aNULL:!MD5;

        location = /favicon.ico { access_log off; log_not_found off; }


        location / {
            proxy_set_header Host               $host;
            proxy_set_header X-Real-IP          $remote_addr;
            proxy_set_header X-Forwarded-For    $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Host   $host:443;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-Port   443;
            proxy_set_header X-Forwarded-Proto  https;
            proxy_read_timeout 300s; 
            proxy_pass http://localhost:8000;
        }
    }
}

gunicorn.py:

##bind = ":8000"
workers = 3
timeout = 300
proc_name = "dupe"
user = "<your user>"
group = "<your group>"
raw_env = []
capture_output = True
loglevel = "debug"
errorlog = "<path-and-name-for-log-file>"

gunicorn.service:

[Unit]
Description=gunicorn daemon
After=network.target

[Service]
Environment=DJANGO_DEBUG=False
Environment=DJANGO_SECRET_KEY=<your SECRET_KEY>
Environment=DJANGO_ALLOWED_HOSTS=[*]
Environment=DJANGO_STATIC_ROOT=<application.path>/staticfiles
Environment=DJANGO_SETTINGS_MODULE=<project>.settings.devl
User=<your user>
Group=<your group>
WorkingDirectory=<application.path>
ExecStart=<virtualenv.path>/bin/gunicorn -c <application.path>/gunicorn.py  --bind 0.0.0.0:8000 <project>.wsgi:application

[Install]
WantedBy=multi-user.target

Notes:

nginx.conf holds all of the interesting stuff

The first server section grabs any http traffic coming in on port 80 and redirects it to the https URL on port 443.

Handling https traffic was trickier. We need to pass all of the header settings from the incoming request on to gunicorn/Django at localhost. All of the proxy-set attribute values in the location section came from a little bit of trial and error.

The setting for proxy-read-timeout is admittedly high – five minutes! This application has one longer running transaction that was exceeding the 60 second default. This value needs to match the timeout setting in the gunicorn settings. I’ll probably lower it to a more respectable two minutes before hitting production.

Finally, astute readers will notice that there is no code to handle requests for static resources. This is because I am using Whitenoise to route those calls through Django. This works only because my app just has eight users and Django can handle the load. Anything that is public facing should use Nginx to serve static files directly (or look into using a CDN, etc.)

Not much in gunicorn.py

I’m using 3 workers – plenty for my expected load. The timeout is set to 300 seconds, which is the same found in nginx.conf above. These should match.

The bind setting is commented and moved to gunicorn.service so that I can use this file for more than one instance of the application. A second service file can specify a unique port for its copy of gunicorn.

gunicorn.service – Instance specific settings

Since my app is stored in a git repo (on a company server, not github), I don’t want to include any security information like the SECRET_KEY in a versioned file. Therefore, this guy is not included there – although an example file is.

The Environment variables set here are read by Django at startup.

The ExecStart line starts by running the version of gunicorn found in the virtualenv for this instance. Setting up a virtualenv is beyond this scope of this post, but I hope that if you’ve gotten this far you know all about them.

Note that the bind parameter includes the port number used for this instance of gunicorn as mentioned above.

Questions? Comments?

I hope this helps someone setting up a Django app. Please drop a note in the comments with your experience. I’d also invite questions, as well as comments on how I can improve my configuration. Thanks all!
 

Django and LDAP

LDAP GraphicA project at work requires that I authenticate users using an LDAP service. Who knew it would be so easy?!?

I’m using Django 1.9.2, but I imagine that this will work with other versions +/- a few releases. Python version is 3.5.

INSTALL

The library that does the magic is django-auth-ldap. For Python 3.x, pyldap is required, while python-ldap is used for Python 2.x. I also had to install some system libraries: libsasl2-dev, python-dev, libldap2-dev, and libssl-dev (on Ubuntu).

AUTHENTICATION_BACKENDS

The AUTHENTICATION_BACKENDS setting is not defined in the default Django settings.py file using v1.9.2. However, the code I use below expects the variable to exist. Therefore, I added it in with the default setting. You may have other backends already in use.

AUTHENTICATION_BACKENDS = (
    'django.contrib.auth.backends.ModelBackend',
)

LDAP SETTINGS

This code lives at the end of settings.py. Use your own LDAP URI, ou (Organizational Unit), and dc (Domain Component).

#-----------------------------------------------------------------------------#
#
#   LDAP Settings
#
#-----------------------------------------------------------------------------#

AUTHENTICATION_BACKENDS += ('django_auth_ldap.backend.LDAPBackend',) 

AUTH_LDAP_SERVER_URI = "ldaps://your.ldap.server"

AUTH_LDAP_USER_DN_TEMPLATE = "uid=%(user)s,ou=users,dc=example,dc=com"

The first line adds an additional authentication backend to the tuple. Next, we define the path to the server. Note that I’ve used a secure protocol (ldaps). This is essential to keep the credentials passed to the server encrypted. The third line sets up a direct bind with the supplied user.

In place of the direct bind method, the documentation also suggests a user search configuration. I was able to get this to work as well, but it requires that your password be included in settings.py (or you are performing some trickery to get the password). My LDAP admin is not fond of this method, but will create an application-only account if needed. Here is the user search configuration:

import ldap
from django_auth_ldap.config import LDAPSearch

AUTH_LDAP_BIND_DN = "<user>"
AUTH_LDAP_BIND_PASSWORD = "<password>"
AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,dc=example,dc=com",
    ldap.SCOPE_SUBTREE, "(uid=%(user)s)")

I’m sorry to say that I’m not an LDAP expert at all, so I can’t supply much advice if this doesn’t work for you. Buy your LDAP admin a lunch sometime and ask your questions while waiting for the food.

Best of luck!!

Looking for OSS or Pro Bono Django Work

I’m settling in to a new job in a new town, and life is pretty good. Unfortunately, my duties don’t include any chance to program in Django. Our main software vendor is moving from Oracle Forms to Groovy/Grails, so I’m sure I’ll be learning more about that platform.

However, I would still like to keep my Django skills sharp, so I am offering my services to any non-profit group that needs an extra Django hand. No salary is expected, just a chance to get involved.

An open source project would be a great place for me to assist, and would also be a learning experience for me. Is there a project out there that needs a contributor with Django knowledge?

A charity or other non-profit would be another option. Perhaps there is an association developing OSS software for non-profits to use. With my experience supporting fundraising and higher education, I feel I could make a meaningful contribution.

If you have an opportunity fitting the description above, or know someone who does, please send me a message dan@dashdrum.com. I am ready to help!

Getting Things Done to GTD (Jabber with Google Apps)

This is a complicated process I followed to try to make things easier. It started with an article discussing how to Make Gmail Your Gateway to the Web. Basically, he is trying to make his GMail account his gateway to everything. I’ve got my Google Apps account all setup to received email from every account with filters and tags and alternate accounts. The calendars are shared with the rest of the family (if I could only get everyone else to use them).

The only thing he’s done that I haven’t is what he calls “update and track your social networks via IM”. So, I setup the ping.fm and notify.me accounts as he describes, and tried it out. It all worked pretty well except I coulnd’t get the notify.me account to validate GTalk.

A little Google research and I found that I have to add 10 SRV entries in my DNS for the domain to property route the jabber messages to Google. This Google article explains it pretty well. Next, I had to figure out how to enter this info into a Dreamhost account. I found that the correct method is to enter “_xmpp-server._tcp” in the name field and “5 0 5269 xmpp-server.l.google.com.” in the value field (be sure to include the period at the end). After a little time for the DNS to get settled, I tried the validate process again, and it worked great.

OK, so after all of that, let’s try it out.

Ping.fm works exactly as advertised. I setup micro-blog messages to go to Twitter, and status updates to both Twitter and Facebook. It all works via the chat client in GMail.

Notify.me also did what I expected. I couldn’t find a way to get my entire Twitter stream to come through, but the messages sent to me came through fine. (Still working on direct messages)

However, there is a problem. In the IM that comes in from notify.me, I can’t tell who sent the message. There isn’t any setup of the format that I can see, but it wasn’t there. I posted a suggestion message to the service.

We’ll see how long I keep this setup going.


This post originally appeared on the Linux Server Diary.