Looking for OSS or Pro Bono Django Work

I’m settling in to a new job in a new town, and life is pretty good. Unfortunately, my duties don’t include any chance to program in Django. Our main software vendor is moving from Oracle Forms to Groovy/Grails, so I’m sure I’ll be learning more about that platform.

However, I would still like to keep my Django skills sharp, so I am offering my services to any non-profit group that needs an extra Django hand. No salary is expected, just a chance to get involved.

An open source project would be a great place for me to assist, and would also be a learning experience for me. Is there a project out there that needs a contributor with Django knowledge?

A charity or other non-profit would be another option. Perhaps there is an association developing OSS software for non-profits to use. With my experience supporting fundraising and higher education, I feel I could make a meaningful contribution.

If you have an opportunity fitting the description above, or know someone who does, please send me a message dan@dashdrum.com. I am ready to help!

CAS with Django 1.5

I’ve posted a couple of times before about using a Central Authentication Service with Django. As I update my applications to Django v1.5, it is time to revisit the subject.

In my Django v1.3 setup, the django-cas library has served me well. However, in this later version of the framework, there are changes that break the library. The maintainer of django-cas has not yet updated their code to work with 1.5 (although there are pull requests waiting that address the issue – stay tuned).

Kansas State WildcatLuckily, another educational institution has forked the library. Our friends at Kansas State University (Go Wildcats!) have released their maintained version of django-cas on github. They have fixed the issue with the get_host() call, plus added some proxy features. Note that the fix is in a branch and not yet in a released version – currently at v0.8.5. I have installed it in my first app to be converted, and it works fine.

Installation

As far as I can tell, this K-State fork is not hosted on PyPi at this time, so I use this line in my requirements file to get the library via pip:

svn+https://github.com/kstateome/django-cas/branches/bugfix/django_1_5

This uses Subversion because that is what is available on the servers, but the git syntax will work just as well.

Configuration

K-State has provided only minimal documentation, but they link back to the original CCPC installation guide. Below are my settings.py and urls.py code. (Note that USE_CAS is a variable I define that is set True or False based on the environment.)

## django_cas settings
if USE_CAS:
    CAS_SERVER_URL = '### your server URL here ###'
    CAS_VERSION = '2'
    CAS_LOGOUT_COMPLETELY = True

    AUTHENTICATION_BACKENDS = (
        'django.contrib.auth.backends.ModelBackend',
        'cas.backends.CASBackend',
    )

    MIDDLEWARE_CLASSES += (
        'cas.middleware.CASMiddleware',
    )
## end django_cas settings

# django_cas urls
if settings.USE_CAS:
    urlpatterns = patterns('',
    ('^admin/logout/$', RedirectView.as_view(url='../../accounts/logout')), ## TODO: This could be better
    ('^admin/password_change/$',RedirectView.as_view(url='### URL to your password change service ###')),
    url(r'^accounts/login/$', 'cas.views.login',name='login'),
    url(r'^accounts/logout/$', 'cas.views.logout',name='logout'),
                       ) + urlpatterns
else:
    urlpatterns = patterns('',
    url(r'^accounts/login/$', 'django.contrib.auth.views.login',{'template_name': 'login.html'},name='login'),
                       ) + urlpatterns       
# end django_cas urls

Biggest change is that the package name changed from django-cas to cas.

Best of luck with your CAS setup!

Factory Boy and Django: Software Updates

Turn of the Century boy in a factory

I’ve been using Factory Boy in all of my tests for a while, but I hadn’t upgraded the package since setting up my development machine in December of last year. This meant I have been happily chugging along with v1.20.

Today I started the task of upgrading my applications to Django v1.5, so I ran pip install --upgrade on my requirements file. Django moved up to v1.5 just fine, but Factory Boy also jumped up to v2.11. I hadn’t noticed that some new features and several deprecations were added in v1.3, and many changes happened for v2.0.

DjangoModelFactory

The first odd behavior I noticed was that the model instances created by the factories weren’t being saved, and I also had trouble adding data to ManyToMany relationships even after I explicitly saved them. Turns out there was a major change to the Factory class. Rather than use that generic class, we are now being directed to use DjangoModelFactory.

The error I was seeing was a recursion problem, so it took a while to track down.

Sequence

Next, all of my Sequence calls were failing – telling me that I couldn’t use a string operation on a type of ‘int’. In this case the problem was that the default data type for a sequence was changed to ‘int’. Since ‘str’ was the former default (although not necessarily documented), my solution was to declare the type as ‘str’.

name = factory.Sequence(lambda n: 'Location ' + n, str)

I’m a big fan of this package for tests, and I am glad that it is a very active project. Read through the change log for more info on these and many other updates.

I promise to do a better job of keeping up with the changes.

Mass Assignment Vulnerability in Django

I was reviewing the Django 1.6 Alpha Release Notes and came across a mention in the deprecated features section on a possible mass assignment vulnerability.

Previously, if you wanted a ModelForm to use all fields on the model, you could simply omit the Meta.fields attribute, and all fields would be used.

This can lead to security problems where fields are added to the model and, unintentionally, automatically become editable by end users. In some cases, particular with boolean fields, it is possible for this problem to be completely invisible. This is a form of Mass assignment vulnerability.

For this reason, this behaviour is deprecated, and using the Meta.exclude option is strongly discouraged. Instead, all fields that are intended for inclusion in the form should be listed explicitly in the fields attribute.

I hadn’t paid attention to the mass assignment kerfuffle in the Rails community last year, so I wasn’t familiar with this issue. A little reading convinced me of the problem and how I can protect myself.

Basically, this vulnerability arises when database fields that are intended to be updated only through special processes are open to general update processes. An example would be an expedite_order field on an Order model. We don’t want our customers checking this box on every order, or the shipping department would be overwhelmed with rush work.

There are several ways in Django to avoid this problem. Let’s discuss them:

  1. Don’t include this special field in the template – I think we all know that it would be pretty easy for a template designer to use the as_p() method of the form to generate a quick template without knowing about the desired limited access for this field. It happens. One also has to worry about the malicious user that crafts a POST response that includes the field in question.

  2. Use the exclude attribute in the Meta class of the form – OK, this is better – our field is successfully hidden away. However, the vulnerability can return when the underlying model is updated to include a new special field. Unless the developer also updates the exclude lists for all forms related to this model, the new field is now accessible.

  3. Use the fields attribute in the Meta class of the form – Now we’re getting somewhere! Only those field explicitly listed for inclusion in the form will be accessible. When a new field is added, it will not be available to the form unless the developer updates the list.

The Django team recommends that option 3 be the solution of choice, but will accept option 2 as well. Future versions of Django will require either the fields or exclude attribute in ModelForms.

The same vulnerability exists in class-based views that use the ModelFormMixin, including CreateView and UpdateView. When these classes are used, the developer must include either a fields or exclude attribute when declaring a model, or use a ModelForm that does.

Thanks to the Django developers for continuing to make Django one of the safest frameworks around!

Custom ErrorList in a Form

I was playing around with a custom ErrorList the other day. The idea is to be able modify the way errors display in a template. The Django documentation gives an example of this technique.

(One thing I noticed while overriding methods in ErrorList, when I only defined one of the existing display methods, such as as_ul, the code was ignored. Only when I also included my own __unicode__ method pointing to my new code did it work correctly. I’m using Django 1.3 right now, so you may find things better in a future version.)

My challenge was to use my custom ErrorList with a form in a class-based views environment. There are several ways to accomplish this.

Handle It in the View

In the view, one can override the get_form_kwargs method to add the error_class attribute to the kwargs dictionary.

def get_form_kwargs(self):

    kwargs = super(MyViewName,self).get_form_kwargs()

    kwargs.update({'error_class': MyCustomErrorList})

    return kwargs

Since I’m a big mixin fan, I developed this little gem that performs this task:

class CustomErrorClassViewMixin(object):

    ''' Will set the error_class attribute
        on the form with the provided value 
        found in the error_class variable  '''

    error_class = None

    def get_form_kwargs(self):
        # Make sure that the error_class attribute is set on the
        # view, or raise a configuration error.
        if self.error_class is None:
            raise ImproperlyConfigured("'CustomErrorClassViewMixin' requires "
                "'error_class' attribute to be set.")

        kwargs = super(CustomErrorClassViewMixin,self).get_form_kwargs()

        kwargs.update({'error_class': self.error_class})

        return kwargs

Include this mixin in your view class definition (class MyView(CustomErrorClassViewMixin, CreateView)) and set a value for the error_class variable, and it will handle the rest.

Form Level Fun

In some situations, it may be desired to use the custom ErrorList on every use of the form. To do this, a little work in the form’s __init__ method is required.

def __init__(self, *args, **kwargs):
    ## Set the error_class attribute
    kwargs['error_class'] = MyCustomErrorList

    ## Call the parent method
    super(MyForm, self).__init__(*args, **kwargs)

Here’s a mixin for this scenario (I love the mixins!):

class CustomErrorClassFormMixin(object):
''' Allows the declaration of a custom error_class for a form

    Requires the error_class attribute to be provided 
'''

    error_class = None ## Default to none

    def __init__(self, *args, **kwargs):
        # Make sure that the error_class attribute is set on the
        # form, or raise a configuration error.
        if self.error_class is None:
            raise ImproperlyConfigured("'CustomErrorClassFormMixin' requires "
                "'error_class' attribute to be set.")

        ## Set the error_class attribute
        kwargs['error_class'] = self.error_class

        ## Call the parent method
        super(CustomErrorClassFormMixin, self).__init__(*args, **kwargs)

A couple of notes for this one:

  • The mixin must be declared before the form class in order to update the error_class in kwargs before the form’s __init__() method fires
  • The error_class attribute must be defined

Best Looking Errors in Town

Now, your users can make gorgeous errors. Let me know how this works for you.

Django EmptyChoiceField

On my current project, I was faced with an interesting problem. My form includes a ChoiceField that is not required. However, the Django ChoiceField doesn’t allow for no choice. What was happening instead was a de facto default to the first option in the choice list unless the user picked another value – no way to select nothing.

A quick search turned up the EmptyChoiceField from Git user davidbgk – https://gist.github.com/davidbgk/651080, and it worked very well. In my form, I used the field type thusly:

suitability = EmptyChoiceField(required=False,empty_label=u"---------",choices=Suitability)

In David’s code, the empty label choice is prepended to the list when the field is not required and the empty_label parameter has a value. However, I wanted to see if I could make the EmptyChoiceField work more like the ModelChoiceField.

  1. The empty_label parameter defaults to u"---------
  2. A required field has the empty field choice prepended to the list, unless an initial value is provided.
  3. When the field is not required, the empty field choice is always included, regardless of whether an initial value exists.

Here is my version of the field – https://gist.github.com/dashdrum/4960474:

from django.forms import ChoiceField

''' Based on https://gist.github.com/davidbgk/651080 
    modified to mirror the functionality of ModelChoiceField '''

class EmptyChoiceField(ChoiceField):
    def __init__(self, choices=(), empty_label=u"---------", required=True, widget=None, label=None,
                       initial=None, help_text=None, *args, **kwargs):

        # prepend an empty label unless the field is required AND
        # an initial value is supplied

        if required and (initial is not None):
            pass # don't prepend the empty label
        else:
            choices = tuple([(u'', empty_label)] + list(choices))

        super(EmptyChoiceField, self).__init__(choices=choices, required=required, widget=widget,
                                               label=label, initial=initial, help_text=help_text,  
                                               *args, **kwargs)

My invocation of the field only changes in that I no longer need to specify a label text:

suitability = EmptyChoiceField(required=False,choices=Suitability)

By changing the default value of the empty_label, I have broken backward compatibility, although I have a tough time seeing a reason to use this field with the empty_label set to None on purpose. I’ll probably send a pull request back to the original author, but I will note that he may not want to merge it.

Hope this post helps someone else faced with a similar dilemma.

More RelatedFieldWidgetWrapper – My Very Own Popup

I posted back in July on how to use the RelatedFieldWidgetWrapper, but it has only been in the last few weeks that I have actually used it in a project outside of the admin.

When setting up the RelatedFieldWidgetWrapper (RFWW) around a FilteredMultipeSelect widget on my own form, the code changes slightly from what is used on a admin page – basically the pointer to the admin site is no longer needed, nor is the relationship information. Here is the code used for a custom page.

First, I subclassed the RFWW to work outside of the admin, basically stripping out much of the init() code that wasn’t needed. (I’ll leave an examination of how this differs from the original to the reader).

widgets.py

class CustomRelatedFieldWidgetWrapper(RelatedFieldWidgetWrapper):

    """
        Based on RelatedFieldWidgetWrapper, this does the same thing
        outside of the admin interface

        the parameters for a relation and the admin site are replaced
        by a url for the add operation
    """

    def __init__(self, widget, add_url,permission=True):
        self.is_hidden = widget.is_hidden
        self.needs_multipart_form = widget.needs_multipart_form
        self.attrs = widget.attrs
        self.choices = widget.choices
        self.widget = widget
        self.add_url = add_url
        self.permission = permission

    def render(self, name, value, *args, **kwargs):
        self.widget.choices = self.choices
        output = [self.widget.render(name, value, *args, **kwargs)]
        if self.permission:
            output.append(u'<a href="%s" class="add-another" id="add_id_%s" onclick="return showAddAnotherPopup(this);"> ' % \
                (self.add_url, name))
            output.append(u'<img src="%simg/admin/icon_addlink.gif" width="10" height="10" alt="%s"/></a>' % (settings.ADMIN_MEDIA_PREFIX, _('Add Another')))
        return mark_safe(u''.join(output))

Next, I used this new widget wrapper in my form definition:

forms.py

class EndowmentForm(ModelForm):
    support_accounts = ModelMultipleChoiceField(queryset=None,
                                                label=('Select Support Accounts'),
                                                required=False)

    def __init__(self, *args, **kwargs):
        super(EndowmentForm,self).__init__(*args, **kwargs)
        # set the widget with wrapper
        self.fields['support_accounts'].widget = CustomRelatedFieldWidgetWrapper(
                                                FilteredSelectMultiple(('Support Accounts'),False,),
                                                reverse('support_create'),
                                                True)
        self.fields['support_accounts'].queryset = SupportAccount.objects.all() 

    class Media:
        ## media for the FilteredSelectMultiple widget
        css = {
            'all':(ADMIN_MEDIA_PREFIX + 'css/widgets.css',),
        }
        # jsi18n is required by the widget
        js = ( ADMIN_MEDIA_PREFIX + 'js/admin/RelatedObjectLookups.js',)

    class Meta:
        model = Endowment  

And include the javascript files in the template:

endow_edit.html

(somewhere in the <head> section)

{{ form.media }}
<script type="text/javascript" src="{% url admin:jsi18n %}"></script>

Also be sure to include javascript.

The parent side is ready. Now let’s tackle the child side.

To add a new related record, the widget will open the page in a popup window. (Try this in the admin to see how it should work.) When the save button is pressed, the record is added, the popup closes, and the new value is added to the chosen side of the FilteredSelectMultiple control. Pretty cool!

But … how can I make my form do that?

After digging around a little in the Django admin code, I have figured it out.

Luckily, all of the heavy lifting is performed by the javascript provided with Django. I just needed to add a little code to make it all work.

First, I added a key in the context to let me know if the form is a popup. Notice that the RelatedFieldWidgetWrapper appends ‘?=_popup=1’ to the URL. My code looks for that key in the GET data and adds the context variable if found:

views.py

SupportAccountCreateView(CreateView)

def get_context_data(self, **kwargs):
    context = super(SupportAccountCreateView,self).get_context_data(**kwargs)
    if ('_popup' in self.request.GET):
        context['popup'] = self.request.GET['_popup'] 
    return context

Next, in the template, the presence of this context key triggers the addition of a hidden field:

support_edit.html

(somewhere in the <form>)
{% if popup %}<input type="hidden" name="_popup" value="1">{% endif %}

Now that I know I’m dealing with a popup form, my post() code can look for it and fire off the javascript:

views.py

SupportAccountCreateView(CreateView)

def post(self, request, *args, **kwargs):
    ## Save the normal response
    response = super(SupportAccountCreateView,self).post(request, *args, **kwargs)
    ## This will fire the script to close the popup and update the list
    if "_popup" in request.POST:
        return HttpResponse('<script type="text/javascript">opener.dismissAddAnotherPopup(window, "%s", "%s");</script>' % \
            (escape(self.object.pk), escapejs(self.object)))
    ## No popup, so return the normal response
    return response

(This little snippet was pretty much copied from the admin code.)

Finally, I need to include the javascript in the form:

forms.py

SupportAccountEditForm(ModelForm)

class Media:
    ## media for the Related Field Wrapper
    # jsi18n is required by the widget
    js = ( ADMIN_MEDIA_PREFIX + 'js/admin/RelatedObjectLookups.js',)

support_edit.html

(somewhere in the <head> section)
{{ form.media }}
<script type="text/javascript" src="{% url admin:jsi18n %}"></script>

Be sure to also include jquery in the template.

That’s it! Give it a try sometime.

Setup Django Environment on a New Mac

MacBook Pro

Setting up a Mac for Django development was easy – even a little easier than I thought. My machine is a new MacBook Pro running 10.8.2.

Xcode

Xcode provides the utilities needed for Python libraries and Subversion. It can be installed from the App Store. Afterwards, open the application, open Preferences and select downloads, then install the command line tools.

Python Utilities

In a terminal window:

  1. sudo easy_install pip
  2. sudo easy_install virtualenv

Eclipse

Setting up Eclipse worked pretty much the same as it does on Linux.

  1. Download desired Eclipse package. I used the Java EE package, Indigo (3.7) version.
  2. Unzip the files
  3. Copy Eclipse folder to Applications
  4. Drag the program file to the dock

As I cover in my earlier post, My Eclipse Setup for Django, I installed Aptana Studio 3 to add Python and Django support, and Subclipse to access my Subversion repositories.

That’s It!

Not tough at all. Happy developing!

Unit Testing Forms in Django

This is a subject that doesn’t seem to be covered well by other sources, so I am recording my notes on testing a Django form. Credit to “fRui Apps” for this post that gives an example. (See the section Testing Forms)

My goal is to get the test as close to the code as possible. Rather than using the view test, or even Selenium, this approach will isolate the form’s functions without going through all of Django’s URL routing, middleware, etc. Save that for the functional tests.

In this first example, my form will pad the entered number with left hand zeroes. This is the quick test:

def test_id_format(self):  
    ## Use factoryboy to create a model instance  
    e = EndowmentFactory.create()
    ## Instantiate form  
    form = EndowmentEntityForm(data={'endowment': e.pk,  
                                     'id_number': '528989',  
                                     'entity_type': EntityType.ORIGINATOR,})
    ## call is_valid() to create cleaned_data  
    self.assertTrue(form.is_valid())  
    ## Test away!  
    self.assertEqual(form.clean_id_number(),'0000528989')

Note that I’m using factoryboy to create a model instance to work with. The form data includes an id_number of fewer than 10 characters. After I call is_valid() to populate cleaned_data, I can test the cleaned id_number for the proper format.

In this next example, I am testing for valid and invalid data. My application allows changing the status of an object, but certain rules apply. Here is a test confirming that a valid change is allowed to continue:

##  Test for valid action types
def test_action_type_valid(self):
    ## Use factoryboy to create a model instance
    e = EndowmentFactory.create(status='E')        
    ## Instantiate form
    form = UserActionForm(data={'endowment': e.pk,
                                'username': 'dgentry',
                                'action_type': 'A',})
    ## is_valid() should return True if no errors are found
    self.assertTrue(form.is_valid())

And this will test for an invalid status change:

##  Test for valid action types
def test_action_type_valid(self):
    ## Use factoryboy to create a model instance
    e = EndowmentFactory.create(status='A')        
    ## Instantiate form
    form = UserActionForm(data={'endowment': e.pk,
                                'username': 'dgentry',
                                'action_type': 'E',})
    ## We run is_valid() and expect False
    self.failIf(form.is_valid())
    ## check for the error
    self.assertIn(u'Invalid Action',form.errors['__all__'])

I’m sure that I will learn more about testing forms as I go. Stay tuned…