Getting Data From a Bound but Unvalidated Form

I have a form with dynamic fields, meaning the answers available for field B depend on the answer for field A.  Therefore, when the form is submitted I get the value from field B, but when it’s redisplayed I have to read the value from field A and populate the choices for field B or else field B is blanked out in the browser.

I looked for a convenient way to read the bound data from a specified form field and was surprised that I couldn’t find one.  I could just read the POST, but this form is in a formset with a prefix, so I’d have to assemble a complicated key name like “foo_bar_form-1-fieldname”.  The closest I saw to an interface to get the data was the _raw_value(fieldname) method which I couldn’t get to work and is suboptimal since it’s an internal call and could change in the future.

I ended up having to go to the POST data anyway, but I saw a helper function in _raw_value(fieldname) that made it easier.  What I ended up doing was this:

field_value = self.request.POST[form.add_prefix('fieldname')]

add_prefix(fieldname) is a public method to take care of the prefix hassle for me, so reading the post value is doable.  The only thing to watch out for is that you have to handle potential invalid values when redisplaying a form with errors.

Get The Display Name Of A ModelChoiceField’s Selected Value in Django

I just had a formset where each form had a pre-loaded ModelChoiceField, and I needed to display the name of the selected choice in the template.  On a model you would just use the get_FOO_display() method, but a form has no such convenience.  I found a number of almost solutions on Stack Overflow, and after poring over them I concluded the best solution for my problem was to add the following method to the form:

class QuestionLanguageForm(forms.Form):
    ''' basic QuestionLanguage form '''
    language = forms.ModelChoiceField(queryset=Language.objects.all())
    short = forms.CharField()
    place_holder = forms.CharField()
    long = forms.CharField(widget=forms.Textarea(attrs={'rows':None, 'cols':None}))

def get_language_name(self):
    ''' returns the name of the selected language '''
    try:
       return Language.objects.get(id=self.initial['language']).name
    except:
       return None

This way I can use

{{ form.get_language_name }}

in my template and it prints the human readable name of the language set in my initial data.  This code is not robust; I catch all exceptions and ignore them because I’m only using it in a circumstance where this code won’t raise an exception (and in case it does, at least it won’t fail catastrophically).  However proper error handling could be added pretty simply.  #ExerciseForTheReader

P.S.  The title says “selected” value but I’m actually using a pre-loaded value, not anything user selected.  If you need to get a user selected value you could use self.data or self.cleaned_data instead of self.initial.

Convert Django Model Instances To Dictionaries

I just searched for a method to convert a model instance to a dictionary in Django and the top few results were a bunch of custom methods in Django Snippets and Stack Overflow.  I was about to use one of those when I clicked on one last link that showed me a better solution.  There is a method already built into Django that does exactly what I want: django.forms.models.model_to_dict.

As I pointed that out to a coworker, he showed me that that’s basically the same output as the .values() method in the queryset api.  If you don’t specify a value to retrieve, it will retrieve the whole model instance as a dictionary.

Here’s the output of these two pieces of code:

>>> Q.objects.filter(id=6).values()
[{'short': u'oui', 'deleted': False, 'language_id': 3L, 'long': u'oui', 'place_holder': u'oui', 'id': 6L, 'question_id': 8L}]
>>> model_to_dict(Q.objects.filter(id=6)[0])
{'short': u'oui', 'language': 3L, 'deleted': False, 'question': 8L, 'long': u'oui', 'place_holder': u'oui'}

There are two differences here, and one of them is pretty important.  First, .values() turns a queryset into an iterable that yields dictionaries, and model_to_dict turns a model instance into a single dictionary.  As long as you know about it this is pretty trivial to overcome.  Second, .values() converts foreign keys to {‘<field_name>_id’: #} and model_to_dict converts them to {‘<field_name>’: #}.  Since my purpose in using this was to populate a form I had to convert the id’s into model instances and update the dictionary.  Not a big problem, but a bigger annoyance to solve than the first difference.

IOError: request data read error

I couldn’t find a good simple explanation of what this error means on the web, so to future googlers, you’re welcome.

This most likely means that the user made a web reqeuest, probably through ajax, on your site and then cut off the connection before the ajax request completed.  The connection can be cut off by clicking on another link, closing the browser, or things in the middle like firewalls and whatnot.

There isn’t a good solution to this issue that I could find, the error is just something that happens from time to time.  If it happens regularly and you don’t think it’s an accidental connection cut off, do some more exploring (particularly with firewalls and other potential connection problems), otherwise it can be ignored.

Testing Email in Django The Easy Way

Today a coworker showed me a very easy way to test django code that sends emails.  It’s straight from the documentation:

Another approach is to use a “dumb” SMTP server that receives the e-mails locally and displays them to the terminal, but does not actually send anything. Python has a built-in way to accomplish this with a single command:

python -m smtpd -n -c DebuggingServer localhost:1025

This command will start a simple SMTP server listening on port 1025 of localhost. This server simply prints to standard output all e-mail headers and the e-mail body. You then only need to set the EMAIL_HOST and EMAIL_PORT accordingly, and you are set.

Proxying web.py through apache

I’ve been wanting to try web.py for a while. It seems like it’s easier to learn than Django for making python web applications. I made the hello world app quickly and simply, and decided I’d use it for a project I’m working on.

That’s when it turned south. I have to use php for another project I’m doing, and I use cgi-irc (in perl through CGI) on that server, so I didn’t want to re-setup all that stuff on lighttpd (the recommended method for running web.py). I tried running it through apache with cgi. After a few hours of that not working I switched to trying fastcgi on apache, also to no avail. I got fastcgi to the point where I was having an error that was common enough to be in an FAQ: I was getting 500 responses from apache because fastcgi/web.py wasn’t starting fast enough for the flup WSGI library to realize it, so it would start them over and over until it eventually gave up. I decided I just wanted to see the thing work at this point, so I installed lighttpd and went through the setup for that. When that method didn’t work I was fed up with their install instructions. I decided to try a method that John Quigley recommended. He said I should just run it with the internal http server that I’ve already used successfully and proxy the requests through apache to web.py. This might not achieve the goal of making the server large scale production ready, but I don’t really have to worry about that so much at this point. It does achieve my more important goal of making my php, cgi, and web.py applications all available through a single url and port, so I gave it a shot.

I looked up apache proxying and found it was done through mod_proxy, and each protocol you want to proxy has it’s own module as well. So I installed mod_proxy and mod_proxy_http and added the following line to my apache2.conf file:


ProxyPass /lifelog http://192.168.1.20:8888

192.168.1.20 being the IP address of the server in question. It would be better to get it resolving localhost so this line can be used on other servers, but I didn’t want to bother with it yet. I also had to change the proxy permission rules in proxy.conf to this:


ProxyRequests Off

<Proxy *>
AddDefaultCharset off
Order allow,deny
Allow from all
</Proxy>

This allows any user online to use my ProxyPass rule. Note that I left “ProxyRequests” set to “Off”. If I turned that on, I would be an open proxy that spammers and hackers could use to hide their identity during their nefarious behavior.

Now all that’s left is to start up my web.py server on port 8888 as specified in apache2.conf and whenever I go to /lifelog/* through apache, it’ll send the request to web.py.

Python User-Defined Exceptions and Error Handling in Python DbAPI 2.0

I just spent some time figuring out how to handle exceptions using the pysqlite library. I couldn’t find a good description of how it works online, so I’m making one.

In python, all exceptions are classes. This means they can use any of the capabilities that regular python classes have, but generally all they do is define a few instance values in the __init__ method. These values are available in the exception object (the instance of the exception class) when it is caught in a “try … except” block.

If you’re receiving a user defined exception from a foreign module and don’t know how to handle it, I’d recommend first wrapping the offending code in a “try…except” like this:

try:
    c.execute(sql)
except:
    logfile.write("ERROR: %s %s %s" % (sys.exc_info()[0], sys.exc_info()[1], sys.exc_info()[2]))

The function sys.exc_info() returns a tuple containing 1) the type of exception being handled, 2) the associated parameter (usually some helpful descriptor), and 3) a traceback to where the exception occurred.

This hopefully tells you two things: a description of why this happened, and more importantly for my situation, the exception type. When I ran that I got something like this:

ERROR: <class 'pysqlite2.dbapi2.OperationalError'> table something already exists <traceback object at 0xb7da3cd4>

So the exception being raised was at pysqlite2.dbapi2.OperationalError. Since the python dbapi returns sql errors through exceptions I can’t just fix the code here, I have to “try … except” it. Here’s what the final code snippet looked like:

try:
    c.execute(sql)
except pysqlite2.dbapi2.OperationalError, msg:
    logfile.write("ERROR: %s" % msg)

To catch the other exceptions specified in the dbapi v2.0, you’d have to include several more exception handlers. The other exception classes are in the same location as the OperationalError class.

Further reading: