Unexpected filter behaviour in Googles AppEngine

Monday, April 14th, 2008

I got a response from google about my weird unexpected behaviour when calling filter in my application.  It turns out to be expected behaviour, but is almost guaranteed to trip up a new developer.

The behaviour is that if you use a filter string like “property=” you get a None back rather than the data that you expected from the BigTable datastore.

The reason is that ‘=’ is a valid character for a property name, so the parser is unable to tell if your filter means the property called “property” = or the property called “property=”.

The end result is that you must put a space between the property name and the equals sign in order to  get valid results

Example behavour:

from google.appengine.ext import db
class A(db.Model):
b = db.IntegerProperty()
A(b=1).put()

#Works
print A.all().filter("b =").get()

# Fails, returns None
print A.all().filter("b=").get()

# Throws exception
print A.all().filter("b= ").get()

AppEngine update

Thursday, April 10th, 2008

So my host just started working this morning, so you can see the first stab at the app at http://shipwreck.mibgames.co.uk

I still haven’t quite got the hang of the developer accounts, but I’m persevering with my gmail account, although I would have liked a way to subscribe to the google group with my mibgames account, as I actually read that one daily, whereas gmail I don’t normally make a habit of reading. Although I’ve also apparently managed to get duplicate emails being sent to my personal email address, but haven’t worked out why yet. I’m sure I’ll work it out.

[Edit: Found out why, I had setup my personal account to download email from my gmail account via POP3 ages ago and hadn't turned it off.]

I also had my first couple of weird issues. I had created a django template, called base.html, which defined the base layout (or will do eventually), and was creating a template called create_character.html which inherited from the base template. I defined my title and main blocks in my template, and filled them in, and when I hit the template I got the weird error, TemplateSyntaxError that appeared to say that {% block main %} had a syntax error, and that it couldn’t read the expected next character.

I scratched my head a bit, tried renaming the blocks, tried rewriting it, to make sure I hadn’t done something really stupid. Eventually, I moved the blocks around, moving the title block to the end, and the main block to before the title, and it suddenly started working. Changing it back now doesn’t break it anymore, so I’ve no idea what caused it. Very weird.

My second issue is to do with getting stuff out of the BigTable, but first I had to get something in, and this is where I met my second favourite feature so far. I created a model for the player’s character, containing some basic stats, skills, and admin values. Having created the model, I want to try to be able to create one, save it to the BigTable, and then get it out again.

But the easiest way of creating a character without needing to generate random numbers, and do weird things, is to create a form with all of the relevant fields, and I can manually enter the values, press save and voila, it would save. So I set about creating a Django form for my model object. Having entered the 17 lines of attributes, run a few vim commands to replace db.StringProperty with forms.CharField and stuff, I thought, there must be an easier way than this.

And there is, in google.appengine.ext.db there is a module called djangoforms. Instead of my 17 lines of code, and the hiddeous thought of writing some manual code to hookup the form.clean_data['strength'] to character.strength, you can instead use the following lines of code

from google.appengine.ext.db import djangoforms

class CreateCharacterForm(djangoforms.ModelForm):
class Meta:
model = Character
exclude = ['location', 'health', 'armour', 'owner']

Yup, thats it, create a djangoforms.ModelForm, and give it a Meta class that specifies which model object you want it to associate with, and it does. In your view you just do

form = CreateCharacterForm(request.POST)
form.save()

and it does… unless you’ve declared anything as required, and excluded it from the form at the same time. Trying to wrok out how to get access to the model object, set it’s owner value before saving it was quite hard, until I noticed a parameter on the save method, commit=True. Checking out the source, it seemed true, if you give the keyword parameter commit and set it to False, the form.save method simply sets up your model object, fills it from the form and returns it to you, unsaved. I can then do

character = form.save(commit=False)
character.owner = users.get_current_user()
character.put()

And bobs your transvestite uncle who keeps turning up to family gatherings even though nobody has invited him.

My only problem now is that having executed this once, the object should exist and be in the datastore. But my code to get it back out, doesn’t seem to work.

I’m probably doing something really stupid, but from what I can read from the docs, this code should work.

character = Character.all().filter('owner=', users.get_current_user()).fetch(1)

I’ll work on it tomorrow and see what I can get working then.

Oh last feature, and this is my favourite feature so far. While looking to try to work out if my data was in the datastore, I found the dev_appserver admin interface. It doesn’t provide you with the logs, CPU cycles used or bandwidth obviously, but it does provide the facility to dynamically examine the datastore, and an interactive python interpreter, so you can just import your model and execute query methods to your hearts content.

AppEngine account activated and first django app uploaded

Wednesday, April 9th, 2008

Yes! My first django app, the framework for the MMO game has been created and uploaded.

Props goes to the person responsible for me getting an account, now all I have to do is blog about it as I go along. Given my level of excitement that shouldn’t be a problem for a few months at least.

Anyway, slightly pre-empting my blogpost tomorrow about settings and styles of MMO, I’ve created the app with the name shipwreck, and you can see the working code at here

My first thoughts, this is cool! I like it.

It’s a bit of a pain to create a django app, you end up creating a wrapping app that encases the django app, so my directory structure has shipwreck/shipwreck/main to get to the django app, inside the django project, inside the wrapper.

Secondly, I’ve had a bit of difficulty with my google apps for domains account. My gmail account is the one I got an AppEngine account for, but I’d like to be able to host the project at shipwreck.mibgames.co.uk, and develop and upload as my mibgames.co.uk account
Rather suprisingly to me, the admin interface actually supports this, I was able to add my mibgames.co.uk email address as a developer, and the email had a special link to confirm if mibgames.co.uk was an apps for domains domain name.
Secondly, you can request an external domain name for your app, requiring a simple CNAME change in DNS, which I did, but for some reason it wont activate yet.

Part of the problem is probably related to the same issues I’ve had before, which is logging into Google services, often the account I get logged in with can appear to be random as to whether it is my apps for domains account, or my gmail account as I have authentication saved on my development machines for both of them.

I’m sure those niggles will iron themselves out as I start to play.

Overview, so far, putting django, with templates, into an app and uploading it, very easy. Next, creating a database model and building the first couple of screens!

Google AppEngine

Wednesday, April 9th, 2008

Yes, everyone else has blogged about it, but I figured I would too.

I’ve had a long interest in python, and now at my new job I’m able to occasionally write some python scripts, as well as working with one of the best python programmers I’ve met, who introduced me into the London Python Developers meetups.

Anyway, since then I’ve been increasingly interested in django. as a web project I’ve wondered how appropriate it would be to develop an MMOG in Django, but I’ve been held back by not wanting to spend extra money on django hosting.

The good news is that Google’s AppEngine is free for small to medium traffic and datastores, and reading the license appears to have no issues with you using it for commercial purposes, and it uses a modified version of the Django framework. (Modified because it uses Google’s BigTable rather than a full relational database)
I’ve signed up for a beta account, but am on the waiting list, if I get an account I’ll try to create a basic MMO in it and let you know how it goes.

For web based independent games creation this could be a lifeline, providing free or cheap hosting. From now on you’ll be writing your high score tables in python on Google’s servers I’ll bet!

Learn Python Game Programming with Scripture Union

Tuesday, November 21st, 2006

There was a post today on the python-tutor mailing list, that brought a new game framework to my attention.

Being as I’m currently doing a little freelance web design work for Scripture Union, and they are a local charity to me here in Milton Keynes, I was pleasantly suprised to find that they have been running courses that teach game programming to children using python.

The livewires packages provide a nice simple interface for pygame, that might make learning to program a little easier.

The LiveWires holidays sound like a great holiday for the kids, They get morning to learn some technical computer or music skills, afternoons to do outdoor activities, and some bible study and worship in the evening.

There is an older childrens camp for those who might be interested  (ages 14 - 18).