See what I'm up to: Bitbucket |  LinkedIn | StackOverflow | Resume (PDF)
Learning to love test driven development

I’ve taken the week off of mobile game development, and in fact have fallen into what has turned out to be a very productive routine. Each project I have on my plate, I have been working on for a week solid, without touching the other projects. This allows just enough time to get into the groove without getting burned out. The first day of each cycle has been a readjustment day, but the other six days have been awesome. So last week I was dealing with game engines, and actually ended up where I started, loving Moai, even with it’s frustrations, and continuing down that path.

This week, though, I’m back on my Magic: The Gathering database. I’m rapidly approaching an MVP and am investigating the route I wish to go for deployment. Last night, though, I ran into an issue. It was in the wee hours of the morning, technically, and my brain was foggy, and I could not remember if I had incremented a counter properly in my view functions. To explain, I had to brew up my own user-tagging django app, because none of the standard packages offered the features I wanted, and adding to them would have been more hassle that writing my own app. At some point I may release it as a stand-alone Django app for integration into other projects, but that’s another issue for another day. In this instance though, not only do I have a table for the tags themselves, but I have a few meta-tables that track the usage counts of tags on various other app models. These meta-tables are there to speed up processing; although I now have to interact with two extra tables per tag-interactoin, I don’t need to pull large querysets and count the results each time a user tries to access certain pages. It’s worked out quite well in testing. That’s manual testing, though, and I couldn’t remember if I had checked if the counts were properly changing.

So rather than take the lazy one-time route of checking the code or doing a manual test, I decided to stop procrastinating and write some unit tests for the project, which I had neglected to do (and often neglect to do until deployment is on the radar). In doing so, I started to see some tests that took too long to complete (on the order of 100ms compared to ~20-50ms), and found some data that was displaying correctly due to over-zealous string formatting, but was stashing data in an un-beautiful way in the database. After seeing these few comparitively small issues I decided to spend today not only writing the test suite for all functionality currently implemented, but most of the other functionality I was planning to get done this week.

With that done, each time I finish a module, checking functionality has been greatly sped up, and overall productivity has increased. Unit testing really is one of those tedious parts of development that pays off tenfold, it just sucks to deal with in the short term.

Once I rotate back to the game development project, I’ll be digging into unit testing libraries for Lua as well.

Ramblings about Lua based mobile game engines

Several months ago I prototyped a basic skeleton for a game idea I had in Love2D, a Lua scripted game engine for desktop computers. Love is an excellent piece of software, that isn’t what a typical professional studio would choose, but is honestly one of the best software pacakges I’ve ever come across. It works flawlessly on Windows, OSX, and Linux. It works identically on all systems, and deployment to any system is as easy as bundling up a zip file of Lua source code and having the operating system open that with the binary, which can be done by stuffing it in an OSX app’s resources directory, a simple .bat or .sh script, or a handful of other ways. Love is incredibly well documented, to boot. Every function is documented correctly, and not only does their docs explain how each piece is used, but also how to fit together the pieces to make a game.

So several weeks ago, when I decided to try to port this game concept to a full game being released on Android and iOS, I naturally wanted to stick with Lua. The language is a little quirky in that it’s over-liberating, but it’s very functional, and a staple in the professional game world. It’s written in pure ANSI C (excepting it’s dynamic library loading, which there’s no ANSI C way to do), meaning it runs on pretty much any processor/environment made in the past 80 years, and it’s very quick, especially when it comes to bridging C/Lua data and functions. Knowing I wanted to port my previous Lua code, continue honing my Lua skills, and keep my cross platform options open, I started looking at mobile game engines that targeted Android and iOS. What follows are some notes from late nights of hair pulling and anaylyzing my options.

I’ll note here that I was/am looking for a free, open source solution. If you’re looking to get running fast, you probably don’t want that. Paid software does have its benefits and perks, primarily when it comes to support, documentation, and things working on the first try. I’m willing to take the extra effort to learn an Open Source package, for some rather philisophical reasones pertaining to “owning” my own software.

Corona SDK

Arguably one of the most used options, and almost definitely so if you’re going by paid-subscription model, Corona is a very well rounded package. Their included simulator is quick to get running, and there’s countless book, tutorials, and blog posts about how to use it. I only tinkered briefly though, as I knew I didn’t want a paid solution. An even bigger turn off was that the software uses their servers to build the final project, even with the higher-end licenses. Only the top tier “Enterprise” level accounts have the ability to do offline local builds.

Gideros Mobile

Gideors was a package I liked even more than Corona. It has a bit less built into the core libraries, but has a fairly extensive number of 3rd party plugins to cover that gap. Corona’s plugin library is still larger, as it’s it’s core set. Where Gideros really shined was on device testing. Gideros ships with it’s own host app, and the IDE will deploy over wifi to a tablet or phone for real world testing. Because this concept of a core host implemented in native code is really at the heart of any Lua game engine, there’s no performance penalty compared to a real instance of the same app being deployed, save for perhaps shaving off the http host running in the background. It’s built in sprite editor is not bad, but if you’re used to any specific tools you’ll honestly stick to those most likely. In the end though, it was once again the financial cost and commercial license model that turned me away.

Moai

Moai is what I spent the most time with, and what I felt looked the most promising. I was able to implement up to where I was in Love, and I was pleased with the result, and how quickly I was able to get there. In the end though, my frustrations with some issues proved to be irreconcilable. For one thing, the documentation is nowhere near where it needs to be. Some functions aren’t documented at all. My code ran quickly, even churning out lots of physics collisions, and complex camera transforms. Also, unlike Corona and Gideros, the fully open source nature means not only are you allowed to tinker with the C++ host/bindings, it’s encouraged. Adding new features in C/C++ doesn’t require a special license. Ultimately though, it was a let down, once I dug past the inital tests.

The engine’s documentation is dated in some areas, and borderline non-existent in others. The build instructions on the main page rarely work. The majority of functions will list the parameters and returned result’s class, but nothing else. The entire Box2D documentation just says “see Box2D documentation” (referring to the docs of the actual library). The Chipmunk physics library is even worse - every math function in Lua uses radians. Every function in MOAI uses degrees - not terrible, converting isn’t hard –EXCEPT FOR CHIPMUNK. Why? These small inconsistencies proved to just be too much.

Moai looked promisisng. It’s marketed as “The mobile platform for pro game developers”. It’s used by Double Fine extensively. But it really is for pros, and not hobbyists. Depending on how you count, there’s 4-5 different branches in the github repo for v1.5. 1.5 development, 1.5 stable, 1.5 master (which is apparently neither stable nor dev). Then there’s the forks from 1.5 that are listed in the documentation, as if they’re official. Furthermore, there’s no release schedule or information. 1.5 is 1.5, and 1.4 is 1.4. There’s no third digit, or date in versioning, and if you download the repo a few weeks apart, it’s pretty likely there will be minor differences. In my case, even my detailed notes on how I got it to build didn’t work a month later, and I ended up thumb-driving it from one computer to another to ensure I was on the same version.

There is an excellent book, Developing Mobile Games with Moai SDK, by Francisco Tufro that is definitely worth picking up if you decide to go with Moai. It’s especially worthwhile if you’re new to Lua gamedev, as it covers some basic patterns for structuring the flow of a game’s codebase.

MOAI is made for studios who are going to be implementing their own Lua host anyways. If you’re doing that, excellent, it’s a wonderful option for the final bridge to your Lua environment, but if you’re a one man shop who doesn’t want to deal with that set of issues, keep shopping.

I’m not trying to fault anyone here, or offend anyone. Moai is developed by a game studio (Zipline) - they’re in the business of making games. Making their tool open-source was a gift to the community, and I don’t want to look a gift horse in the mouth. I do want to warn other first time devs to not get discouraged if they run into the endless sea of issues I have had with MOAI.

Cocos2d-x

I originally looked at Cocos2d-x a little over a year ago. It’s based on Cocos2d, an Objective-C based library/framework, but ported to C++. As of the 3.0 release, it now supports Lua, as well. If none of these languages are your cup of tea, it’s also been ported to Javascript, and I’m sure there’s bindings for everything else.

Cocos2d-x is possibly the most used mobile game library out there, with Unity and Corona not far behind. It’s mature, stable, and well developed. It’s not without issues, I also ran into build issues with it, but the community is significantly larger than MOAI’s which makes finding fixes a whole lot easier. Like MOAI, it suffers from sometimes sparse documenation, but like the build issues, the larger community really helps out. Really, that’s what it comes down to in the open source world, is the community behind the project. Moai’s community was great, an incredibly kind and helpful crowd, but it wasn’t the size needed for the amount of problem solving that a fledging young game engine needs.

So far, I’m still pretty fresh. Cocos2d comes with a bit more than Moai out of the box, including built in scene management. It also has it’s own IDE and toolset now, but I’m avoiding those for the time being. Chipmunk is built in, which is an improvement from 2.x which had no built in physics, but having Box2D as an option would be nice.

So with that, I’m out to try some more testing. I’m sure I’ll be back with more notes in the future.

Pylint - one of the best things that has ever happened to my productivity

Pylint is one of those tools I had heard about for a couple years, but never really examined, thinking it was just a code style checker (making sure to use camelCase when needed, and underscores_when_appropriate). A couple nights ago I decided to finally give it a shot, and wow, the results have been awesome. Not only does it check style, but it catches unused variables, catches runtime errors, lets you know when you’re writing sloppy code. After playing with the default configuration for a bit, I started digging into custom configs(mostly becasue I was sick of getting hit for parenthesizing print functions as is required in Python3).

There’s config options for the max number of class ancestors, function parameters, and exception handling. What’s even better is at the end of code analysis, you get a score, which is helpful for motivating a little cleanup step before that revision control commit. It’s also shaved a lot of time off testing already, a quick run of pylint is a much quicker test to find syntax errors and dumb mistakes rather than loading up the project and executing it in many cases. I have found this especially true with Django so far, loading up the test server, getting a list of obscure errors, and tracking them down is much more hassle than Pylinting the changed files, and seeing very verbose, explicit recommendations, in half the time.

All in all, if you haven’t tried it, give it a shot. It’s a great package.

Django view options cause me more grief than they're worth.

Ask any Perl programmer what they’re favorite aspect of Perl is, and almost inevitably you’re get a cheery “Tim Towdy!”. If you’re not familiar, little Timmy is the mascot of the acronym TIMTOWTDI - There Is More Than One Way To Do It. Perl is an impressively stable, relatively speedy, highly adaptable scripting language nearly as old as I am. I learned it in the early 2000’s back when PHP was still hard to get on cheap web hosting, and was Perl pretty much the only way to get cool things like guestbooks and visitor counters on my awesome website full of tags and animated gifs. I haven’t touched a Perl script in well over a decade, and I would like to keep it that way, but we’re not here to talk about that.

Python, which has in many ways superceded Perl in terms of being the most popular, all around general utility scripting language, takes a very different approach than Perl does. It also clearly draws inspiration from Perl in it’s amazing adaptability to almost any task. We’re here to examine the heart of Pythong (and Django), so I’m going to draw your attention to PEP 20 (that’s Python Enhancement Proposal 20). I’m really looking at lines thirteen (and fourteen which I’ve never really understood):

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

There should one– and preferably only one –obvious way to do it.

So where does Django fit in, and why am I rambling about this at 1:00 AM on a Tuesday? Django, up through, what was it, 1.4 or 1.5, had only one way to create a ‘view’. A view, in Django (and many web frameworks), takes a request, accesses any data in the program it needs, and serves up a web page (or JSON or other HTTP response) to the requesting user/app. We take data in, we process it, we return a result. What do you think would be a good way to implement this? If you said a function, you think the same way I do. And that’s the way it’s worked for a long time, it was elegant, it was simple, going back to PEP20 - it was beautiful, it was explicit, it was flat, it was readable, and the implementation was exceptionally easy to explain. Any view function can access the full HTTP request, access all database information, and render it to a Jinja-like templating system that was a dream to use.

Then around 1.4 or 1.5 Django introduced Class Based Views, or CBVs as they now call them. I cannot lie, there are some excellent times that CBVs have save boilerplate view processing code. They also help another Pythonic principle - DRY (don’t repeat yourself!). The don’t come without downsides though. Now there’s more than one way to render a view. Most of the time neither is obviously the better application for your app until well after you’ve developed your MVP and have gone past deployment into extending functionality later in the projects life (ie too late to change to the better pattern). They’re definitely neither explicit nor flat, hiding large amounts of data processing behind the curtain of a single Class the developer never even once calls an init on. In fact almost zero functionality or data handling is shown at all, the beauty and blissful ignorance of the inner workings of Django is left to simply assigning a few variables and hopping into the template.

My complaint, as I sit here watching a script pull ~15,000 640x480px images from the web for a Django project I’m developing right now, is that oftentimes you cannot tell what is best for the project until after you’ve hit public release and are adding additional features. And from that moment on you’re constantly looking back at your view code, realizing you can make it look a hundred times better (or at least more readable) with a refactor, and debating whether it’s worth the effort. It’s not the CBVs themselves (though you can find plenty of bloggers complaining about CBVs, just as you can find plenty of Django tips that say never touch FBVs again). It’s that there was a wonderfully serviceable option, and some one had to make a completely unnecessary second option. For this project, I have done my planning, my research, I have mapped out current and potential future design goals. I have examined the high likelyhood I’ll be implementing a RESTful API and potentially developing iOS/Android apps, and I have chosen FBVs because I need to do some explicit processing of data between multiple models, and I don’t feel like shoving that into an OOP pattern just because everyone loves OOP. But two years from now when I’m implementing new feature X I’m going to look at the views.py files approaching 1000 lines in length and ask myself why I didn’t just extend the TemplateView or ModelView classes to suit my needs, or develop a small library of mixins to suit my needs.

And I’ll just sigh, and think “There should be one– and preferably only one –obvious way to do it”.

Welcome to the new blog

Well, welcome to the new blog. After some thought I decided to go ahead and rm -r the whole Wordpress site, and drop its database. As much as Wordpress and Drupal have made life easier in some regards, I’ve grown tired of updating them, concerning myself about security holes in the core software and/or plugins, getting emails about comments from innovative new spambots, and constantly looking around for cool new themes and plugins. I’m simplifying a lot in my life, including this site.

Of course that “simplifying” is a lot less accurate than it seems. After trying several static site generators and finding complaints with all of them, I did what any self-respecting, bored programmer with too much time on their hands does, and started to roll my own. The software is called Frood, and is currently in a pre-alpha stage, but I’m actively working on it. Disqus comments will be enabled soon, and more features are being implemented everyday. For now, though, it’s enough to host my own blog. The source code is available on Bitbucket, and I hope to have it in a solid beta stage for public usage in the next month or so. Expect a blog post and it’s own website (or rather, subdomain of this site) in the next week or so.