Last Night A Rake Task Saved My LIfe - Real World Project Automation With Rake

Posted by Jeremy Voorhis Wed, 28 Jun 2006 03:14:00 GMT

Here’s a follow-up story about PLANET ARGON’s asset compiler project in action.

In our client’s application, an Image model is created for each image they upload and the uploaded images are saved into an asset source directory. After the images are saved, an observer launches the build system, acting on any source images whose targets are missing.

While I was speaking at Railsconf, a client of ours posted a bug to our Basecamp. For reasons I am still trying to determine, our client had uploaded four images to their application but the asset compiler task was not executed. It might take a minute to find the disconnect between my application and Rake, but our client needs those images that coincide with a press release as soon as we can deliver them.

Because asset compiler is build on top of Rake, it receives all of the benefits of Rake. I recalled that Rake supported a way to perform a dry run of a task execution (actually the option is called --dry-run). Because each individual target file is a dependency of the top level task, assets:build, running 

rake assets:build --dry-run
tells me exactly which target files were missing. Sure enough, they coincided perfectly with the files the client reported were missing. By running 
rake assets:build --dry-run | grep "** Execute"
I can ignore any file tasks which are not necessary. Within five minutes, I was able to diagnose the problem and allow my client to proceed after building the remaining target images and uploading them to the bandwidth provider, which was as simple as running 

rake assets:build

The beautiful thing here is how easy asset compiler lets me manage 4000 images. The application knows when to run build tasks, and in my assets.rake file, I can concisely define 11 types of image transformations. I can run each type of transformation individually, or I can run them in one shot. Even better, since rebuilding all of the images takes considerable CPU power, I can rsync my asset source directory to a separate build server and run the build tasks there, exporting them to our web server and the bandwidth provider.

For those of you unfamiliar, the asset_compiler project is hosted at planetargon.org and there is an article on InfoQ that tells the story of its motivation and implementation. Asset compiler is free software.

UPDATE

Here is the amazingly helpful output I got from rake assets:build --dry-run | grep "** Execute". 

** Execute (dry run) assets:navigation_icon:build
** Execute (dry run) assets:bronze_navigation_icon:build
** Execute (dry run) assets:navigation_icon_no_resize:build
** Execute (dry run) assets:bronze_navigation_icon_no_resize:build
** Execute (dry run) /path/to/assets/full_size/Ronaldo9mVert1.jpg
** Execute (dry run) /path/to/assets/assets/full_size/Ronaldo9mHoriz2.jpg
** Execute (dry run) /path/to/assets/assets/full_size/Ronaldo9mVert2.jpg
** Execute (dry run) /path/to/assets/assets/full_size/Ronaldo9mHoriz1.jpg
** Execute (dry run) assets:full_size:build
<snip />
** Execute (dry run) assets:build

This is exactly the kind of clear and unambiguous feedback I like to see when investigating this kind of problem. A big thank you to Jim Weirich for his thoughtful implementation of Rake.

Posted in Rails, web development, PLANET ARGON, Architecture | no comments

Heterogeneous Rails Applications

Posted by Jeremy Voorhis Mon, 17 Apr 2006 21:49:00 GMT

Or, Don’t be afraid to use the right tool

At Canada on Rails, I showed screenshots of a recent PLANET ARGON Rails development project which showcased my work with Globalize. What the slides did not explain was how we were able to process 1000s of images in an agile way.

We began by writing our own plugin, similar to FileColumn. Problem is, each image we had to manage required up to 4 variations, as well as transport the files to a bandwidth provider. In the beginning, the rules to produce these variations changed frequently. As you might guess, our models had suffered some bloat. My solution to coping with this problem was inspired by Jason Watkins’ experience in the video game industry: an asset compiler. In short, an asset compiler is a build tool that works from raw assets living in source directories and transforms them into their final state.

What I wanted was a flexible tool to regenerate – or build – all of these image variations, and I wanted it to stay out of my way. I had decided the best platform to build this tool might be Rake – a Ruby build system. I followed Jim Weirich’s RDocTask library as a template, and in a couple of days I had the asset compiler up and running. With the asset compiler in place, it was a simple matter to write an observer to fire off the right tasks. Our plugin and our image models were greatly simplified and now it simply saves or deletes a file in a source directory.

There is something desireable about heterogenous architecture. Specialist components with clear lines of responsibility become a joy to work with, and easier to maintain. Don’t force heterogenous design, but consider it as an agile approach when you suspect your application is growing into a monolith.

Posted in Rails, web development, PLANET ARGON, Architecture | no comments