An updated rails template for gem bundler

A few months ago I wrote a rails template for gem bundler. Since then, bundler has changed a lot, and my template no longer works. Here then is an updated version, based on this gist from Andre Arko. Using it, you should be able to get a rails 2.3.5 project working with bundler in less than 5 minutes.

The first step is to install the latest bundler. At the time of writing, this was 0.9.9.

gem install bundler

Now you should be able to run the template, either on a new project, or on an existing rails 2.3.5 project.

rails -m http://github.com/tomafro/dotfiles/raw/master/resources/rails/bundler.rb <project>

On a fresh project, that should be all you need to do. On an existing that used an older version of bundler, you'll need to remove the old hooks in config/preinitializer.rb and config/environment.rb, and the gems folder.

Explaining the template, step by step

The first step creates the project Gemfile, with rails available in all environments, and ruby-debug included in development. If the project has other gems, they should be added here, rather than using rails own config.gem mechanism.

file 'Gemfile', %{
source 'http://rubygems.org'

gem 'rails', '#{Rails::VERSION::STRING}'

group :development do
  gem 'ruby-debug'
end
}.strip

The next step is get bundler to load correctly. This is done in two stages. First, in config\preinitializer.rb bundler needs to be setup. This adds all the bundled gems to the ruby load path, but doesn't initialise them.

  
append_file '/config/preinitializer.rb', %{
begin
  # Require the preresolved locked set of gems.
  require File.expand_path('../../.bundle/environment', __FILE__)
rescue LoadError
  # Fallback on doing the resolve at runtime.
  require "rubygems"
  require "bundler"
  if Bundler::VERSION <= "0.9.5"
    raise RuntimeError, "Bundler incompatible.\n" +
      "Your bundler version is incompatible with Rails 2.3 and an unlocked bundle.\n" +
      "Run `gem install bundler` to upgrade or `bundle lock` to lock."
  else
    Bundler.setup
  end
end
}.strip

Second, the rails boot process is modified to start the bundler environment. This 'requires' all gems in the bundle, letting them run initialisation code.

gsub_file 'config/boot.rb', "Rails.boot!", %{
  
class Rails::Boot
 def run
   load_initializer
   extend_environment
   Rails::Initializer.run(:set_load_path)
 end

 def extend_environment
   Rails::Initializer.class_eval do
     old_load = instance_method(:load_environment)
     define_method(:load_environment) do
       Bundler.require :default, Rails.env
       old_load.bind(self).call
     end
   end
 end
end

Rails.boot!
}

All that's left now is a little cleaning up. The .bundle folder should never be checked into the code repository as it holds machine-local configuration, so it's added to .gitignore. Finally, bundle install is run to fetch the bundled gems.

append_file '/.gitignore', %{
/.bundle
}

run 'bundle install'

And that's it. I hope you find it useful.

Rails 3 direct column reader

Whilst trying to get my head around arel and it's relationship to ActiveRecord in rails 3, I've updated the simple ColumnReader class I introduced last year. It lets you read the (correctly cast) column values for an ActiveRecord class, without the overhead of instantiating each object.

Here's the updated code:

module ColumnReader
  def column_reader(column_name, options = {})
    name = options.delete(:as) || column_name.to_s.pluralize
    column = columns_hash[column_name.to_s]
  
    self.module_eval %{
      def self.#{name}
        query = scoped.arel.project(arel_table[:#{column_name}])
        connection.select_all(query.to_sql).collect do |value| 
          v = value.values.first
          #{column.type_cast_code('v')}
        end
      end
    }
  end

  ActiveRecord::Base.extend(self)
end

The code isn't that different, though using scoped over construct_finder_sql feels a lot nicer. If you've got suggestions for improvement gist away.

Usage is similar to before, only using the new rails 3 syntax:

class Animal < ActiveRecord::Base
  column_reader 'id'
  column_reader 'name'  
 
  named_scope :dangerous, :conditions => {:carnivorous => true} 
end

Animal.names 
#=> ['Lion', 'Tiger', 'Zebra', 'Gazelle']
 
Animal.limit(1).names 
#=> ['Lion'] (Normal finder options supported)
 
Animal.dangerous.names 
#=> ['Lion', 'Tiger'] (Scoping respected)
 
Animal.ids
#=> [1, 2, 3] (Values cast correctly)

I'm still not entirely convinced of the value of this helper, so if you find a good use tweet me. Enjoy!

How to easily use Rails 3 now

gem uninstall bundler
gem install tzinfo builder memcache-client rack rack-test rack-mount 
gem install erubis mail text-format thor bundler i18n
gem install rails --pre

Now that rails 3 is getting closer to release, I wanted to start playing around with it. I've seen a few articles on getting it up and running, but they all seemed a little bit complicated. To use rails 2.3.5 before its release, I just built the gems myself and installed them. It turns out you can easily do the same with rails 3.

First, install rails main dependencies:

gem install rake rack bundler
gem install arel --version 0.2.pre

Next, get the latest rails code from github, and install the rails gems. There may be a few errors you can safely ignore:

git clone git://github.com/rails/rails.git
cd rails
rake install

And bang, you can start your first rails 3 project:

rails ~/apps/playground/rails3 

Your existing projects shouldn't be affected as rails is installed as a prerelease gem, but to be safe I'd recommend a tool like rvm to switch to a clean set of gems.

Tip: Relative paths with File.expand_path

You probably know about the __FILE__ magic constant. It holds the filename of the currently executing ruby source file, relative to the execution directory. So with the following saved as /code/examples/path_example.rb:

puts __FILE__

Running this file from the /code folder will output examples/path_example.rb

This is often used to load files on paths relative to the current file. The way I've used it before is like this:

config_path = File.expand_path(File.join(File.dirname(__FILE__), "config.yml"))

This works, but it's a bit clunky.

What I didn't realise until reading the rails source code the other day, is that File.expand_path can take a second argument - a starting directory. Also, this argument doesn't actually have to be a path to a directory, it also accepts a path to a file. With this knowledge we can shorten the above to the following:

config_path = File.expand_path("../config.yml", __FILE__)

Much simpler.

Building rails gems from the 2-3-stable branch

For the latest application I've been working on, I wanted to use Michael Koziarski's rails_xss plugin, to turn default escaping on in my erb templates. I'm also using wycats gem bundler to manage gems and their dependencies, including rails.

This posed a problem: xss_rails requires changes made in rails 2-3-stable branch, but not yet released in a gem (though they will be included in 2.3.5).

The solution was obvious: build my own gems, and get bundler to use them. Luckily, rails makes this an easy process.

First, clone rails from github, and change to the 2-3-stable branch:

git clone git://github.com/rails/rails.git
cd rails
git co -b 2-3-stable origin/2-3-stable

Next, we need to build the gems. Rails currently doesn't seem to have a Raketask to build all its constituent projects (though I'm planning a patch to include one), so you have to build each one in turn:

cd actionmailer
rake gem PKG_BUILD=1
cd ../actionpack
rake gem PKG_BUILD=1
cd ../activerecord
rake gem PKG_BUILD=1
cd ../activeresource
rake gem PKG_BUILD=1
cd ../activesupport
rake gem PKG_BUILD=1
cd ../railties
rake gem PKG_BUILD=1
cd ..

The key is the PKG_BUILD variable. It appends a suffix to the rails version, so rather than building 2.3.4 (the version I checked out), it will build 2.3.4.1. If I decided to update my gems, I'd use PKG_BUILD=2, then 3 and so on.

Finally, once all these gems are built, it's simply a case of finding them and using them. For gem bundler, this means placing them in the cache and updating the Gemfile to look for rails '2.3.4.1'. The gems are all built in pkg folders in their respective subprojects, so to copy them all somewhere else you can run:

cp **/pkg/*.gem <project-folder>/gems/cache

A rails template for gem bundler

Following Nick Quaranto's article 'Gem Bundler is the Future', I was inspired to try out bundler on my latest rails project. Previously, I've found rails' own gem management a massive headache. In contrast, using bundler has been a pleasure.

Getting it set up how I wanted took a fair bit of experimentation, so to make things easier both for me and the wider community, I've made a rails template to do the hard work.

Give it a try by running the following. You should be up and running in a couple of minutes:

rails -m http://github.com/tomafro/dotfiles/raw/master/resources/rails/bundler.rb <project>

That will give you a bundled project, ready for you to add your own gems. Here's what each step of the template actually does:

Gem bundler is itself a gem. It can't be used to manage itself, so to ensure that all environments use the same version, the first step is to install it right into the project:

inside 'gems/bundler' do  
  run 'git init'
  run 'git pull --depth 1 git://github.com/wycats/bundler.git' 
  run 'rm -rf .git .gitignore'
end

Just having bundler installed is no good without any way to run it; a script is needed. Once this is installed the local bundler can be run with script/bundle <options>:

file 'script/bundle', %{
#!/usr/bin/env ruby
path = File.expand_path(File.join(File.dirname(__FILE__), "..", "gems/bundler/lib"))
$LOAD_PATH.unshift path
require 'rubygems'
require 'rubygems/command'
require 'bundler'
require 'bundler/commands/bundle_command'
Gem::Commands::BundleCommand.new.invoke(*ARGV)
}.strip

run 'chmod +x script/bundle'

Bundler uses Gemfiles to declare which gems are required in each environment. This simple Gemfile includes rails in all environments, and ruby-debug in all environments other than production:

file 'Gemfile', %{
clear_sources
source 'http://gemcutter.org'

disable_system_gems

bundle_path 'gems'

gem 'rails', '#{Rails::VERSION::STRING}'
gem 'ruby-debug', :except => 'production'
}.strip

Most of the files bundler will place in the gem path can be regenerated; they shouldn't be added to the project repository. The only things that should be added are the .gem files themselves, and the local copy of bundler. All the rest should be ignored:

append_file '.gitignore', %{
gems/*
!gems/cache
!gems/bundler}

The bundle script needs to be run for the first time:

run 'script/bundle'

Finally rails needs to be modified to ensure the bundler environment is loaded. This is done it two parts. First, a preinitializer is added to load the bundler's environment file before anything else:

append_file '/config/preinitializer.rb', %{
require File.expand_path(File.join(File.dirname(__FILE__), "..", "gems", "environment"))
}

Second, rails initialization process is hijacked to require the correct bundler environment:

gsub_file 'config/environment.rb', "require File.join(File.dirname(__FILE__), 'boot')", %{
require File.join(File.dirname(__FILE__), 'boot')

# Hijack rails initializer to load the bundler gem environment before loading the rails environment.

Rails::Initializer.module_eval do
  alias load_environment_without_bundler load_environment
  
  def load_environment
    Bundler.require_env configuration.environment
    load_environment_without_bundler
  end
end
}

And that's it. The project is now fully bundled. More gems can be added to the Gemfile and pulled into the project with script/bundle.

Tip: The case for from_param

There's one small method I add to every new rails project I work on:

module Tomafro::FromParam
  def from_param(param)
    self.first :conditions => { primary_key => param }
  end
end

ActiveRecord::Base.extend(Tomafro::FromParam)

In my controllers, where you might use Model.find(params[:id]) or Model.find_by_id(params[:id), I use Model.from_param(params[:id]) instead.

All three methods have almost the same behaviour, the only difference being the handling of missing records. find throws a RecordNotFound, while find_by_id and from_param return nil. So why use from_param over the others?

The answer comes when you want to change to_param, the method rails uses to turn a record into a parameter. It's a good principal (though often broken) not to expose database ids in urls. An example might be to use a users nickname rather than their id in user urls, so /users/12452 becomes /users/tomafro. In rails this is easy to achieve, by overriding the to_param method:

class User < ActiveRecord::Base
  def to_param
    self.nickname
  end
end

Rails will automatically use this method when generating routes, so users_path(@user) will return /users/tomafro as we'd like. If I was using find or find_by_id in my controllers, I'd then have to go through each one and change it to find_by_nickname. Luckily though, I've used from_param, so whenever I override to_param I just have to remember to provide an equivalent implementation for from_param, and my controllers will work without modification:

class User < ActiveRecord::Base
  def self.from_param(param)
    self.first :conditions => {:nickname => param}
  end
  
  def to_param
    self.nickname
  end
end

I've been doing this for years, but it's hardly a new principle, to provide a from method for every to method. There's even an old ticket on trac asking for it, but it's been considered too trivial to add.

I disagree - for me the value comes from having the method from the start, not when you need it. Luckily it's easy to add to my own projects.

The cost of explicit returns in ruby

Yesterday Pratik Naik wrote a post on one of his pet hates - explicit returns in ruby. I agree 100% with Pratik, I can't stand them either.

In one of the comments, Chris Wanstrath posted this microbenchmark showing that using explicit returns incurs a surprising performance hit. It seemed crazy to me, so I thought I'd investigate further, running the benchmark on jruby and ruby 1.9 for comparison.

ruby 1.8.7 (2008-08-11 patchlevel 72)

               user     system      total        real
explicit   3.420000   0.020000   3.440000 (  3.478501)
implicit   2.220000   0.000000   2.220000 (  2.236413)

jruby 1.1.6 (ruby 1.8.6 patchlevel 114)

               user     system      total        real
explicit   2.611000   0.000000   2.611000 (  2.611001)
implicit   2.541000   0.000000   2.541000 (  2.541385)

ruby 1.9.1p0 (2009-01-30 revision 21907)

               user     system      total        real
explicit   1.580000   0.010000   1.590000 (  1.614273)
implicit   1.520000   0.000000   1.520000 (  1.537492)

So neither jruby nor ruby 1.9 incur this penalty.

Is this good news or not? If, like me, you consider explicit returns a code smell, I guess it doesn't matter either way.

Pimp my script/console

For a long time I've had an .irbrc file and a .railsrc file, setting up some simple helpers methods in my irb and script/console sessions. Today though, I wanted to add some more helpers specific to the project I'm working on. Specifically, I wanted to be able to use my machinist blueprints, to help me play around with some models.

Adding machinist isn't as simple as just requiring my blueprints though -- they require my ActiveRecord models, which aren't available when .irbrc is loaded. Luckily the solution is simple -- just add a couple of lines to the configuration in environment.rb:

Rails::Initializer.run do |config|
  if defined?(IRB)
    config.gem 'faker'
    config.gem 'notahat-machinist', :lib => 'machinist', :source => "http://gems.github.com"
    IRB.conf[:IRB_RC] = Proc.new { require File.join(RAILS_ROOT, "config", "console") }
  end
end

The key part is the line starting IRB.conf[:IRB_RC], which tells irb to run the given when the session starts. Luckily, this happens after rails has finished initializing. I've set it to require config/console.rb, to which I can add all sorts of configuration and helpers, knowing it will only get loaded in script/console sessions where I want these shortcuts, not in my general code. Here it is:

require File.expand_path(File.dirname(__FILE__) + "/../spec/blueprints.rb")

def tomafro
  Account.find_by_login("tomafro")
end

def bbi
  Client.find_by_name("Big Bad Industries")
end

Read ActiveRecord columns directly from the class

Sometimes you want to read just a single column from a collection of records, without the overhead of instantiating each and every one. You could just execute raw SQL, but it's a shame to do away with the nice type conversion ActiveRecord provides. It'd also be a pity to get rid of find scoping, amongst other goodness.

Enter Tomafro::ColumnReader:

module Tomafro::ColumnReader
  def column_reader(column_name, options = {})
    name = options.delete(:as) || column_name.to_s.pluralize
    column = columns_hash[column_name.to_s]
    
    self.module_eval %{
      def self.#{name}(options = {})
        merged = options.merge(:select => '#{column_name}')
        connection.select_all(construct_finder_sql(merged)).collect do |value| 
          v = value.values.first
          #{column.type_cast_code('v')}
        end
      end
    }
  end
end

Once you've extended ActiveRecord::Base with it, usage is simple. In your models, declare which columns you want access to:

ActiveRecord::Base.extend Tomafro::ColumnReader
 
class Animal < ActiveRecord::Base
  column_reader 'id'
  column_reader 'name'  
 
  named_scope :dangerous, :conditions => {:carnivorous => true} 
end

Once you've done this, you can access values directly from the class, respecting scope, limits and other finder options.

Animal.names 
#=> ['Lion', 'Tiger', 'Zebra', 'Gazelle']
 
Animal.names :limit => 1 
#=> ['Lion'] (Normal finder options supported)
 
Animal.dangerous.names 
#=> ['Lion', 'Tiger'] (Scoping respected)
 
Animal.ids
#=> [1, 2, 3] (Values cast correctly)

Using Rack Middleware for good and evil

So we all know that Rack is awesome, and that we can use Rack::Middleware for all sorts of things: debugging, caching and a whole host more.

What all these have in common (apart from maybe Rack::Evil) is that they're all helpful. They all make writing Rack applications easier. Not my Middleware though.

Introducing Rack::Shuffler

module Rack
  class Shuffler
    def initialize(app)
      @app = app
      @responses = []
    end
    
    def call(env)
      @responses << @app.call(env)
      @responses[rand(@responses.size)]
    ensure
      @responses.delete_at(rand(@responses.size)) if @responses.size > 100
    end
  end
end

I suggest you add it to a colleague's app late on a Friday afternoon, and see how long it takes to drive them to insanity.

Automatching rails paths in cucumber

If you're using cucumber as part of your testing, you probably have a paths.rb file that looks something like this:

module NavigationHelpers
  def path_to(page_name)
    case page_name
    
    when /the home page/
      root_path
    when /the new client page/
      new_client_path
    when /the clients page/
      clients_path    
    # Add more page name => path mappings here
    else
      raise "Can't find mapping from \"#{page_name}\" to a path.\n" +
      "Now, go and add a mapping in features/support/paths.rb"
    end
  end
end

World(NavigationHelpers)

This let's us use nice descriptive names in our scenarios, but it starts to become a pain when we add more and more paths. So how can we make it better?

By automatically matching some rails paths. Here's the code:

module NavigationHelpers
  def path_to(page_name)
    case page_name
    
    when /the home page/
      root_path   
    # Add more page name => path mappings here
    else
      if path = match_rails_path_for(page_name) 
        path
      else 
        raise "Can't find mapping from \"#{page_name}\" to a path.\n" +
        "Now, go and add a mapping in features/support/paths.rb"
      end
    end
  end

  def match_rails_path_for(page_name)
    if page_name.match(/the (.*) page/)
      return send "#{$1.gsub(" ", "_")}_path" rescue nil
    end
  end
end

World(NavigationHelpers)

What it does is pretty simple. Given a page name the clients page (with no other matches defined) it will try and send clients_path. If successful, then it returns the result, otherwise nil.

Not the biggest improvement in the world, but it's made my cucumber tests just a little bit easier to write.

Adam Sanderson's open_gem

The latest version of rubygems (1.3.2) now has an interface to add commands. Making great use of this feature, Adam Sanderson has written open_gem, a simple but amazingly useful tool.

You use it like this:

$ gem open activerecord

This opens the activerecord gem in your favourite editor (taken from either $GEM_OPEN_EDITOR or $EDITOR environment variables). If there are multiple versions of the gem installed, it will show a menu, letting you choose which version you require.

$ gem open activerecord
Open which gem?
 1. activerecord 2.1.0
 2. activerecord 2.3.2
> 

open_gem itself is a gem, and can be installed with:

$ gem install open_gem

To get it working, you need to have $EDITOR set to something sensible:

$ export EDITOR=mate

If you're running on OS X and use TextMate, you may have already set $EDITOR to mate -w, which let's you use TextMate as the editor for git commit messages and much more. However, the -w flag doesn't work with open_gem, so set the $GEM_OPEN_EDITOR variable, and open_gem will use that instead:

$ export GEM_OPEN_EDITOR=mate

You should now be good to go. If you want to see how it works, just use it on itself!

$ gem open open_gem