Tip: Bundler with --binstubs

In a previous blog, I wrote how I’d aliased commands such as rake, cap and rspec to run either with or without bundle exec, based on the presence of a Gemfile. I gave up on that a while ago. Instead, I’ve started installing all my bundles like this:

bundle install --path .bundle/gems --binstubs .bundle/bin

I often use features like bundle open <gem> to debug and edit failing gems, so I like to keep each application’s gems isolated. The --path .bundle/gems installs them within an application’s .bundle directory. As well as isolating my gems, it has the added benefit that I can blow away the gemset with rm -rf .bundle

The --binstubs .bundle/bin option installs bundle-aware scripts for each command provided by a bundled gem. For example, a bundle including rake will generate a .bundle/bin/rake script. By adding ./.bundle/bin to the front of my environment PATH, the bundled version of rake will run when I’m in the application folder. I never have to type bundle exec!

Obviously typing that long bundle install command each time is tedious, so I’ve aliased it to bi:

alias bi='bundle install --path .bundle/gems --binstubs .bundle/bin'

I’ve been using these options for a few months, and so far I’m very happy with them.

Tip: Automatic bundle exec for rake and other gems

It’s irritating to run gem commands like rake, cap, rspec and others, only to find they needed to be executed via bundle exec. As a simple solution, I use a simple zsh function, combined with aliases for commonly used commands.

Here’s the function (which I’ve named be):

if [[ -a Gemfile ]]; then
  bundle exec $*
else
  command $*
fi

It’s very simple. If there’s a Gemfile in the pwd, it runs commands through bundle exec. Otherwise it just runs them.

I’ve combined this with some aliases for much less pain and less frustration:

alias rake='be rake'
alias cap='be cap'
alias rspec='be rspec'

An updated rails template for gem bundler

Update 8th February 2011:

Bundler has changed a lot since I wrote these instructions. Use them at your own risk!

A few months ago I wrote a rails template for gem bundler. Since then, bundler has changed a lot, and my template no longer works. Here then is an updated version, based on this gist from Andre Arko. Using it, you should be able to get a rails 2.3.5 project working with bundler in less than 5 minutes.

The first step is to install the latest bundler. At the time of writing, this was 0.9.9.

gem install bundler

Now you should be able to run the template, either on a new project, or on an existing rails 2.3.5 project.

rails -m http://github.com/tomafro/dotfiles/raw/master/resources/rails/bundler.rb <project>

On a fresh project, that should be all you need to do. On an existing that used an older version of bundler, you’ll need to remove the old hooks in config/preinitializer.rb and config/environment.rb, and the gems folder.

Explaining the template, step by step

The first step creates the project Gemfile, with rails available in all environments, and ruby-debug included in development. If the project has other gems, they should be added here, rather than using rails own config.gem mechanism.

file 'Gemfile', %{
source 'http://rubygems.org'

gem 'rails', '#{Rails::VERSION::STRING}'

group :development do
  gem 'ruby-debug'
end
}.strip

The next step is get bundler to load correctly. This is done in two stages. First, in config\preinitializer.rb bundler needs to be setup. This adds all the bundled gems to the ruby load path, but doesn’t initialise them.

  
append_file '/config/preinitializer.rb', %{
begin
  # Require the preresolved locked set of gems.
  require File.expand_path('../../.bundle/environment', __FILE__)
rescue LoadError
  # Fallback on doing the resolve at runtime.
  require "rubygems"
  require "bundler"
  if Bundler::VERSION <= "0.9.5"
    raise RuntimeError, "Bundler incompatible.\n" +
      "Your bundler version is incompatible with Rails 2.3 and an unlocked bundle.\n" +
      "Run `gem install bundler` to upgrade or `bundle lock` to lock."
  else
    Bundler.setup
  end
end
}.strip

Second, the rails boot process is modified to start the bundler environment. This ‘requires’ all gems in the bundle, letting them run initialisation code.

gsub_file 'config/boot.rb', "Rails.boot!", %{
  
class Rails::Boot
 def run
   load_initializer
   extend_environment
   Rails::Initializer.run(:set_load_path)
 end

 def extend_environment
   Rails::Initializer.class_eval do
     old_load = instance_method(:load_environment)
     define_method(:load_environment) do
       Bundler.require :default, Rails.env
       old_load.bind(self).call
     end
   end
 end
end

Rails.boot!
}

All that’s left now is a little cleaning up. The .bundle folder should never be checked into the code repository as it holds machine-local configuration, so it’s added to .gitignore. Finally, bundle install is run to fetch the bundled gems.

append_file '/.gitignore', %{
/.bundle
}

run 'bundle install'

And that’s it. I hope you find it useful.

Building rails gems from the 2-3-stable branch

For the latest application I’ve been working on, I wanted to use Michael Koziarski’s rails_xss plugin, to turn default escaping on in my erb templates. I’m also using wycats gem bundler to manage gems and their dependencies, including rails.

This posed a problem: xss_rails requires changes made in rails 2-3-stable branch, but not yet released in a gem (though they will be included in 2.3.5).

The solution was obvious: build my own gems, and get bundler to use them. Luckily, rails makes this an easy process.

First, clone rails from github, and change to the 2-3-stable branch:

git clone git://github.com/rails/rails.git
cd rails
git co -b 2-3-stable origin/2-3-stable

Next, we need to build the gems. Rails currently doesn’t seem to have a Raketask to build all its constituent projects (though I’m planning a patch to include one), so you have to build each one in turn:

cd actionmailer
rake gem PKG_BUILD=1
cd ../actionpack
rake gem PKG_BUILD=1
cd ../activerecord
rake gem PKG_BUILD=1
cd ../activeresource
rake gem PKG_BUILD=1
cd ../activesupport
rake gem PKG_BUILD=1
cd ../railties
rake gem PKG_BUILD=1
cd ..

The key is the PKG_BUILD variable. It appends a suffix to the rails version, so rather than building 2.3.4 (the version I checked out), it will build 2.3.4.1. If I decided to update my gems, I’d use PKG_BUILD=2, then 3 and so on.

Finally, once all these gems are built, it’s simply a case of finding them and using them. For gem bundler, this means placing them in the cache and updating the Gemfile to look for rails ‘2.3.4.1’. The gems are all built in pkg folders in their respective subprojects, so to copy them all somewhere else you can run:

cp **/pkg/*.gem <project-folder>/gems/cache

A rails template for gem bundler

Update 28th February 2010:

Bundler has changed a lot since I first wrote this template, so I've written a new version. Please use the updated version rather than the one below.

Following Nick Quaranto’s article ‘Gem Bundler is the Future’, I was inspired to try out bundler on my latest rails project. Previously, I’ve found rails' own gem management a massive headache. In contrast, using bundler has been a pleasure.

Getting it set up how I wanted took a fair bit of experimentation, so to make things easier both for me and the wider community, I’ve made a rails template to do the hard work.

Give it a try by running the following. You should be up and running in a couple of minutes:

rails -m http://github.com/tomafro/dotfiles/raw/master/resources/rails/bundler.rb <project>

That will give you a bundled project, ready for you to add your own gems. Here’s what each step of the template actually does:

Gem bundler is itself a gem. It can’t be used to manage itself, so to ensure that all environments use the same version, the first step is to install it right into the project:

inside 'gems/bundler' do  
  run 'git init'
  run 'git pull --depth 1 git://github.com/wycats/bundler.git' 
  run 'rm -rf .git .gitignore'
end

Just having bundler installed is no good without any way to run it; a script is needed. Once this is installed the local bundler can be run with script/bundle <options>:

file 'script/bundle', %{
#!/usr/bin/env ruby
path = File.expand_path(File.join(File.dirname(__FILE__), "..", "gems/bundler/lib"))
$LOAD_PATH.unshift path
require 'rubygems'
require 'rubygems/command'
require 'bundler'
require 'bundler/commands/bundle_command'
Gem::Commands::BundleCommand.new.invoke(*ARGV)
}.strip

run 'chmod +x script/bundle'

Bundler uses Gemfiles to declare which gems are required in each environment. This simple Gemfile includes rails in all environments, and ruby-debug in all environments other than production:

file 'Gemfile', %{
clear_sources
source 'http://gemcutter.org'

disable_system_gems

bundle_path 'gems'

gem 'rails', '#{Rails::VERSION::STRING}'
gem 'ruby-debug', :except => 'production'
}.strip

Most of the files bundler will place in the gem path can be regenerated; they shouldn’t be added to the project repository. The only things that should be added are the .gem files themselves, and the local copy of bundler. All the rest should be ignored:

append_file '.gitignore', %{
gems/*
!gems/cache
!gems/bundler}

The bundle script needs to be run for the first time:

run 'script/bundle'

Finally rails needs to be modified to ensure the bundler environment is loaded. This is done it two parts. First, a preinitializer is added to load the bundler’s environment file before anything else:

append_file '/config/preinitializer.rb', %{
require File.expand_path(File.join(File.dirname(__FILE__), "..", "gems", "environment"))
}

Second, rails initialization process is hijacked to require the correct bundler environment:

gsub_file 'config/environment.rb', "require File.join(File.dirname(__FILE__), 'boot')", %{
require File.join(File.dirname(__FILE__), 'boot')

# Hijack rails initializer to load the bundler gem environment before loading the rails environment.

Rails::Initializer.module_eval do
  alias load_environment_without_bundler load_environment
  
  def load_environment
    Bundler.require_env configuration.environment
    load_environment_without_bundler
  end
end
}

And that’s it. The project is now fully bundled. More gems can be added to the Gemfile and pulled into the project with script/bundle.

ZSH Completion for gem and gem open

I’ve been trying to get my head around the ZSH completion system. It’s not easy, but I’m slowly making progress.

Here’s my first semi-successful attempt. It provides command completion for gem (including installed commands) and gem name completion for specific gem commands (update, and open from Adam Sanderson).

So typing gem <tab> gives a list of possible commands, whilst gem open <tab> will complete with the names of the currently installed gems.

#compdef gem

_gem_commands () {
  if [[ -z $gem_commands ]] ; then
    gem_commands=$(gem help commands | grep '^    [a-z]' | cut -d " " -f 5)
  fi
  
  # This seems unnecessary, but if I try to set gem_commands to an array, it falls over.
 
  typeset -a gem_command_array
  gem_command_array=($(echo $gem_commands))
  compadd $gem_command_array
}
 
_installed_gems () {
  if [[ -z $installed_gems ]] ; then
    installed_gems=($(gem list | grep '^[A-Za-z]' | cut -d " " -f 1))
  fi
  
  typeset -a installed_gem_array
  installed_gem_array=($(echo $installed_gems))
  compadd $installed_gem_array
}
 
if (( CURRENT == 2 )); then
  _gem_commands
else
  if [[ $words[2] == open || $words[2] == update ]] ; then
    _installed_gems
  fi
fi

As it’s a first attempt, it’s a long way from perfect. I’ve put it on gist, for other people to play with, and I’d appreciate any advice or improvements. Specifically I’d like to know how to avoid the use of both gem_command_array and gem_commands. I’d also like to know a better way to check if the given command is in the list [open, update].

Please fork the gist, or tweet me with your suggestions.

Adam Sanderson's open_gem

The latest version of rubygems (1.3.2) now has an interface to add commands. Making great use of this feature, Adam Sanderson has written open_gem, a simple but amazingly useful tool.

You use it like this:

$ gem open activerecord

This opens the activerecord gem in your favourite editor (taken from either $GEM_OPEN_EDITOR or $EDITOR environment variables). If there are multiple versions of the gem installed, it will show a menu, letting you choose which version you require.

$ gem open activerecord
Open which gem?
 1. activerecord 2.1.0
 2. activerecord 2.3.2
>

open_gem itself is a gem, and can be installed with:

$ gem install open_gem

To get it working, you need to have $EDITOR set to something sensible:

$ export EDITOR=mate

If you’re running on OS X and use TextMate, you may have already set $EDITOR to mate -w, which let’s you use TextMate as the editor for git commit messages and much more. However, the -w flag doesn’t work with open_gem, so set the $GEM_OPEN_EDITOR variable, and open_gem will use that instead:

$ export GEM_OPEN_EDITOR=mate

You should now be good to go. If you want to see how it works, just use it on itself!

$ gem open open_gem