Mongo instrumentation released as a gem

Enough people seemed to comment and like the mongo instrumentation code I wrote about yesterday that I've packaged it up and released it as a gem.

The mongo-rails-instrumentation gem is available on rubygems, and the code is up on github.

Adding it to a project is simple, just put the following in your Gemfile, run bundle install and restart your app.

    gem 'mongo-rails-instrumentation', '~>0.1'

Please add any suggestions, improvements and comments to the code in github. I hope people find it useful.

Experimental Mongo instrumentation (for Rails 3)

Update: Changed to instrument methods on the Mongo::Connection

One of our latest rails projects uses Mongo as a backend. We're just starting to get some traffic, and as we do, we're monitoring the logs for slow requests. When using ActiveRecord, rails splits out the recorded request time like so:

    Completed 200 OK in 6ms (Views 370.5ms | ActiveRecord: 2.3ms)

We wanted to do the same for our Mongo accesses, just to give a rough idea as to what our requests were doing. Luckily Rails 3 makes this relatively straightforward, providing hooks to instrument methods, subscribe to log messages and add information to the request log. Here, then, is my first stab (mainly harvested from ActiveRecord):

module Mongo
  module Instrumentation
    def self.instrument(clazz, *methods)
      clazz.module_eval do
        methods.each do |m|
          class_eval %{def #{m}_with_instrumentation(*args, &block)
            ActiveSupport::Notifications.instrumenter.instrument "mongo.mongo", :name => "#{m}" do
              #{m}_without_instrumentation(*args, &block)
            end
          end
          }

          alias_method_chain m, :instrumentation
        end
      end
    end

    class Railtie < Rails::Railtie
      initializer "mongo.instrumentation" do |app|
        Mongo::Instrumentation.instrument Mongo::Connection, :send_message, :send_message_with_safe_check, :receive_message

        ActiveSupport.on_load(:action_controller) do
          include Mongo::Instrumentation::ControllerRuntime
        end

        Mongo::Instrumentation::LogSubscriber.attach_to :mongo
      end
    end

    module ControllerRuntime
      extend ActiveSupport::Concern

      protected

      attr_internal :mongo_runtime

      def cleanup_view_runtime
        mongo_rt_before_render = Mongo::Instrumentation::LogSubscriber.reset_runtime
        runtime = super
        mongo_rt_after_render = Mongo::Instrumentation::LogSubscriber.reset_runtime
        self.mongo_runtime = mongo_rt_before_render + mongo_rt_after_render
        runtime - mongo_rt_after_render
      end

      def append_info_to_payload(payload)
        super
        payload[:mongo_runtime] = mongo_runtime
      end

      module ClassMethods
        def log_process_action(payload)
          messages, mongo_runtime = super, payload[:mongo_runtime]
          messages << ("Mongo: %.1fms" % mongo_runtime.to_f) if mongo_runtime
          messages
        end
      end
    end

    class LogSubscriber < ActiveSupport::LogSubscriber
      def self.runtime=(value)
        Thread.current["mongo_mongo_runtime"] = value
      end

      def self.runtime
        Thread.current["mongo_mongo_runtime"] ||= 0
      end

      def self.reset_runtime
        rt, self.runtime = runtime, 0
        rt
      end

      def mongo(event)
        self.class.runtime += event.duration
      end
    end
  end
end

It looks complicated, but it's actually pretty simple. Data access methods in Mongo::DB and Mongo::Collection Mongo::Connection are hijacked and surrounded by an ActiveSupport::Notifications.instrumenter.instrument block. This triggers events which are listened to by the LogSubscriber, summing the total time spent in Mongo. The ControllerRuntime then collects this count to be displayed, and resets the sum to zero ready for the next request. The output looks like this:

    Completed 200 OK in 838ms (Views: 370.5ms | ActiveRecord: 2.3ms | Mongo: 441.5ms)

It's just a first stab, so any comments and improvements are more then welcome. It's here on gist so please fork away.

Rails 3 direct column reader

Whilst trying to get my head around arel and it's relationship to ActiveRecord in rails 3, I've updated the simple ColumnReader class I introduced last year. It lets you read the (correctly cast) column values for an ActiveRecord class, without the overhead of instantiating each object.

Here's the updated code:

module ColumnReader
  def column_reader(column_name, options = {})
    name = options.delete(:as) || column_name.to_s.pluralize
    column = columns_hash[column_name.to_s]
  
    self.module_eval %{
      def self.#{name}
        query = scoped.arel.project(arel_table[:#{column_name}])
        connection.select_all(query.to_sql).collect do |value| 
          v = value.values.first
          #{column.type_cast_code('v')}
        end
      end
    }
  end

  ActiveRecord::Base.extend(self)
end

The code isn't that different, though using scoped over construct_finder_sql feels a lot nicer. If you've got suggestions for improvement gist away.

Usage is similar to before, only using the new rails 3 syntax:

class Animal < ActiveRecord::Base
  column_reader 'id'
  column_reader 'name'  
 
  named_scope :dangerous, :conditions => {:carnivorous => true} 
end

Animal.names 
#=> ['Lion', 'Tiger', 'Zebra', 'Gazelle']
 
Animal.limit(1).names 
#=> ['Lion'] (Normal finder options supported)
 
Animal.dangerous.names 
#=> ['Lion', 'Tiger'] (Scoping respected)
 
Animal.ids
#=> [1, 2, 3] (Values cast correctly)

I'm still not entirely convinced of the value of this helper, so if you find a good use tweet me. Enjoy!

How to easily use Rails 3 now

Update 10th February 2010:

The instructions below were useful earlier in the development cycle. Now the beta gem has been released, the process is much easier:
gem uninstall bundler
gem install tzinfo builder memcache-client rack rack-test rack-mount 
gem install erubis mail text-format thor bundler i18n
gem install rails --pre

Now that rails 3 is getting closer to release, I wanted to start playing around with it. I've seen a few articles on getting it up and running, but they all seemed a little bit complicated. To use rails 2.3.5 before its release, I just built the gems myself and installed them. It turns out you can easily do the same with rails 3.

First, install rails main dependencies:

gem install rake rack bundler
gem install arel --version 0.2.pre

Next, get the latest rails code from github, and install the rails gems. There may be a few errors you can safely ignore:

git clone git://github.com/rails/rails.git
cd rails
rake install

And bang, you can start your first rails 3 project:

rails ~/apps/playground/rails3 

Your existing projects shouldn't be affected as rails is installed as a prerelease gem, but to be safe I'd recommend a tool like rvm to switch to a clean set of gems.