A fire drill is a method of practicing the evacuation of a building for a fire or other emergency. Generally, the emergency system (usually an alarm) is activated and the building is evacuated as though a real fire had occurred. Usually, the time it takes to evacuate is measured to ensure that it occurs within a reasonable length of time, and problems with the emergency system or evacuation procedures are identified to be remedied.

- http://en.wikipedia.org/wiki/Fire_drill

At The Conversation, we’ve been running fire drills recently. Not to evacuate a building, but to see how long it takes us to rebuild our entire production environment. Take a new VPS, and time how long it takes to get our production environment running again.

Our process for doing this has evolved over the past few weeks, to the point where we can now run a single command to provision an entire VPS and restore from our database backups.

The first fire drill

The first test was to figure out if we could actually do it, without the help of our resident systems expert. So we hosed our staging environment, and told a random developer on the team to fix it. Despite all of our documentation and existing build scripts, it took nearly 2 days.

Our documentation claimed a record of 30 minutes to accomplish this feat, so we had failed miserably.

Houston… we have a problem

We use dollhouse and babushka to provision our servers. It’s great to be able to build a full stack on a remote machine by running one command. So when we sat down to run the fire drill, we were pretty confident that anyone in our team could get things going again.

So why did it take 2 days?

Along with our build scripts, we have a document that describes all of the manual processes required to truly build a new production environment:

• How to spin up a new VM
• Updating IP addresses in the dollhouse config file
• Removing IP address from your known_hosts file
• Updating the IP address for the remote server in your git config
• How to find and restore a database backup

Like most documents, it was a little out of date, as were some of our Babushka scripts.

What should have taken 30 minutes, took 2 days of document and build script debugging.

We re-ran the fire drill, fixing problems as they arose, until we were confident in our ability to rebuild the staging environment.

Then we handed it all over to another developer, to see if we had really nailed it.

We failed again

What happened? We thought we had the build nailed, I guess we didn’t. New pieces of infrastructure had been installed, and while scripted via babushka, they hadn’t been tested in conjunction with all of the other scripts.

This time it only took a day to get everything up to date. We were getting better at this, but we weren’t yet comfortable with what we had.

We worked hard on the build scripts, wrapping our manual steps up into another script, so that we couldn’t forget a step, or mistype something. We weren’t happy until we had one command to run, passing in an IP address and environment variable, to build a full server stack, restore from a database backup and start our application.

Our new build document is much cleaner:

1. Spin up a new VM
2. $ rake server:provision IP=xxx.xxx.xxx.xxx ENV=staging
3. Drink beer

Our current setup isn’t without flaws, but we’ve learnt a lot in the past few weeks.

What did we learn?

• Documentation gets out of date.
• Build scripts get out of date.
• Automate everything, manual steps get missed.
• Fire drills should be run, all the time, to ensure correctness.

In the future, we’re hoping to set up a build on our CI server that will run nightly, completely rebuilding our staging environment from scratch.

Essentially we want an automated fire drill.

Firstly a disclaimer - this code was written as part of a funday friday. It is not on production and we wouldn’t recommend to anyone.

At The Conversation we publish hundreds of articles a month and we were thinking “It’d be great to have a way to validate links on articles prior to publication!”

We didn’t want to write any server side code and client side checking wouldn’t work due to cross site scripting issues — or would it…

The hack

We knew you could make a request using JSONP to any domain (the same way you’d load a list of tweets). We found that by using the error state you can determine whether it was a parser error (the document was found but was not JS) or some other error (the document most likely didn’t exist).

This turned out to be an excellent way to do client side link validation.

The code:

$("a").each ->
  link = $(this)
  $.ajax
    url: link.attr("href")
    dataType: "jsonp"
    jsonp: ""
    timeout: 8000
    success: (data, status) ->
      alert("SUCCESS?!!!")
    error: (XHR, textStatus, errorThrown) ->
      if textStatus == "parsererror"
        link.addClass("valid")
      else
        link.addClass("invalid")

We also found that we could rescue Script errors on window to lead to less console errors:

$(window).error (error) ->
  if error.originalEvent.message == "Script error."
    false

Putting it all together with some styling:

The styling was done exclusively with CSS (no additional markup) using content: attr(href): code

This code works in Chrome/Safari (untested in Firefox).

Owning up…

Ok so it was in production for a while.

We pulled it when we switched to SSL for editors as it caused issues with mixed content warnings. I have no idea whether it would cause security vulnerabilities (if one of the sites returned valid JS/JSONP). YMMV

For our 6 month milestone at The Conversation we wanted to do something a little bit special. The nostalgia of these occasions usually calls for some kind of imagery to jog the memory (and jerk the tears). We didn’t want to dissapoint…

The editors at The Conversation publish an amazing amount of content every day. The site is constantly growing and changing. How about a screenshot of the home page (twice a day) for the past six months? Sounds like it’s time for some crazy Ruby hax!

The basic algorithm is:

• download an hourly database backup
• restore the database
• reset the codebase to the git refspec at the time of the backup
• install the dependant gems with bundler
• start the app
• save a screenshot
• stop the app
• rinse and repeat

The backups

Thanks to some forward-thinking by Ben Hoskings, our database is backed up hourly to Cloud Files with a special filename format. For example:

tc_production-2011-09-11-12:18:02-3568a4a.psql.gz

This format is really the key to being able to pull off a stunt like this. Importantly it captures not only the time when the database was backed up, but also the git refspec of the app in production. Having these two pieces of information allows us to marry up the database and the codebase at every hour since the site was launched.

The download_backup method streams the backup (if we haven’t already downloaded it) given a Cloud Files object:

def download_backup(object)
  Dir.mkdir BACKUP_DIR unless Dir.exists?(BACKUP_DIR)
  path = File.join(BACKUP_DIR, object.name)

  puts "downloading backup to #{path}..."

  unless File.exists?(path) && Digest::MD5.file(path) == object.etag
    open(path, "w") do |file|
      object.data_stream do |chunk|
        putc '.'
        file << chunk
      end
    end
  end

  path
end

Once we’ve downloaded the backup it’s simply a matter of restoring it:

def restore_backup(path)
  puts "creating database..."
  %x{dropdb tc_development; createdb tc_development}

  puts "restoring database..."
  %x{gzip -dc #{path} | psql -U `whoami` -d tc_development >> #{LOG_PATH} 2>&1}
end

The codebase

From the backup filename we can extract the git refspec we need to be running, then it’s just a matter of a git reset hard:

def set_refspec(refspec)
  puts "setting refspec to #{refspec}..."
  %x{cd #{APP_DIR} && git reset --hard #{refspec} >> #{LOG_PATH} 2>&1}
end

Bundling the app

Bundling the app prooved trickier than expected. The problem was that bundling from within a Ruby process which is itself running bundler is like Inception: shit gets complicated. It turns out that you’ve got to shell out within a Bundler.with_clean_env block, this tells bundler to clean up its environment.

RVM was the other beast which needed taming, running everything inside a bash login subshell seemed to make it happy.

Here’s the bundle_app method:

def bundle_app
  puts "bundling..."
  Bundler.with_clean_env do
    %x{bash -lc "cd #{APP_DIR} && source ~/.rvm/scripts/rvm && rvm use 1.9.3@workspace --create && gem install bundler && pwd && bundle install >> #{LOG_PATH} 2>&1"}
  end
end

Firing up the app

The next challenge was starting the Rails server. Obviously we can’t just run it within the same process as the server will block, we need to fork it. But how will we know when Rails has started and is ready to accept requests? The answer lies in the lsof utility, one of those unix power tools with a man page so long it make your eyes glaze over.

Running lsof -Fp -i :3000 will output the PID of the process running on port 3000, which will be the Rails server when it’s booted up and ready to receive incoming connnections. If lsof returns nothing then we sleep for a second and try again.

def start_rails
  rails_pid = nil

  puts "starting rails..."

  fork do
    Bundler.with_clean_env do
      %x{bash -lc "cd #{APP_DIR} && source ~/.rvm/scripts/rvm && rvm use 1.9.2@workspace && bundle exec rails s -p #{RAILS_PORT} >> #{LOG_PATH} 2>&1"}
    end
  end

  while true
    putc "."
    rails_pid = %x{lsof -Fp -i :#{RAILS_PORT}}
    rails_pid = rails_pid[1..-1] # strip prefix 'p' char off the lsof PID output.
    break if rails_pid
    sleep 1
  end

  puts "running on pid #{rails_pid}"
  rails_pid.to_i
end

Screenshot

Once Rails is running we need to browse to the homepage and save a screenshot. Selenium is the perfect tool for the job, after we hit the homepage we fire off some javascript to resize the browser window and then save a screenshot.

def save_screenshot(path)
  Dir.mkdir SCREENSHOT_DIR unless Dir.exists?(SCREENSHOT_DIR)
  puts "saving screenshot to #{path}..."
  Selenium::WebDriver.for(:firefox).tap do |driver|
    driver.navigate.to "http://localhost:#{RAILS_PORT}/"
    driver.execute_script %Q{window.resizeTo(#{SCREENSHOT_WIDTH}, #{SCREENSHOT_HEIGHT});}
    driver.save_screenshot(path)
    driver.quit
  end
end

Post-processing

After we’ve replayed the history of the app and have a swag full of screenshots, we’re done right? Not exactly, because Selenium doesn’t crop the screenshots for you the height of the screenshot varies with the length of the content on the homepage. We need to crop every image to 1024x768, enter Mogrify. Mogrify is utility that comes with ImageMagick and is used to batch-process image files. To crop the top of every image to 1024x768, use the following:

mogrify -crop 1024x768 -gravity North *.png

The results

Watch the video.

Read the code.

We were looking around for an A/B solution for TC to answer various questions such as “Does average time on site increase if related articles are on the top of an article or the bottom?”

Of the solutions we looked at, all seemed pretty complicated so we rolled our own using Google Analytics and it came out with just a few lines of code.

The approach

Google Analytics allows you to set custom variables of which you can have 5 per visitor at various levels (page/session or visitor). By setting custom variables using controlled randomisation we could then compare behaviours with users that were presented with different options.

Firstly every user needs a uid as cookie. In application_controller

before do
  unless cookies[:uid]
    cookies.permanent[:uid] = Digest::SHA1.hexdigest(rand.to_s)
  end
end

Next in a helper add the following helpers:

def ab_test(test_name, *choices)
  choices = choices.any? ? choices : [true, false]
  get_choice(test_name, *choices)
end

def get_choice(test_name, *choices)
  user = cookies[:uid]
  choice_index = Digest::MD5.hexdigest('NaCl' + test_name.to_s + user).to_i(16) % choices.length
  choices[choice_index]
end

Then in your layout

if ab_test(:related_placement, "top", "bottom") == "top"
  # Render related articles at the top
end
...
if ab_test(:related_placement, "top", "bottom") == "bottom"
  # Render related articles at the bottom of the page
end

Finally before you call trackPageview with your google analytics code call:

_gaq.push(['_setCustomVar', 1, 'related_article_placement', '#{ab_test(:related_placement, "top", "bottom")}', 1])

This solution allowed us to easily test a couple of assumptions without deploying much additional tech. Using Google Analytics segmentation we could then slice and dice against any other metric we wanted to check.

Gotchas

Setting the custom variables needs to be done before you call trackPageview - the variables ride on trackPageview’s request.

Wouldn’t it be nice if you could write acceptance specs for your email like this?

feature 'Subscribe to email list' do

  scenario 'Successful subscription' do
    visit '/'
    fill_in 'Email', with: 'don@example.com'
    click_button 'Subscribe'
    page.should have_content 'Check your email'

    "don@example.com".should receive_email(:subject => 'Welcome')

    click_link_in_email "Confirm my subscription", :subject => 'Welcome'
    page.should have_content 'Subscription confirmed'
  end

end

Yes, yes it would. Here is how you do it. Drop this in your spec support folder:

module AcceptanceEmailHelpers

  # Finds the latest email to match the given options, locates an HTML link
  # matching the given text, and visits it. Options are not validated - they
  # are passed straight through to the found mail object.
  def click_link_in_email(text, options)
    mail = find_email_matching(options)
    unless mail
      fail "Could not locate mail matching: #{options.inspect}"
    end

    element = Nokogiri::HTML(mail.body.to_s).css('a').detect {|x| x.text == text }
    unless element
      fail "Could not find link with text '#{text}' in:\n\n#{mail.body.to_s}"
    end

    visit strip_hostname(element.attribute('href').value)
  end

  def strip_hostname(url)
    uri = URI.parse(url)
    if uri.host.blank?
      raise "Host was not set in email url => #{url}"
    end
    [uri.path, uri.query].compact.join('?')
  end

  def find_email_matching(options)
    options[:from] = extract_plain_email(options[:from]) if options[:from]
    options[:to]   = extract_plain_email(options[:to]) if options[:to]

    ActionMailer::Base.deliveries.reverse.detect do |mail|
      options.all? do |method, value|
        [*mail.send(method)].any? {|x| x.include?(value) }
      end
    end
  end

  # Emails can look like "bill@example.com" or "Bill <bill@example.com>", hence
  # grab only the plain email address bit
  def extract_plain_email(email)
    email[/([\w\.%\+\-]+)@([\w\-]+\.)+([\w]{2,})/i]
  end

  # Called on an Author, Editor, User, or Institution. Will regexp match all the
  # given fields against any email delivered to that user. Supported fields:
  #   :subject
  #   :body
  RSpec::Matchers.define :receive_email do |options = {}|
    match do |recipient|
      find_email_matching(options_with_recipient(options, recipient))
    end

    failure_message_for_should do |recipient|
      <<-EOS
        Expected to receive an email matching #{options_with_recipient(options, recipient).inspect}"}
        Received:
          #{ActionMailer::Base.deliveries.map {|x| "#{x.to.join(', ')}: #{x.subject}" }.join("\n  ")}
      EOS
    end

    failure_message_for_should_not do |recipient|
      <<-EOS
        Expected not to receive an email matching #{options_with_recipient(options, recipient).inspect}"}
        Received:
          #{ActionMailer::Base.deliveries.map {|x| "#{x.to.join(', ')}: #{x.subject}" }.join("\n  ")}
      EOS
    end

    def options_with_recipient(options, recipient)
      options.merge(to: email_for_recipient(recipient))
    end

    def email_for_recipient(recipient)
      if recipient.is_a?(String)
        recipient
      else
        recipient.email
      end
    end
  end
end

RSpec.configuration.include EmailHelpers

There’s a bit of complexity in there but it’s all helpful. It’s a veritable treasure chest.

You’ve profiled your application, you’re running a patched version of ruby, and still your application just doesn’t start up fast enough. What’s a poor developer to do? At The Conversation, our start up time has been hovering at a mite over 10 seconds. I know it’s symbolic, but I really want it below 10. It’s taunting me.

At RailsCamp last weekend I had a play around with an idea I had for eeking out a bit more speed: optimizing the load path. When you require a file, ruby scans the load path sequentially looking for the file you asked for. If you moved the most common file locations to the top of the load path, that should speed things up, right?

Analysis

The first step is to analyze the files loaded by your application to determine the most popular load paths. Loaded files are stored in $LOADED_FEATURES, and it’s not difficult to work backwards to the load path:

class LoadPathAnalyzer

  def initialize(load_path, loaded_features)
    @load_path       = load_path
    @loaded_features = loaded_features
  end

  def frequencies
    load_paths.inject({}) {|a, v|
      a[v] ||= 0
      a[v] += 1
      a
    }
  end

  private

  def load_paths
    @loaded_features.map {|feature|
      @load_path.detect {|path|
        feature[0, path.length] == path
      }
    }.compact
  end

end

We need to store this information for later use. I dump it into a file in config using this rake task:

desc "Recalculate $LOAD_PATH frequencies."
task :recalculate_loaded_features_frequency => :environment do
  require 'load_path_analyzer'

  frequencies     = LoadPathAnalyzer.new($LOAD_PATH, $LOADED_FEATURES).frequencies
  ideal_load_path = frequencies.to_a.sort_by(&:last).map(&:first)

  File.open(IDEAL_LOAD_PATH_FILE, "w") do |f|
    f.puts ideal_load_path
  end
end

Application

Now we need to reshuffle our load path at an opportune time before Bundler kicks in and requires everything. The top of config/application.rb directly beforehand is a great spot.

# config/application.rb
require File.expand_path('../boot', __FILE__)

require 'rails/all'

IDEAL_LOAD_PATH_FILE = "config/ideal_load_path"

if File.exists?(IDEAL_LOAD_PATH_FILE)
  order = File.open(IDEAL_LOAD_PATH_FILE).lines.map(&:chomp)
  $LOAD_PATH.sort_by! {|x| order.index(x).to_i * -1 }
end

# If you have a Gemfile, require the gems listed there, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(:default, Rails.env) if defined?(Bundler)

# And so on...

Results

Does it work?

# Before
> time ruby -r./config/environment.rb -e ''
8.20s user 2.05s system 99% cpu 10.272 total

# After
> time ruby -r./config/environment.rb -e ''
7.91s user 1.62s system 99% cpu 9.560 total

The difference wasn’t as dramatic as I’d hoped, but still a tidy 700 milliseconds. Most importantly though, it got us under 10 seconds!

Note that on a new application you’ll barely notice any difference since it won’t require many files. If your application has been around the block a few times however, try it out and let us know how you go.

I just upgraded DataMapper from 1.0 to 1.1 in our app.

DataMapper 1.1 brings several minor API changes, warranting the minor version bump, and closes 52 tickets in Lighthouse. There have been many performance improvements, some closing bottlenecks that result in as much as a 20x speedup from the 1.0.2 behaviour.

It took me about three hours, and most of that was waiting for bundle update and builds to run. We were running our own forks of a few of the 1.0 gems as well to get some much needed bug fixes, so that added a bit of time also. Here are the major changes I needed to make.

Get real internet

My wireless kept dropping out doing a big bundle update. Got fed up and found an ethernet cable and then it all worked. Far quicker. There is a larger lesson in here somewhere.

DataMapper no longer uses ActiveSupport::Inflector

DataMapper uses it’s own Inflector class, thankfully it appears to be mostly API compatible with ActiveSupport. Our config/initializers/inflections.rb file now looks like this:

[
  ActiveSupport::Inflector,
  DataMapper::Inflector
].each do |klass|
  klass.inflections do |inflect|
    inflect.uncountable %w( research content published )
  end
end

DataMapper no longer monkey-patches core classes

Previously DataMapper monkey-patched many core classes to add convenience methods such as String#compress_lines and Hash#only. This caused no end of conflicts with ActiveSupport and others, so with 1.1 these extensions were moved into explicit classes.

"Hello\nthere".compress_lines                          # old
DataMapper::Ext::String.compress_lines("Hello\nthere") # new

We had a few only calls in our app, which can be replaced with slice (provided by ActiveSupport). I also had to patch dm-paperclip and database_cleaner, since they were depending on the old extensions.

Always use ZonedTime

We had a few (incorrect) uses of Property::DateTime in our models. This is provided by DataMapper out of the box, but does not work with Rails timezones. Instead, you should use ZonedTime, provided by dm-zone-types. Don’t forget to write a data migration to get your dates into UTC time in the database!

Use a newer version of dm-do-adapter

The 1.1.0 gem release of dm-do-adapter contains a deprecated call to DataObjects::URI.new. This isn’t broken, it’s just noisy. It will be fixed in the soon to be released 1.1.1, but to silence it now, pick up a newer version from git. Add this to your Gemfile:

# We can technically use the 1.1 gem, but it throws up deprecation warnings.
# This ref is the commit the warnings were silenced, but is before the DM
# dependencies were bumped to 1.1.1 (which hasn't been released yet)
gem 'dm-do-adapter',
  git: 'git://github.com/datamapper/dm-do-adapter',
  ref: '7f0b53d1ada8735910e0'

Update forks

Not all of the patches and bug fixes we rely on made it into 1.1, so I rebased our forks against the 1.1 tag.

$ cd dm-core-tc
$ git add remote dm git://github.com/datamapper/dm-core.git
$ git fetch
$ git checkout -b tc-1.1
$ git rebase v1.1.0
$ git push origin tc-1.1

There were a few duplicates and faux-conflicts, so I made liberal use of the --skip option to rebase. In our Gemfile, we reference the fork like so:

gem 'dm-core', 
  git:    'git://github.com/conversation/dm-core',
  branch: 'tc-1.1'

That’s basically it. It wasn’t a huge deal. You should do it too.

Sprinkled throughout our code are some little helpers that check the app is working as expected. These little helpers implement the Design By Contract (DBC) style of fail often, fail fast.

The DBC helpers act similar to the assert macro in the C programming language, where the failing of a DBC check represents a bug, and the exception raised would not normally be caught by the app (i.e. the app would stop running).

An example of a helper is:

DBC.require(check, msg = "")

A classic example of a DBC.require check would be in a square_root function.

def Math.square_root(x)
  DBC.require(x >= 0, "Cannot get the square root a number less than zero")

  # Code to calculate square root
  # ...
end

This particular helper (require) does a sanity check on the parameters passed into the method in question. Like most DBC methods, the require method throws an exception if the check does not evaluate to true. Some examples in our code are:

DBC.require(content.state != :published, "Cannot delete published content")

DBC.require(record.valid?, record.errors.inspect)

Why use DBC instead of raise?

Design By Contract has some nice advantages over using raise:

• You get a nice little DSL that formalizes the raising of exceptions within an app (see The DBC methods below)
• The DSL provides a wrapper that does whatever extra error-checking, tracing, etc you wants to do. For instance, our DBC prints out a stack trace, where as if we used raise we would be at the mercy of particular shell, irb, passengers etc as to whether a stack trace is shown.
• DBC throws its own specific exceptions, so you can treat them separate to other exceptions.

On top of that, DBC helps highlight the different parts of what you can be checking at the method level.

The DBC methods

We use 4 different versions of DBC helpers throughout the code, which are:

DBC.require(check, msg = "")
DBC.assert(check, msg = "") 
DBC.ensure(check, msg = "") 

DBC.fail(msg = "")

DBC.require

Written at the top of methods, use it to check:

• Was the method called with the right state?
• Was the method called with the right parameters?

DBC.assert

Written half-way through a method, use it to check:

• Is the method currently doing the right thing?

Note: One place I consistently use DBC.assert is within Rails migrations that use native ruby code.

DBC.ensure

Written at the end of methods, use it to check:

• Is the method returning the right values?

• Has the method left the app in the right state?

  Note: DBC.ensure is not often used as the cost of doing a check can often be the cost of re-doing the method.

DBC.fail

Insert this where you know the app must now be in an invalid state. For example, it can be useful in the else part of a ruby case statement, when an unexpected value has come through:

Credits

The Design By Contract pattern was first shown to me by Dan Prager. Thanks Dan!

The code

# Design By Contract pattern
class DBC 
  class PreconditionException < RuntimeError; end
  class AssertconditionException < RuntimeError; end
  class PostconditionException < RuntimeError; end
  class FailException < RuntimeError; end

  def self.require(condition, message = "")
    unless condition
      error(PreconditionException, message, caller)
    end
  end

  def self.assert(condition, message = "")
    unless condition
      error(AssertconditionException, message, caller)
    end
  end

  def self.ensure(condition, message = "")
    unless condition
      error(PostconditionException, message, caller)
    end
  end

  def self.fail(message = "")
    error(FailException, message, caller)
  end

  private 

  def self.error(klass, message, caller)
    raise klass.new("#{klass.name}-condition failed: #{message}\nTrace was: #{caller.join("\n")}")
  end
end

Our app takes no small amount of time to start up. This appears to be par for the course for Rails on 1.9.2, but Pete (on loan from Envato for a few days, hooray for dev share!) and I did a quick spike to see if there was anything specific to our environment causing unnecessary delays.

There are a number of ways to approach this problem, of which in this post I will talk about just one: ruby-prof. You can use it unobtrusively to get a call graph for your application.

gem install ruby-prof
ruby-prof --printer=graph_html ./script/rails runner > profile.html

This will show large amounts of time spent in requires and initializers. That is dissapointing, but to be expected. We were looking for something more anomolaus. And we found it:

40.45%  11.83%     22.35      6.54      0.00     15.81          1962610     Pathname#to_s

Pathname#to_s was being called millions of times and taking more than its fair share of the cake. To confirm this assumption we also ran ruby-prof against a blank Rails app, which only called Pathname#to_s a measly 59 times.

The next step was to find what was calling our rogue method so many times. There are no doubt elegant ways to do this, but we are not above a quick hack. Copying the source of the method from the rubydocs, we added some extra diagnostic information and pasted the new definition in config/environment.rb above any other requires.

class Pathname
  def to_s
    puts
    puts
    puts 
    puts @path
    puts caller
    @path.dup
  end
end

Of course, letting our startup process run to completion would create a lot of output, but CTRL+C a few moments in should give us enough information to notice any recurring patterns.

Sure enough, some paths showed up repeatedly. (I haven’t included the stack traces because they turned out not to be relevant.)

/Volumes/conversation/tc/lib
/Volumes/conversation/tc/app

lib, app, lib, app, over and over and over again. That smells like our load path.

# config/environment.rb
module TheConversation
  class Application < Rails::Application
    config.autoload_paths << Rails.root.join("app")
    config.autoload_paths << Rails.root.join("lib")
  end
end

That looks like it! Let’s convert them to strings before adding them.

# config/environment.rb
module TheConversation
  class Application < Rails::Application
    config.autoload_paths << Rails.root.join("app").to_s
    config.autoload_paths << Rails.root.join("lib").to_s
  end
end

Sure enough this cut a good few seconds off our load time. ruby-prof, I owe you a beer.

A little over a year ago I met Jack Rejtman and Andrew Jaspan, two guys who had a big idea. Knowledge of a what needed to exist but less of a technical perspective on how to get there.

Fast forward to the 24th of March 2011 and we have just released The Conversation — a not-for-profit media service featuring professional editors working with academics to produce high quality analysis and comment as well as covering the latest news and research.

The site is built using Rails 3 (ruby) with a dash of Node.js for real time chat between authors, and a python service from Google called mobwrite that powers realtime collaboration.

All of the javascript is written in CoffeeScript and our CSS is written in SCSS, making extensive use of partials.

We deploy using Git/Babushka, test using Jenkins and send in-app email via Postmarkapp.

The whole site is backed by PostgreSQL 9 which is backed up regularly to Rackspace Cloudfiles, a service that also hosts our images.

Through this blog we hope to cover the whys and the hows as a way of giving back to the open source community that has helped this project so much. If you have any requests for articles get in touch.

Mike, Xav, Ben, Mat & Justin.