I'm happy to announce that Little Stream Software is now offering commercial support for Redmine.

If your organization runs Redmine and needs more support than the open source community support can provide, I'd be happy to have you as a client. I'm offering four different support plans to choose from, including a simple single support plan to get help with a specific problem.

Since it's just me at Little Stream Software, I'm putting a limit on the number of support clients I accept and will stop the service I'm full. I'd rather do a killer job for a few clients than a mediocre job for many clients.

Feel free to contact me or leave a comment if you have any questions.

It's time I share my secret formula I've been using to be successful:

Do one thing each day to grow towards who you want to become

(it's a common secret)

That's it. No magic 12 step program, no killer coaching system (though they will give you direction and accelerate the learning). Just one improvement after another, one step at a time.

A lot of people get hung up on picking the right step. This causes them to start the "journey" to find the right step, six months later they can finally get started.

What I recommend is to take one step, any step everyday. The worst case is you went the wrong way but now you know a better way and can adjust.

After looking at the other RestClient::Resource methods from yesterday, I noticed that they are all almost identical to #get. So today I'm going to dig into the code one level down, RestClient::Request.

The Code

RestClient::Request#execute

module RestClient
  class Request
    def self.execute(args, &block)
      new(args).execute &block
    end
  end
end

RestClient::Resource#get calls this method, which is just a wrapper for creating a new Request object and calling it's #execute method.

RestClient::Request#initialize

    def initialize args
      @method = args[:method] or raise ArgumentError, "must pass :method"
      @url = args[:url] or raise ArgumentError, "must pass :url"
      @headers = args[:headers] || {}
      @cookies = @headers.delete(:cookies) || args[:cookies] || {}
      @payload = Payload.generate(args[:payload])
      @user = args[:user]
      @password = args[:password]
      @timeout = args[:timeout]
      @open_timeout = args[:open_timeout]
      @raw_response = args[:raw_response] || false
      @verify_ssl = args[:verify_ssl] || false
      @ssl_client_cert = args[:ssl_client_cert] || nil
      @ssl_client_key = args[:ssl_client_key] || nil
      @ssl_ca_file = args[:ssl_ca_file] || nil
      @tf = nil # If you are a raw request, this is your tempfile
      @processed_headers = make_headers headers
      @args = args
    end

#initialize just checks the args and sets a bunch of attributes. The :method and :url attributes are required, which if you remember #get sets automatically. It's also good that the original arguments are stored away into @args, I always forget to do that.

RestClient::Request#execute

    def execute &block
      uri = parse_url_with_auth(url)
      transmit uri, net_http_request_class(method).new(uri.request_uri, processed_headers), payload, &block
    end

After initialize, the class method #execute calls the object method #execute above. There is a lot happening in these short lines so I need to break it out a bit:

• the request uri is parsed out using parse_url_with_auth.
• net_http_request_class converts the :get request into Get which is an actual [net/http](http://www.ensta.fr/~diam/ruby/online/ruby-doc-stdlib/libdoc/net/http/rdoc/classes/Net/HTTP.html) class.
• then a new Net/HTTP object is created, given the uri and request headers.
• finally the uri, Net/HTTP request object, payload, and block is passed to #transmit

RestClient::Request#transmit

    def transmit uri, req, payload, &block
      setup_credentials req

      net = net_http_class.new(uri.host, uri.port)
      net.use_ssl = uri.is_a?(URI::HTTPS)
      if @verify_ssl == false
        net.verify_mode = OpenSSL::SSL::VERIFY_NONE
      elsif @verify_ssl.is_a? Integer
        net.verify_mode = @verify_ssl
      end
      net.cert = @ssl_client_cert if @ssl_client_cert
      net.key = @ssl_client_key if @ssl_client_key
      net.ca_file = @ssl_ca_file if @ssl_ca_file
      net.read_timeout = @timeout if @timeout
      net.open_timeout = @open_timeout if @open_timeout

      RestClient.before_execution_procs.each do |before_proc|
        before_proc.call(req, args)
      end

      log_request

      net.start do |http|
        res = http.request(req, payload) { |http_response| fetch_body(http_response) }
        log_response res
        process_result res, &block
      end
    rescue EOFError
      raise RestClient::ServerBrokeConnection
    rescue Timeout::Error
      raise RestClient::RequestTimeout
    end
  end
end

Now I've found where the request is sent.

This is one of the core methods of RestClient. It starts with a bit of setup for the credentials and SSL. Sending the actual request is pretty simple and just uses net/http and a few helper methods:

      net.start do |http|
        res = http.request(req, payload) { |http_response| fetch_body(http_response) }
        log_response res
        process_result res, &block
      end

It also looks like transmit is use two separate net/http objects; req and net. I'm not really sure why they needed two objects, especially since only one of them is actually used to connect to the server.

Now that I've seen the Request side, I think I'll take a look at the response side.

From yesterday's code reading, I found that the restclient command uses RestClient::Resource for most of the work. So I'm going to start with it's methods, starting with it's HTTP verbs, #get.

The Code

module RestClient
  class Resource
    def get(additional_headers={}, &block)
      headers = (options[:headers] || {}).merge(additional_headers)
      Request.execute(options.merge(
              :method => :get,
              :url => url,
              :headers => headers), &(block || @block))
    end
  end
end

Review

The #get method takes a hash of HTTP headers and a block. Since a Resource object is created with the url already, it's doesn't need to be passed into this method.

The first thing #get does is to merge the HTTP headers from it's initialization (options[:headers]) and the additional headers from the method call. This way you can create one Resource object with common headers and then override them in each request as needed. Next, #get passes all of the data onto RestClient::Request#execute which I'm assuming would build and sent the actual HTTP request.

One interesting technique that is in #get is how it passes either the method's block or initializer's block into Request#execute. By checking for a nil object and using the & outside the parenthesis, this is a simple one liner.

&(block || @block)

This week I'm going to read through the code for RestClient. RestClient is a simple HTTP and REST client for Ruby that can be used to consume RESTful resources. I've used it in a Single Sign On system I built for Redmine, redmine_sso_client and redmine_sso_server.

Today I started with the command line binary for RestClient. There are two things the binary does, creates a request from the command line augments or opens an IRB shell to run commands interactively.

The Code for creating a request

def r
    @r ||= RestClient::Resource.new(@url, @username, @password)
end

if @verb
    begin
        if %w(put post).include? @verb
            puts r.send(@verb, STDIN.read)
        else
            puts r.send(@verb)
        end
        exit 0
    rescue RestClient::Exception => e
        puts e.response.body if e.respond_to? :response
        raise
    end
end

This code is run after the initialization blocks in restclient and it handles sending the request to the server. @verb should be 'get', 'post', 'put', or 'delete' so they are sent directly into the RestClient::Resource as method calls (e.g. r.send('get')). put and post methods will also pass in the standard input as additional headers for the request.

On a successful request, the response will be printed by the puts. Invalid or unsuccessful requests will raise a RestClient::Exception which will print out the response body and then exit the program.

The Code for the interactive IRB shell

The following three code snippets create RestClient's interactive IRB shell.

%w(get post put delete).each do |m|
 eval <<-end_eval
def #{m}(path, *args, &b)
   r[path].#{m}(*args, &b)
end
 end_eval
end

This bit of metaprogramming creates four methods that are proxied to the RestClient::Resource object: #get, #post, #put, and #delete. This will let the shell have more of a DSL feel to it.

def method_missing(s, *args, &b)
    super unless r.respond_to?(s)
    begin
        r.send(s, *args, &b)
    rescue RestClient::RequestFailed => e
        print STDERR, e.response.body
        raise e
    end
end

Using method_missing RestClient is able to proxy all of the other methods to the RestClient::Resource object, as long as that object responds to that method. So calling user in the shell will really call RestClient::Resource#user.

require 'irb'
require 'irb/completion'

if File.exists? ".irbrc"
    ENV['IRBRC'] = ".irbrc"
end

if File.exists?(rcfile = "~/.restclientrc")
    load(rcfile)
end

ARGV.clear

IRB.start
exit!

Finally, the binary loads several optional configuration files and starts IRB.

What I found interesting was how simple it was to add a custom shell using IRB. Using this code, I might be able to do something similar with redmine_client to create a "shell" for Redmine.

I'm finishing up my code reading of Capistrano's recipes with the recipe that is run most often, deploy.

The Code and Review

deploy

namespace :deploy do
  task :default do
    update
    restart
  end
end

The deploy recipe itself is simple and just runs deploy:update and deploy:restart.

deploy:update

namespace :deploy do
  task :update do
    transaction do
      update_code
      symlink
    end
  end
end

deploy:update uses a transaction to make sure both deploy:update_code and deploy:symlink run successfully.

deploy:update_code

namespace :deploy do
  task :update_code, :except => { :no_release => true } do
    on_rollback { run "rm -rf #{release_path}; true" }
    strategy.deploy!
    finalize_update
  end
end

Now deploy:update_code starts to do some interesting work. First it defines an on_rollback so Capistrano knows how to revert the code updates. Next it calls Capistrano::Deploy::Strategy#deploy! to do the code update. This class provides a standard interface for how each of the different :deploy_via options work on the remote servers. Finally, deploy:finalize_update is called.

deploy:finalize_update

namespace :deploy do
  task :finalize_update, :except => { :no_release => true } do
    run "chmod -R g+w #{latest_release}" if fetch(:group_writable, true)

    # mkdir -p is making sure that the directories are there for some SCM's that don't
    # save empty folders
    run <<-CMD
      rm -rf #{latest_release}/log #{latest_release}/public/system #{latest_release}/tmp/pids &&
      mkdir -p #{latest_release}/public &&
      mkdir -p #{latest_release}/tmp &&
      ln -s #{shared_path}/log #{latest_release}/log &&
      ln -s #{shared_path}/system #{latest_release}/public/system &&
      ln -s #{shared_path}/pids #{latest_release}/tmp/pids
    CMD

    if fetch(:normalize_asset_timestamps, true)
      stamp = Time.now.utc.strftime("%Y%m%d%H%M.%S")
      asset_paths = %w(images stylesheets javascripts).map { |p| "#{latest_release}/public/#{p}" }.join(" ")
      run "find #{asset_paths} -exec touch -t #{stamp} {} ';'; true", :env => { "TZ" => "UTC" }
    end
end

deploy:finalize_update does a few housecleaning tasks to make sure all of the files are setup correctly:

1. Changes the release files to be group writable if the configuration option for :group_writable is set.
2. Removes the log, system, and tmp/pids directories, which should be symlinks.
3. Creates a public and tmp directory in the latest release.
4. Symlinks the shared log, system, and pids directories into the latest release.
5. Finally, it updates the timestamps of the public assets. This is done so when Rails links to assets they will have a newer url.

Next deploy:finalize_update and deploy:update_code return control back to deploy:update which runs deploy:symlink next.

deploy:symlink

namespace :deploy do
  task :symlink, :except => { :no_release => true } do
    on_rollback do
      if previous_release
        run "rm -f #{current_path}; ln -s #{previous_release} #{current_path}; true"
      else
        logger.important "no previous release to rollback to, rollback of symlink skipped"
      end
    end

    run "rm -f #{current_path} && ln -s #{latest_release} #{current_path}"
  end

deploy:symlink is pretty simple:

1. It defines an on_rollback to revert errors.
2. It removes the current path (remember that it's just a symlink to the deployed release directory).
3. It creates a symlink from the latest release (being deployed) to the current path, thus "moving" the symlink target.

Now deploy:update is done so it returns back to deploy which now runs deploy:restart.

deploy:restart

namespace :deploy do
  task :restart, :roles => :app, :except => { :no_release => true } do
    warn "[DEPRECATED] `deploy:restart` is going to be changed to Passenger mod_rails' method after 2.5.9 - see http://is.gd/2BPeA"
    try_runner "#{current_path}/script/process/reaper"
  end
end

The try_runner method is just a simple wrapper that will try to run a command using sudo but as a different user. In this case, Capistrano is running Rails' reaper script, which is supposed to restart the application servers. I haven't used reaper scripts for a long time, I think pre Rails 1.0 days. Instead I just override deploy:restart with my own configuration.

# Example deploy:restart for Passenger
#
namespace :deploy do
  desc "Restart Application"
  task :restart, :roles => :app do
    run "touch #{current_path}/tmp/restart.txt"
  end
end

This concludes my code reading on the Capistrano codebase. I looked at how the cap command is translated into the recipes and then examined a few of the common Capistrano recipes and what commands they run.

Next week, I'll be starting to read through a new project. Which one would you be interested in:

• AWS::S3 gem
• rest-client
• Sinatra
• Moonshine
• RailsBestPractices
• flog
• reek
• roodi

During my last code reading of Capistrano, I traced the deploy recipe to find out what other recipes it uses. Since overriding and hooking into these recipes are used when you customize your application deploy, I wanted to document them here so I can find it again.

deploy
|-- deploy:update
|---- deploy:update_code
|------ Code update based on the :deploy_via
|------ deploy:finalize_update
|---- deploy:symlink
|-- deploy:restart

All of these recipes come from lib/capistrano/recipes/deploy.rb.

I'm building a compilation of my daily refactorings into a new product. It will include additional content that isn't available here but I wanted to get some ideas about what you'd like to see added. If you answer the short survey below, I'll give you a free copy of the product when it's ready. I'll be pulling the form offline after I get enough questions.

Thanks

Eric Davis

The survey is closed. Thank you everyone for your feedback.

Today I'm reading through Capistrano's deploy:setup recipe. This recipe is used to setup a new server for deployment by creating the directories Capistrano uses and checking permissions.

The Code

namespace :deploy do
  desc <<-DESC
    Prepares one or more servers for deployment. Before you can use any \
    of the Capistrano deployment tasks with your project, you will need to \
    make sure all of your servers have been prepared with `cap deploy:setup'. When \
    you add a new server to your cluster, you can easily run the setup task \
    on just that server by specifying the HOSTS environment variable:

      $ cap HOSTS=new.server.com deploy:setup

    It is safe to run this task on servers that have already been set up; it \
    will not destroy any deployed revisions or data.
  DESC
  task :setup, :except => { :no_release => true } do
    dirs = [deploy_to, releases_path, shared_path]
    dirs += shared_children.map { |d| File.join(shared_path, d) }
    run "#{try_sudo} mkdir -p #{dirs.join(' ')} && #{try_sudo} chmod g+w #{dirs.join(' ')}"
  end

Review

(If you are new to Capistrano, my post on Capistrano variables will help clear up what each variable means)

deploy:setup begins by building a list of directories to get created:

• deploy_to path
• releases path
• shared path

Then it adds the shared children to these, which include:

• shared/system
• shared/log
• shared/pids

run "#{try_sudo} mkdir -p #{dirs.join(' ')} && #{try_sudo} chmod g+w #{dirs.join(' ')}"

Finally, deploy:setup creates all of those directories (mkdir -p) and changes the permissions to be group writable (chmod g+w).

I've used deploy:setup and Capistrano for years now but I never realized how simple this recipe really was. Reading through the code has given me the understanding of what's happening and the confidence to be able to change things that don't suit my applications.

While reading through Capistrano's code I've come across many the variables used to configure deploy.rb. So I don't forget them when I need them, I'll document them here:

• application - required
• repository - required
• scm - defaults to :subversion
• deploy_via - defaults to :checkout
• revision - defaults to the latest head version
• source - Capistrano::Deploy::SCM object
• real_revision
• strategy - Capistrano::Deploy::Strategy object defined by deploy_via
• release_name - timestamp in the form of "%Y%m%d%H%M%S"
• version_dir - defaults to 'releases'
• shared_dir - defaults to 'shared'
• shared_children - defaults to ['system', 'log', 'pids']
• current_dir - defaults to 'current'
• releases_path - path of deploy_to + version_dir (e.g. /u/apps/example/releases )
• shared_path - path of deploy_to + shared_dir (e.g. /u/apps/example/shared )
• current_path - path of deploy_to + current_dir (e.g. /u/apps/example/current )
• release_path - path of releases_path + release_name (e.g. /u/apps/example/releases/20100624000000 )
• releases - list of releases, found by running ls -x
• current_release - path of releases_path + last release (e.g. /u/apps/example/releases/20100624000000 )
• previous_release - path of releases_path + previous release (e.g. /u/apps/example/releases/20100623000000 )
• current_revision - revision id of the current release, found in the REVISION file
• latest_revision - revision id of the latest release
• previous_revision - revision id of the previous release
• run_method - either :run or :sudo depending on if :use_sudo is set
• latest_release - release path or the current release depending on if the current symlink is valid

All of these variables come from lib/capistrano/recipes/deploy.rb.