Continuing from my tour of Capistrano's internals, I'm going to start looking through Capistrano's recipes now. A recipe for Capistrano is a unit of work, similar to how Rake uses tasks. I'm going to start with the basic invoke recipe today.

The Code

desc <<-DESC
  Invoke a single command on the remote servers. This is useful for performing \
  one-off commands that may not require a full task to be written for them. \
  Simply specify the command to execute via the COMMAND environment variable. \
  To execute the command only on certain roles, specify the ROLES environment \
  variable as a comma-delimited list of role names. Alternatively, you can \
  specify the HOSTS environment variable as a comma-delimited list of hostnames \
  to execute the task on those hosts, explicitly. Lastly, if you want to \
  execute the command via sudo, specify a non-empty value for the SUDO \
  environment variable.

  Sample usage:

    $ cap COMMAND=uptime HOSTS=foo.capistano.test invoke
    $ cap ROLES=app,web SUDO=1 COMMAND="tail -f /var/log/messages" invoke
DESC
task :invoke do
  command = ENV["COMMAND"] || ""
  abort "Please specify a command to execute on the remote servers (via the COMMAND environment variable)" if command.empty?
  method = ENV["SUDO"] ? :sudo : :run
  invoke_command(command, :via => method)
end

Review

invoke is pretty straight forward. It:

1. Extracts the command to be run from the COMMAND environment variable
2. Checks that command isn't empty
3. Decides if the command should be run as the normal user or with sudo
4. Finally invokes the command

Bases on this, the invoke recipe is just a thin wrapper around Capistrano::Configuration::Actions::Invocation#invoke_command. And #invoke_command itself is just a dispatcher to other methods in Capistrano:

def invoke_command(cmd, options={}, &block)
  options = options.dup
  via = options.delete(:via) || :run
  send(via, cmd, options, &block)
end

I like how #invoke_command is using send to let the user control how the command is invoked. By default it's called with :run which will call the run method. But back in the invoke recipe, if SUDO is set then the command will be run using the sudo method. I've used a similar pattern in my code, where I need to call a method based on some dynamic input.

I'm reading through Capistrano's code this week and decided to start with something different. Instead of jumping right into different methods, I'm going to review the overall flow to get an understanding of how one part works. For Capistrano, I'm going to figure out how cap deploy works.

#!/usr/bin/env ruby

require 'capistrano/cli'
Capistrano::CLI.execute

Capistrano includes a cap script which just loads a Capistrano::CLI class.

module Capistrano
  class CLI
    # ...
    # Mix-in the actual behavior
    include Execute, Options, UI
    include Help # needs to be included last, because it overrides some methods
  end
end

Capistrano::CLI doesn't define an #execute method but it does include one as a helper so I'll have to look there for the behavior.

module Capistrano
  class CLI
    module Execute
      module ClassMethods
        def execute
          parse(ARGV).execute!
        end
      end
      # ...
    end
  end
end

Capistrano::CLI::Execute#execute still isn't doing much, it's just chaining #parse and #execute!. It looks like Capistrano::CLI::Options defines the #parse method so I'll jump over to there now.

module Capistrano
  class CLI
    module Options
      module ClassMethods
        # Return a new CLI instance with the given arguments pre-parsed and
        # ready for execution.
        def parse(args)
          cli = new(args)
          cli.parse_options!
          cli
        end
      end
      # ...
    end
  end
end

Now I'm starting to get somewhere. #parse will create a new instance of Capistrano::CLI and parse it's options. Since Capistrano::CLI::Options was mixed into Capistrano::CLI, the #new method is called on Capistrano::CLI and not Capistrano::CLI::Options.

module Capistrano
  class CLI
    def initialize(args)
      @args = args.dup
      $stdout.sync = true # so that Net::SSH prompts show up
    end
  end
end

Back in Capistrano::CLI, the only significant thing that #initialize does is to store the command line arguments into @args. So the control flow returns back in Capistrano::CLI::Options and calls #parse_options!

module Capistrano
  class CLI
    module Options
      def parse_options! #:nodoc:
        @options = { :recipes => [], :actions => [],
          :vars => {}, :pre_vars => {},
          :sysconf => default_sysconf, :dotfile => default_dotfile }

        if args.empty?
          warn "Please specify at least one action to execute."
          warn option_parser
          exit
        end

        option_parser.parse!(args)

        coerce_variable_types!

        # if no verbosity has been specified, be verbose
        options[:verbose] = 3 if !options.has_key?(:verbose)

        look_for_default_recipe_file! if options[:default_config] || options[:recipes].empty?
        extract_environment_variables!

        options[:actions].concat(args)

        password = options.has_key?(:password)
        options[:password] = Proc.new { self.class.password_prompt }
        options[:password] = options[:password].call if password
      end
    end
  end
end

There is a lot of code in here but the only things it's doing is to set and check different configuration options. So the control flow returns back to Capistrano::CLI::Options#parse which returns the configured Capistrano::CLI object back to Capistrano::CLI::Execute#execute which calls Capistrano::CLI##execute!.

module Capistrano
  class CLI
    module Execute
      # Using the options build when the command-line was parsed, instantiate
      # a new Capistrano configuration, initialize it, and execute the
      # requested actions.
      #
      # Returns the Configuration instance used, if successful.
      def execute!
        config = instantiate_configuration(options)
        config.debug = options[:debug]
        config.dry_run = options[:dry_run]
        config.preserve_roles = options[:preserve_roles]
        config.logger.level = options[:verbose]

        set_pre_vars(config)
        load_recipes(config)

        config.trigger(:load)
        execute_requested_actions(config)
        config.trigger(:exit)

        config
      rescue Exception => error
        handle_error(error)
      end
    end
  end
end

Now I'm getting into Capistrano's processing. Execute is doing a few things to run the action:

• creating a local version of the configuration (config object)
• loading the recipes (load_recipes(config))
• running load callback (config.trigger(:load))
• executing the action (execute_requested_actions(config))
• running the exit callback (config.trigger(:exit))

module Capistrano
  class CLI
    module Execute
      # ...
      def execute_requested_actions(config)
        Array(options[:vars]).each { |name, value| config.set(name, value) }

        Array(options[:actions]).each do |action|
          config.find_and_execute_task(action, :before => :start, :after => :finish)
        end
      end
    end
  end
end

Following the request into #execute_requested_actions I see that the processing of the action is getting delegated to Capistrano::Configuration#find_and_execute_task.

module Capistrano
  class Configuration
    # The logger instance defined for this configuration.
    attr_accessor :debug, :logger, :dry_run, :preserve_roles

    def initialize(options={}) #:nodoc:
      @debug = false
      @dry_run = false
      @preserve_roles = false
      @logger = Logger.new(options)
    end

    # make the DSL easier to read when using lazy evaluation via lambdas
    alias defer lambda

    # The includes must come at the bottom, since they may redefine methods
    # defined in the base class.
    include Connections, Execution, Loading, Namespaces, Roles, Servers, Variables

    # Mix in the actions
    include Actions::FileTransfer, Actions::Inspect, Actions::Invocation

    # Must mix last, because it hooks into previously defined methods
    include Callbacks
  end
end

This class doesn't define the #find_and_execute_task so once again, I have to go hunting for it inside the modules.

module Capistrano
  class Configuration
    module Execution
      # Attempts to locate the task at the given fully-qualified path, and
      # execute it. If no such task exists, a Capistrano::NoSuchTaskError will
      # be raised.
      def find_and_execute_task(path, hooks={})
        task = find_task(path) or raise NoSuchTaskError, "the task `#{path}' does not exist"

        trigger(hooks[:before], task) if hooks[:before]
        result = execute_task(task)
        trigger(hooks[:after], task) if hooks[:after]

        result
      end
    end
  end
end

I found the method inside Capistrano::Configuration::Execution but it looks like the #execute_task method is what runs the command.

module Capistrano
  class Configuration
    module Execution
      # Executes the task with the given name, without invoking any associated
      # callbacks.
      def execute_task(task)
        logger.debug "executing `#{task.fully_qualified_name}'"
        push_task_call_frame(task)
        invoke_task_directly(task)
      ensure
        pop_task_call_frame
      end
    end
  end
end

Ignoring the #push_tasks_call_frame and #pop_task_call_frame for now, the #invoke_task_directly method is called.

module Capistrano
  class Configuration
    module Execution

    protected

      # Invokes the task's body directly, without setting up the call frame.
      def invoke_task_directly(task)
        task.namespace.instance_eval(&task.body)
      end
    end
  end
end

You know you are getting into the good stuff when a program's execution starts using the protected methods. These are the dirty little methods where all the work gets done.

From this, it looks like Capistrano just runs instance_eval on the task's body (a block). So I've traced the call from the commandline (cap deploy) all the way into Capistrano where the code calls a specific task/recipe. If I jump ahead and look at a few of the recipe definitions for deploy, the rest of the call stack starts to make sense:

namespace :deploy do
  task :default do
    update
    restart
  end

  task :update do
    transaction do
      update_code
      symlink
    end
  end

  task :update_code, :except => { :no_release => true } do
    on_rollback { run "rm -rf #{release_path}; true" }
    strategy.deploy!
    finalize_update
  end

  task :restart, :roles => :app, :except => { :no_release => true } do
    warn "[DEPRECATED] `deploy:restart` is going to be changed to Passenger mod_rails' method after 2.5.9 - see http://is.gd/2BPeA"
    try_runner "#{current_path}/script/process/reaper"
  end
end

deploy's block runs instance_eval which makes sense:

• update
  • update_code
    • deploys via the deploy strategy
    • finalizes the update
  • symlink
• restart

Based on what I've read today, I would separate Capistrano into two components:

1. Configuration and option parsing component
2. Recipes

Since I've done a lot of system administration in the past, I'm going to focus on reading through the code for the recipes this week. The configuration component is interesting but as you can see from this example, there is a lot of redirection and delegation going on in there that doesn't interest me at all. Tomorrow's code reading will start to take a look at the different recipes that make up cap deploy.

I use passenger in development now. Having all of my applications ready to launch at any time makes it easier to test cross application integrations.

One problem with passenger is that even in development mode, Rails plugins are cached from request to request. This means if you edit a plugin, you won't see the changes until you restart passenger. Since 90% of my development is on Redmine plugins, I was having to constantly restart passenger for every change I made. Fortunately, I remember rstakeout from a few years back...

rstakeout was originally written by topfunky to have a bit more control over his autotest. Basically what it does is watch a directory or files and when they change, to run a command.

This let me cobble together a script that watches the plugins on a project and restart passenger by touching tmp/restart.txt when any code changes:

# bin/watch_plugins_for_passenger.rb
#!/usr/bin/env ruby
system("touch tmp/restart.txt")
system('ruby ~/dev/rstakeout/rstakeout.rb "touch tmp/restart.txt" "vendor/**/*"')

Here's an example run from yesterday during a marathon coding session.

=> vendor/plugins/redmine_contracts/test/integration changed, running 'touch tmp/restart.txt'

=> done
=> vendor/plugins/redmine_contracts/test/integration changed, running 'touch tmp/restart.txt'

=> done
=> vendor/plugins/redmine_contracts/test/integration/contracts_list_test.rb changed, running 'touch tmp/restart.txt'

=> done
=> vendor/plugins/redmine_contracts/test/integration changed, running 'touch tmp/restart.txt'

=> done
=> vendor/plugins/redmine_contracts/test/integration changed, running 'touch tmp/restart.txt'

=> done
=> vendor/plugins/redmine_contracts/test/integration/contracts_list_test.rb changed, running 'touch tmp/restart.txt'

=> done

To use rstakeout, you'll need to grab the source directly as there isn't a RubyGem for it yet. Reading topfunky's blog post would also be good to get some other ideas about what it can be used for.

The third step in flay's process is to run the s-expressions through the #analyze method.

The Code

  def analyze
    self.prune

    self.hashes.each do |hash,nodes|
      identical[hash] = nodes[1..-1].all? { |n| n == nodes.first }
      masses[hash] = nodes.first.mass * nodes.size
      masses[hash] *= (nodes.size) if identical[hash]
      self.total += masses[hash]
    end
  end

Review

Flay#analyze is performing four steps:

1. Prune the trees to remove data that isn't needed and then for each tree:
2. Track identical trees
3. Track the masses of each tree
4. Track the total mass of all of the trees

1. Prune the trees to remove data that isn't needed

self.prune

flay's #prune method walks all of the trees and removes ones that are too small or don't contain any duplication. Since flay is only concerned with code duplication, throwing away data that isn't needed is a good optimization.

2. Track identical trees

identical[hash] = nodes[1..-1].all? { |n| n == nodes.first }

Here flay is iterating over each node in the tree and checking if the s-expression's body is the same as the first node (the type is the first element, the body is the remainder like a Lisp cons cell). When all of the elements match, the tree/hash is tracked in the identical data structure.

3. Track the masses of each tree

masses[hash] = nodes.first.mass * nodes.size
masses[hash] *= (nodes.size) if identical[hash]

Flay is doing two things here. First it's calculating the mass of the tree by multiplying the first node's mass by the number of nodes. Second, if the tree was identical the total mass is multiplied again by the number of nodes. I think this is done to give a larger score to identical duplication so they appear higher in the report.

4. Track the total mass of all of the trees

self.total += masses[hash]

Finally, the total mass for the tree is added to the total counter. This counter is used in the report for the overall score.

With this method, I've completed my code reading of flay. It was very educational stumble through how flay works since it's using some advanced programming concepts and libraries I've never used before. If reading flay's code was interesting, there are a few methods that I didn't cover that would be useful to read through.

Next week I'm going to start reading the code for Capistrano and see what I can find under it's hood.

Now I'm starting to get into the deep dark corners of flay. The #process_sexp method is the next step in the process.

The Code

  def process_sexp pt
    pt.deep_each do |node|
      next unless node.any? { |sub| Sexp === sub }
      next if node.mass < self.mass_threshold

      self.hashes[node.structural_hash] << node
    end
  end

Review

#process_sexp is running a collection routine to store the parts of each s-expression into the shared hashes data structure. It starts by recursively applying the block to each s-expression using #deep_each. This should make sure that each section of code is looked at for duplication:

• the class as a whole
• the methods in the class
• the code inside a method
• the single line of code
• the atoms of each Ruby statement

Each of those gets yielded to the block in #process_sexp which does two checks before adding it to the hashes.

next unless node.any? { |sub| Sexp === sub }

This is checking if the current node (s-expression) includes only other s-expressions.

next if node.mass < self.mass_threshold

This check is to make sure that only nodes above the mass threshold are included in the report. Flay defaults this to 16 but includes an option to change it.

self.hashes[node.structural_hash] << node

Finally, if both those checks above pass, the node is added to the shared hashes structure. From what I see node.structural_hash is a simplified version of the structure that just holds the s-expression types. I think this is what lets flay track how many times a statement was used, similar statements are simplified down to the same structural hash.

If anyone has anything they want to add, please post a comment below. Parsing and s-expressions are not my strengths so I might be totally reading this wrong.

Today I'm reading through flay's #process method. This is one of the core methods that performs the analysis.

The Code

  def process(*files) # TODO: rename from process - should act as SexpProcessor
    files.each do |file|
      warn "Processing #{file}" if option[:verbose]

      ext = File.extname(file).sub(/^\./, '')
      ext = "rb" if ext.nil? || ext.empty?
      msg = "process_#{ext}"

      unless respond_to? msg then
        warn "  Unknown file type: #{ext}, defaulting to ruby"
        msg = "process_rb"
      end

      begin
        sexp = begin
                 send msg, file
               rescue => e
                 warn "  #{e.message.strip}"
                 warn "  skipping #{file}"
                 nil
               end

        next unless sexp

        process_sexp sexp
      rescue SyntaxError => e
        warn "  skipping #{file}: #{e.message}"
      end
    end

    analyze
  end

  def process_rb file
    RubyParser.new.process(File.read(file), file)
  end

Review

There are three steps to #process, each of which is run on every file:

1. Run the Ruby file through RubyParser to convert it to s-expressions
2. Process the s-expressions using Flay#process_sexp
3. Run Flay#analyze over the data

I'm going to look at the first one today, the conversion to an s-expression.

Processing the file

ext = File.extname(file).sub(/^\./, '')
ext = "rb" if ext.nil? || ext.empty?
msg = "process_#{ext}"

unless respond_to? msg then
  warn "  Unknown file type: #{ext}, defaulting to ruby"
  msg = "process_rb"
end

This chunk of flay tries to figure out the extension of the file so it knows what method to run on it. A Ruby file (snake.rb) would then be sent to the #process_rb method.

If there isn't a method for the known extension, it defaults to #process_rb. For example ladder.jruby would try to be sent to the #process_jruby method but since it doesn't exist, #process_rb would be used.

process_rb - the s-expression conversion

Assuming the file is a Ruby file, then it will be processed by #process_rb. This method is a simple wrapper for RubyParser's #process.

def process_rb file
  RubyParser.new.process(File.read(file), file)
end

I wrote some example code with RubyParser to see how it works. It's returning a tree of s-expressions that represent the Ruby code. For example, this simple class is turned into the following s-expression tree.

class Post
  def author
    @author
  end

  def author=(name)
    @author = name
  end
end

s(:class,
 :Post,
 nil,
 s(:scope,
  s(:block,
   s(:defn, :author, s(:args), s(:scope, s(:block, s(:ivar, :@author)))),
   s(:defn,
    :author=,
    s(:args, :name),
    s(:scope, s(:block, s(:iasgn, :@author, s(:lvar, :name))))))))

Tomorrow I'll dig into the next step flay takes to process the s-expression with #process_sexp.

I'm looking at flay's #expand_dirs_to_files today. This method takes a list of files and directories and will recursively walk them to collect all of the Ruby source files that need to be analyzed.

The Code

  def self.expand_dirs_to_files *dirs
    extensions = ['rb'] + Flay.load_plugins

    dirs.flatten.map { |p|
      if File.directory? p then
        Dir[File.join(p, '**', "*.{#{extensions.join(',')}}")]
      else
        p
      end
    }.flatten
  end

Review

#expand_dirs_to_files has four steps:

1. Collect all of the file extensions
2. Iterative over each item in the dirs list
3. If the item is a directory, find all files inside. Otherwise just list the item
4. Collect and flatten all of the files that were found

1. Collect all of the file extensions

extensions = ['rb'] + Flay.load_plugins

Flay builds an array of extensions here. What's interesting to me is that this gets sent to File#join separated by commas:

# Source
File.join(p, '**', "*.{#{extensions.join(',')}}")

# Expanded with extensions = ['rb', 'liquid', 'xml']
File.join(p, '**', "*.{rb,liquid,xml}}")

It doesn't look like the {} syntax is documented in File#join at all.

Update: Gregor Schmidt explained in the comments below that {} is documented in Dir#glob. So File#join assembles a string like ./**/*.{rb,liquid,xml} which gets passed into Dir#[] below.

Something else I didn't know was that flay supports plugins. An ERB plugin is included with Flay but it looks like it might be easy to write plugins for other Ruby languages like HAML or Liquid.

2. Iterative over each item in the dirs list

dirs.flatten.map { |p| ... }

This is just a simple iterator and since it's the last statement in #expand_dirs_to_files it will return the array to the caller.

3. If the item is a directory, find all files inside. Otherwise just list the item

if File.directory? p then
  Dir[File.join(p, '**', "*.{#{extensions.join(',')}}")]
else
  p
end

Here is where the files and directories are recursively walked to find all files with matching extensions. I like how this code itself isn't recursive, but it makes use of Dir#[]'s recursive globs to find all files in sub-directories (using '**/').

4. Collect and flatten all of the files that were found

To finish up the search, #expand_dirs_to_files flattens the results into a single array and then returns it to the caller.

It's interesting how #expand_dirs_to_files made use of File#join and Dir#[] to do all of the file searching, but still has a nice and simple public API. This pattern would be useful anytime I need to recursively search for files in Ruby.

This week I'm looking at some of the metrics libraries included in metric_fu. Since these libraries analyze Ruby code, I suspect that I'll find some very interesting Ruby inside them.

I'm starting with flay, a library that analyzes code for duplication and structural similarities. Since flay includes a command line executable, I'm starting with that and will trace the application flow from there.

The Code

#!/usr/bin/ruby

require 'flay'

flay = Flay.new Flay.parse_options

ARGV << '.' if ARGV.empty?
files = Flay.expand_dirs_to_files(*ARGV)

flay.process(*files)
flay.report

Review

Flay's executable does five things to create a Flay report:

1. Initializes a new Flay object by parsing command line options using optparse.
2. Defaults the arguments to the current directory.
3. Expands all directories to get a complete list of files.
4. Processes the files.
5. Outputs a report of the Flay results.

1. Initializes a new Flay object by parsing command line options using optparse.

optparse is a core Ruby class that helps to write Ruby scripts with command line options and flags. Flay is using this to set different options in Flay#parse_options.

2. Defaults the arguments to the current directory.

ARGV << '.' if ARGV.empty?

Flay is meant to be run as flay path/to/ruby.rb or flay lib/ and load all the Ruby files it can find. The code above also lets Flay have a default option of checking all files in the current directory, flay .. It's pretty simple and I think I can use it in my own programs.

3. Expands all directories to get a complete list of files.

Flay#expand_dirs_to_files(*ARGV) has Flay recurse the file system to collect all Ruby files in nested sub-directories. This is just like having grep -R turned on.

4. Processes the files.

Flay#process is the core of Flay, where it checks and analyzes each file. I'll be looking into it in more depth later this week, but this is where all the magic happens.

5. Outputs a report of the Flay results.

This just outputs a report on the command line with puts calls. Nothing fancy.

Now I see #expand_dirs_to_files and #process are the interesting parts of Flay that I'll need to dig into next.

I'm looking at the Rack::Staging middleware today. This middleware looks like a great idea for staging servers, since it requires all requests to have a valid staging code (cookie). If a user doesn't have one yet, you can create an HTML page that lets them submit their staging code.

The Code

# Hi! I'am rack middleware!
# I was born for protect you staging application from anonymous and prying

# My author's name was Aleksandr Koss. Mail him at kossnocorp@gmail.com
# Nice to MIT you!

module Rack
  class Staging
    def initialize app, options = {}
      @app = app
      @options = {
        :session_key => :staging_auth,
        :code_param => :code
      }.merge options
    end

    def call env
      request = Rack::Request.new env

      # Check code in session and return Rails call if is valid

      return @app.call env if \
        request.session[@options[:session_key]] != nil and
        request.session[@options[:session_key]] != '' and
        code_valid? request.session[@options[:session_key]]

      # If post method check :code_param value

      if request.post? and code_valid? request.params[@options[:code_param].to_s]
        request.session[@options[:session_key]] =
          request.params[@options[:code_param].to_s]

        [301, {'Location' => '/'}, ''] # Redirect if code is valid
      else
        render_staging
      end
    end

  private

    # Render staging html file

    def render_staging
      [200, {'Content-Type' => 'text/html'}, [
        ::File.open(@options[:template_path], 'rb').read
      ]]
    end

    # Validate code

    def code_valid? code
      @options[:auth_codes].include? code
    end
  end
end

Code

Review

There are three different cases I want to review in Rack::Staging.

1. No staging code

def render_staging
  [200, {'Content-Type' => 'text/html'}, [
    ::File.open(@options[:template_path], 'rb').read
  ]]
end

This is the first case a user will run into. It happens when they don't have a staging code in their session yet. The request ends up falling through most of #call until render_staging is reached. render_staging then reads the HTML file and sends it back to the browser.

2. Posting a new staging code

if request.post? and code_valid? request.params[@options[:code_param].to_s]
  request.session[@options[:session_key]] = request.params[@options[:code_param].to_s]
  [301, {'Location' => '/'}, ''] # Redirect if code is valid
else
  # ...
end

When a new staging code is posted from the form, it runs the code above. If the staging code posted is valid, Rack::Staging will then set the staging code in the session and redirect back to the root.

3. Responding to a request that has a staging code

return @app.call env if \
  request.session[@options[:session_key]] != nil and
  request.session[@options[:session_key]] != '' and
  code_valid? request.session[@options[:session_key]]

The final case happens when a request already has a valid staging code. This would send the request to the next application in the Rack stack, letting the final app handle the response like normal.

I think Rack::Staging would be very useful. I'm using some HTTP basic authentication to block access to the different staging sites and it's clunky at best. I'd like to see Rack::Staging have some tests added, I think there is an edge case that might not be handled correctly.

Since I've reviewed five different Rack middlewares this week, I thought it would be useful to list the other ones I'm interested in. Some are great for production deployments, some are great for development, and some are just plain silly.

1. DbIndexes - shows the database indexes used by ActiveRecord on the page.
2. OnlyForMe - simple authentication.
3. Rack::Auth::Cheat - lets you login to an application as any user with a master password.
4. Rack::EnforceSSL - forces specific paths to be requested over SSL.
5. Rack::Environmental - shows a colored badge on the page for each environment.
6. Rack::GemAssets - sends static files from gems to the browser. Might be useful for Rails Engines.
7. Rack::KarmaChameleon - switches the extension of each page. e.g. /login.php
8. Rack::LocaleSetter - "Set Rails locale using subdomain."
9. Rack::Pot - It's a little teapot, short and $stdout.
10. Rack::Rakismet - checks each request to see if there is spam comments against the Akismet API.
11. Rack::Rewrite - allows defining request rewrite rules in Rack/Ruby instead of in the web server (Apache).
12. Rack::Runtime - adds an X-Runtime HTTP header that shows how fast the application generated the content.
13. Rack::SpellCheck - highlights speeling errors on the web page.
14. Rack::Steroscope - adds an HTML interface to your RESTful API.
15. Rack::Tidy - automatically indents and formats valid HTML markup.
16. Rack::Validate - runs the page content against the w3c validator.
17. ResourceCount - adds a REST endpoint to get a count of the number of records in ActiveRecord.

There are the five from my code reading series where I've taken a more in depth look at:

1. Rack::MemoryBloat
2. Rack::GoogleAnalytics
3. Rack::StaticFallback
4. Rack::ForceDomain
5. Rack::Staging

What other Rack middleware is interesting to you?