Daily Code Reading #14 – Flay#process

Today I’m reading through flay’s #process method. This is one of the core methods that performs the analysis.

The Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

  def process(*files) # TODO: rename from process - should act as SexpProcessor
    files.each do |file|
      warn "Processing #{file}" if option[:verbose]
 
      ext = File.extname(file).sub(/^\./, '')
      ext = "rb" if ext.nil? || ext.empty?
      msg = "process_#{ext}"
 
      unless respond_to? msg then
        warn "  Unknown file type: #{ext}, defaulting to ruby"
        msg = "process_rb"
      end
 
      begin
        sexp = begin
                 send msg, file
               rescue => e
                 warn "  #{e.message.strip}"
                 warn "  skipping #{file}"
                 nil
               end
 
        next unless sexp
 
        process_sexp sexp
      rescue SyntaxError => e
        warn "  skipping #{file}: #{e.message}"
      end
    end
 
    analyze
  end
 
  def process_rb file
    RubyParser.new.process(File.read(file), file)
  end

Review

There are three steps to #process, each of which is run on every file:

  1. Run the Ruby file through RubyParser to convert it to s-expressions
  2. Process the s-expressions using Flay#process_sexp
  3. Run Flay#analyze over the data

I’m going to look at the first one today, the conversion to an s-expression.

Processing the file

1
2
3
4
5
6
7
8

ext = File.extname(file).sub(/^\./, '')
ext = "rb" if ext.nil? || ext.empty?
msg = "process_#{ext}"
 
unless respond_to? msg then
  warn "  Unknown file type: #{ext}, defaulting to ruby"
  msg = "process_rb"
end

This chunk of flay tries to figure out the extension of the file so it knows what method to run on it. A Ruby file (snake.rb) would then be sent to the #process_rb method.

If there isn’t a method for the known extension, it defaults to #process_rb. For example ladder.jruby would try to be sent to the #process_jruby method but since it doesn’t exist, #process_rb would be used.

process_rb – the s-expression conversion

Assuming the file is a Ruby file, then it will be processed by #process_rb. This method is a simple wrapper for RubyParser’s #process.

1
2
3

def process_rb file
  RubyParser.new.process(File.read(file), file)
end

I wrote some example code with RubyParser to see how it works. It’s returning a tree of s-expressions that represent the Ruby code. For example, this simple class is turned into the following s-expression tree.

1
2
3
4
5
6
7
8
9

class Post
  def author
    @author
  end
 
  def author=(name)
    @author = name
  end
end
 
1
2
3
4
5
6
7
8
9
10

s(:class,
 :Post,
 nil,
 s(:scope,
  s(:block,
   s(:defn, :author, s(:args), s(:scope, s(:block, s(:ivar, :@author)))),
   s(:defn,
    :author=,
    s(:args, :name),
    s(:scope, s(:block, s(:iasgn, :@author, s(:lvar, :name))))))))

Tomorrow I’ll dig into the next step flay takes to process the s-expression with #process_sexp.