ruby - How to tokenize a simple mixed string into either ints or symbols?

Question

Trying to convert a simple math string "1 2 3 * + 4 5 - /" to an array of integers and/or symbols like [1, 2, 3, :*, :+, 4, 5, :-, :/].

Is there a more elegant (and extendable) solution than this?

def tokenize s
    arr = s.split(/ /)
    symbols = %w{ + - / * }
    arr.collect! do |c| 
        if symbols.include?(c)
            c.to_sym
        else
            c.to_i
        end
    end
end

Answer 1

def tokenize(str)
  str.split.map! { |t| t[/\d/] ? t.to_i : t.to_sym }
end

Instead of checking if the token is in a set of operations, you can just check if it contains a numeral digit (for your use case). So it's either an integer, or an operation.

Also note that Array#collect! and Array#map! are identical, and String#split by default splits on white space.

Answer 2

How about...

h = Hash[*%w{+ - / *}.map { |x| [x, x.to_sym] }.flatten]

And then,

arr.map { |c| h[c] || c.to_i }

So together:

def tokenize s
    h = Hash[*%w{+ - / *}.map { |x| [x, x.to_sym] }.flatten]
    s.split.map { |c| h[c] || c.to_i }
end

Answer 3

def tokenize s
  s.scan(/(\d+)|(\S+)/)
  .map{|num, sym| num && num.to_i || sym.to_sym}
end