Perl 6: Grammar issues (simple defines)

My goal is to parse defines in Objective-C code:

#define wowFont [UIFont fontWithName:(DEV)?@"one":@"two" size:(DEV? 10: 12)]
#define wowColor [UIColor colorWithRed:(DEV?240/255.0f:255.0f) green:((DEV)? 230.0f / 255.0f) blue: 10.0f]

I try to write grammar for it:

grammar Defines::Grammar {
    token TOP {
        \s*\n
    }
    token define_expression {

        ^^ #define $=\w+ \s+ $ = 
    }

    token general_expression{
        # here, determine parenthes
        |[(][)]|
    }

    token simple_expression{
        # here, determine ?: behaviour
        \s*\s*[?]\s*\s*[:]\s*\s*
    }

    token value_expression {
        # stop here, need to determine ?: operator here
        |||
    }

    token function_invocation_expression {
        [\[] (|) \s+ \s* [\]]
    }

    token function_body {
        | \s* [:] 
    }

    token bareword_expression {
        [\w_]+
    }

    token string_expression {
        \@["][^"]["]
    }

    token numeric_expression {
        # we could +-*/ numeric expressions, also we can put lead sigil [-]. put parentheses and it could be a 
         [*/+-]  | [-+] | [(]  [)] | 
    }

    token number {  
        # number expression here, ok
        # not needed legal sign here, numeric_expression already have it
        \d+ [\.] \d+
    }
}

Could anybody help to improve it?

P.S: As a part of answer (I hope that somebody will find this documents helpful) I place here a link to Perl 6 book (I found out that Grammars section is good enough for beginning). Perl 6 Book

I was about to submit this when I saw that you had visited #perl6. I'll leave this post as is because it is supposed to help others that might one day read this as well as you.

TL;DR I recommend you: make sure you're ready for an immature compiler, ecosystem, doc, etc; use Rakudo Star and the Rakudo debugger; start small and take small steps, making each step work before moving forward; rely very heavily on #perl6 -- they welcome any and all comers, including "newbies" and lots of basic questions.

First, let me make sure you are not wasting your time due to wrong expectations. I am pretty sure there are currently only a handful of folk using P6 in production and they are doing so for only a small range of use cases. Powerful parsing is one of them, so what you are attempting is reasonable, but it's slow and you need to be willing to patiently deal with the many other frustrations that can result from P6's immaturity. If this hasn't put you off, welcome aboard! :)

Second, let me make sure I am not wasting your time. Your code doesn't compile using the tools I'm using. I used a procedure you can repeat to test your code with the latest Rakudo Perl 6 compiler HEAD:

• I joined #perl6, the freenode IRC channel. (A very friendly channel. All are welcome.)

• I tested code by entering a line of the form "m: ...". (Replace the "..." with code, or a suitable "pastebin" URL as explained below, otherwise you'll get a "Stub code executed" exception. An evalbot spots such lines, runs the code using the HEAD of the Rakudo compiler using the MoarVM backend, and displays the result.)

• I ran your code privately (recommended if you don't want to "spam" the channel with attempts that aren't interesting to others) so you won't be able to see this in the public IRC log. To run code privately enter the command /msg camelia r: say $*PERL, switch to the "camelia" window, and enter lines there.

• So I entered "m: https://gist.github.com/raiph/11084741" which attempted to compile a copy of your code. It failed to compile.

Third, let's radically improve your debugging options. One option when debugging grammars is to use the Grammar::Tracer module. This is occasionally useful but for most purposes there's a much better option: Rakudo's good-trending-great debugger. Use the debugger all the time unless you have a compelling reason not to. The Rakudo debugger is not available using the on channel evalbots I just discussed above; it requires that you install something. I recommend the Rakudo Star package unless you have a compelling reason to use something else. Rakudo Star includes the debugger, modules, and doc including info on how to start the debugger.

Fourth, start simple and build up from there, only making another small step forward when what you currently have works. For example, a first step might be:

say so '#define foo [blah]' ~~ rule { '#define'  .* }

prints:

True

meaning the string on the left of the ~~ was "accepted" by the thing on the right.

The thing on the right is a rule which is one of several convenient forms of regex introduced by P6. The P6 regex engine never backtracks once it's matched an atom in a rule and it replaces appropriate spaces in your rule with implicit calls to the whitespace matching rule called `. The above line is equivalent to:

say so '#define foo [blah]' ~~ regex { :ratchet '#define'    .* }

(The . in tells P6 to call the ` rule without saving the text it matched in the match object being built up by the processing of the regex.)

StackExchange Code Review Q#47639, answer score: 4