April 30th, 2006 -- Small improvement

There is a course project deadline on midnight today (several minutes before). It took I and my partner about 50+ hours each. However, we've done it in time.

Since the rule ProductionRule consist of Statements and Return goes into infinite loop (I'm not sure, but I don't want to wait any more), I currently restrict it to have only a return statement. And I played with capture aliasing and quantified subpattern captures in Pugs::Compiler::Rule. It basically works but seems to have some problem on deeper matching?

I have a (self recursive) rule

rule AdditiveExpr {
    <FunctionAppExpr> <ws>?
    [ $<op> := (\+ | \- | ~) <ws>?
      $<rhs> := (<AdditiveExpr>) <ws>? ]?
    { use v5;
      if($<op>[0]){
          return "(" . $<FunctionAppExpr>() . " " . $<op>[0] . " " . $<rhs>[0] . ")";
      }else{
          return $<FunctionAppExpr>();
      }
    }
}

This works for matching "$<a> + t" there both terms can be matched by FunctionAppExpr, but when I feed "$<a> + t + 3", the "+ 3" part disappeared.

But some good news. r10165 can translate "return $<a> + t" to "return $ (a + t)", which valid Haskell and is what I want.

April 28th, 2006 -- The Grammar

Came back from the future today, it's still April now, not May. Thanks TimToday.

I woke up really late today. It was about noon and I went to the office for the $part_time_job in the afternoon. But I can still spend some time on IRC. At about five o'clock, I fed the rule I drafted for MiniPerl6 to Pugs::Compiler::Rule and found some compilation error. Audrey pointed out that the comma may have to be escaped. That worked and many other problems are resolved in the same way including the plus sign, minus sign, tilde (string concatenation operator) and semicolon.

The draft of the grammar is now here. It's full of "<ws>?" since I do not yet know how to turn the word parsing mode on in P::C::Rule. And the string matching part lacks escaping sequence matching because Pugs::Emit::Rule::Perl5 currently ignores character class rule and I met some problem handling backslashes. I will spend some time to fix it. Also, production rules are mostly omitted but, as the sample parsing script demos, the written one works.

You might find that there is currently no Pugs/Grammar/MiniPerl6.pm in the repository. Yes, it's not there since it can be generated by feeding the grammar to the script under Pugs-Compiler-Rule. The automatic generating script will be added soon, maybe right after some kind of completion of the mini-grammar.

April 27th, 2006 -- The Plan

I tried to feed the yadaLiteral rule describe yesterday (this morning) to Pugs::Compiler::Rule and found it failed to parse. The problem is that there is a $<sym> ~ ' - not yet implemented' in it but it's not valid perl 5 nor does the ad-hoc translator changed it correctly. But why is the production rule executed (eval-ed) before the rule is used to parse anything? Since the constructor did that.

I quickly wrote it down and planned to ask fglock, the main author of P::C::Rule, after dinner if s/he's online again. But when I attached the screen, the window eleven of the irssi is marked red and I found that audrey found this blog :) Welcome Audrey.

I had some thing to ask her, too. It's about the (just mentioned) perl 6 to perl 5 translator. After the conversation, the bootstrap plan forms (full text is here):

1. This module, Pugs::Grammar::MiniPerl6, uses Pugs::Compiler::Rule to read a special *mixed* perl 6 rule whose production rules are written in perl 5 (the current requirement of P::C::Rule).
   
   The rule is used to translate a subset of perl 6, MiniPerl6,
   to Haskell.



2. Then, Pugs::Compiler::Emit::Parsec lands and uses this module to translate the full perl 6 grammar into a parser. The full grammar can now write production rules in MiniPerl6 since P::C::E::Parsec can use this module to translate such production rules to Haskell and makes the final output be pure Haskell.



3. When compiling Pugs, the .hs preprocessor will use
   Pugs::Compiler::Emit::Parsec to accept the full perl 6 grammar
   generating Parser.hs. Then GHC will compile it to executable.



4. The executable can now read the full perl 6 grammar again generating compiler in PIL. Then self hosting is done.

That's all about today. There's a deadline this weekend so that there may not be any progress before Monday, sorry :(

April 26th, 2006 -- The Second Beginning

I mainly focused on parser hacking in the development one year ago, so that this time when I asked Audrey, she remember that I've added the parsing function for :key(val) (an alternative pair constructing syntax) and "?? !!" ("?? ::" when I proposed it, in fact I didn't really added it. I wrote an analysis and she added it). Fortunately, she was refactoring Parser.hs preparing to replace it by a perl 6 rule file generated one.

So that's my task. I'm now writing a translator in perl 5 which accepts a perl 6 rule file (with some restrictions on the production rules). Here's an example. (source code of pugs can be obtained from http://svn.openfoundry.org/pugs/)

{- From src/Pugs/Parser/Literal.hs line 24 -}
yadaLiteral :: RuleParser Exp
yadaLiteral = expRule $ do
    sym  <- choice . map symbol $ words " ... ??? !!! "
    return $ App (Var $ doYada sym) Nothing [Val $ VStr (sym ++ " - not yet implemented")]
    where
      doYada "..." = "&fail_" -- XXX rename to fail() eventually
      doYada "???" = "&warn"
      doYada "!!!" = "&die"
      doYada _ = error "Bad yada symbol"

Can be written as (perl 6 rule format)

rule yadaLiteral {
   $<sym> := [ <'...'> | <'???'> | <'!!!'> ]
   { return App(
       Var( doYada( $<sym> ) ),
       Nothing,
       [ Val( VStr( $<sym> ~ ' - not yet implemented') ) ]
     )
   }
}

Now, there is already a perl 6 rule parser module written in perl 5: Pugs::Compiler::Rule. Currently it emits recursive-descent perl 5 parsing code. I'm going to add a Haskell backend for it.

< autreyt> so, your task, should you choose to accept it, is to use whatever the most expedient way to produce a parser for such a language, and emit Haskell code for it
< scw> in perl 5, but using three languages at once :D
< audreyt> :D
< scw> so what I could do is adding a backend for Pugs::Compiler::Rule
< scw> which targets on Haskell

How is the progress? Have you seen that it's April 27th now in +0800 time zone? What does that mean? Oh, nothing, just a joke. The idea of creating a new blog for this didn't come up until this morning. I finally understood how to use Pugs::Compiler::Rule last night, which was the goal I set for my self yesterday, then went back to the VFX ( Digital Visual Effects) project again. So that's all. I can now write a small peace of perl 5 code

use Pugs::Compiler::Rule;
my $rule = Pugs::Compiler::Rule->compile("moose { return 23 }");
print $rule->{perl5}

Which converts

rule foo{ moose { return 23 } }

into

do {
   package Pugs::Runtime::Rule;
   my $matcher =
      concat(
          constant( q!m! )
,         concat(
            constant( q!o! )
,           concat(
              constant( q!o! )
,             concat(
                constant( q!s! )
,               concat(
                  constant( q!e! )
,                   abort(
                      sub {
                          return { bool => 1, tail => $_[0], return => sub { return 23 } };
                      }
                  )
              )
            )
          )
        )
      )
 ;
 sub {
   my $grammar = shift;
   my $tree;
   rule_wrapper( $_[0],
       $matcher->( $_[0], $_[1], $tree, $tree, $grammar, 0, $_[0], $_[2] )
   );
 }
}

Scary, isn't it? However, my goal is

ruleFoo = do{ symbol "moose"; return $ VInt 23 }

Much clearer now! Hope I can, and see you next time.

Restart Hacking!

Hello, I'm Shu-Chun Weng, a.k.a. scw. The commit bit was gained on March 1st, 2005 and I did some parser and builtin function hacks. But all these stopped on Apral 18th, 2005 due to the National University Sports Game hold on April 29th. There were two extra commits after then but those are not really "hacking."

However, after a whole year of stopping, I asked Autrey Tang if there's some thing I can help again, just as the first time I joint #perl6 and asked her (him, then). So that I'm now hacking *again*. *grin*