[PATCH] experimenting with new parsing rules

[David Relson]

On Tue, 16 Sep 2003 02:07:44 +0200
Matthias Andree <matthias.andree at gmx.de> wrote:

> David Relson <relson at osagesoftware.com> writes:
> 
> >> $ echo "Testheader: test" | BOGOTEST=2 ./bogofilter -Ph -vvv
> >> X-Bogosity: No, tests=bogofilter, spamicity=0.038938,
> >version=0.15.3,> date=2003-09-15T23:41:09Z
> >>                                      n    pgood     pbad      fw   
> > U
> >> "test"                              16  0.097403  0.003922 
> >0.038938 +> "Testheader"                         0  0.000000 
> >0.000000  0.415000 -> N_P_Q_S_s_x_md                       1 
> >9.61e-01  3.89e-02  3.89e-02>                                        
> >1.00e-02  4.15e-01  0.100
> >
> > With the inclusion of the BOGOTEST code to CVS this morning, I
> > changed BOGOTEST=1 to mean "enable lexer debug mode" (since the
> > debug code is now present, by default) and BOGOTEST=2 to enable
> > Michael's changes(which are present for testing purposes).
> 
> Yup. As you can see above, I tried BOGOTEST=2. Still, I don't see
> "head:" tags. Is there anything besides BOGOTEST=2 and -Ph that I need
> to enable this code or is there something wrong? 

Try the following code (near the end of lexer_v3.l):

void yy_set_state_initial(void)
{
    int normal = (bogotest & 2) == 0;

    if (normal) {		/* 1 - INITEST, 2 - debug (display lexer states)
*/
 	BEGIN INITIAL;
    }
    else {
 	BEGIN INITEST;
	if (header_line_markup)
	    set_tag("Header");
    }

    msg_header = true;

    if (DEBUG_LEXER(1))
	fprintf(dbgout, "BEGIN %s\n", normal ? "INITIAL" : "INITEST");
}