Ignoring headers [was: SpamAssassin's header lines]

[Mark M. Hoffman]

[David]
> > Perhaps the two of you should put your heads together and see what kind of 
> > solution you can design...

[Eric]
> Well, I don't think that the "ignore-list" idea can (or should) be 
> expanded to be aware of message structure (what's a header, what's not) 
> because the ignore-list support is literally only a few lines of code when 
> we look up the spamicity of a token.
> 
> The idea that bogofilter should be aware of message headers, possibly 
> having the ability to add/remove/modify them, has merit.  I think that for 
> now, anyway, our treatment of headers should be a separate problem, 
> because it has to look at the message early on, before we shred it into a 
> "bag of tokens".
> 
> I don't think that the treatment (keep or discard; examine or 
> don't-examine) of headers like spamassasin adds should be hard-wired into 
> the lexer; I think there will be users who will be dead-set against this, 
> and others that are dead-set for it.

I agree with Eric on all counts above.  I only meant that header-ignores be
spec'ed in the option file... not that it should be integrated with Eric's
ignore-tokens stuff.

> I have a gut feeling that one day, the flex lexer.l won't be good enough.  
> Possible reasons: MIME-encoded messages, specialized treatment of header 
> fields, HTML handling, international charset support...

I (grudgingly) have this feeling also.

Regards,

-- 
Mark M. Hoffman
mhoffman at lightlink.com