effects of lexer changes

[Matthias Andree]

Gyepi SAM <gyepi at praxis-sw.com> writes:

> On Fri, Jan 03, 2003 at 12:27:29AM +0100, Matthias Andree wrote:
>> David Relson <relson at osagesoftware.com> writes:
>
>> > Adding "stackp > 0" to the ignore expression fixes the immediate problem.
>> 
>> Is the nesting depth the correct parameter? I'd think that we ought to
>> look at the mime type instead. If it's message/rfc822, text/*, we want
>> to have a look, as these can happen at top level.
>
> I hope you meant message/rfc822 and multipart/*.
> text/* does not nest. Minor issue, but at this stage it is important to
> agree on these things.

Well, we can have text/* at any level, including top level. We certainly
also want to have a look at tokens in the message/rfc822 parts. We might
not want to look into the tokens of multipart/*, but only of its
subparts. But then again, we might want to tokenize these as well as
their subparts, in case there is an indicative MIME prologue (the part
between the end of the multipart/* headers and the first boundary line,
that is usually something like "This is a MIME-encapsulated message.").

-- 
Matthias Andree