[maildropl] error writing to filter (bogofilter)

[David Relson]

On Sun, 16 Jun 2013 16:05:58 +0100
Christian Ebert wrote:

> * David Relson on Sunday, June 16, 2013 at 09:31:27 -0400
> > David Relson wrote:
> >> Of note is that Christian's stdout.txt file is of size 8192.  This
> >> indicates that bogofilter didn't process the complete message
> >> (which is approx 350k).  The question becomes is this a bogofilter
> >> issue or an issue in passing the message to bogofilter?
> >> 
> >> I've created a maildroprc file that runs a script that saves
> >> stdin, stdout, and stderr.  Here they are:
> >> 
> >> #### BEGIN /etc/maildroprc ####
> >> # Global maildrop filter file
> >> 
> >> #DEFAULT="$HOME/.maildir/"
> >> 
> >> xfilter "/tmp/bogofilter.sh -u -e -p -vvv" 
> >> #### END /etc/maildroprc ####
> >> 
> >> #### BEGIN bogofilter.sh ####
> >> tee /tmp/stdin.out bogofilter $* -x amlt 2> /tmp/stderr.out | tee \
> >> /tmp/stdout.out 
> >> #### END bogofilter.sh ####
> >> 
> >> Then I compared maildropbogo/stderr.txt (Christian's) with
> >> stderr.out (mine).  As mail clients tend to wrap lines, I've
> >> attached an edited part of the diff.  Here are my observations:
> >> 
> >> The textblock.c lines have different addresses due to MacOS and
> >> Linux differences -- not a problem.  Some of these lines have been
> >> snipped (because there are lots of them).
> >> 
> >> Then at after "@@ -1079,32 +1078,39 @@" the diff becomes
> >> interesting. Function yyinput (part of the lexer) reports 327915
> >> bytes received by Linux but only 8192 bytes received by MacOS.  My
> >> diagnosis is that MacOS has a problem.
> >> 
> >> Christian, as a test for you, modify maildroprc to use
> >> bogofilter.sh and see if stdin.out is big (350k) or small (8k).
> > 
> > I forgot to include the sizes of the std* files:
> > 
> > MacOS:
> >  270557 Jun 16 06:21 stderr.txt
> >    8192 Jun 16 06:21 stdout.txt
> > 
> > Linux:
> > 1532529 Jun 16 09:03 stderr.out
> >  333309 Jun 16 09:03 stdin.out
> >  351879 Jun 16 09:03 stdout.out
> > 
> > As can be seen, not only does MacOS produce an 8k stdout file, but
> > stderr varies significantly is size (270K vs 1532K).
> 
> $ du /tmp/stdin.out
> 80      /tmp/stdin.out
> $ ls -l /tmp/std*
> -rw-rw----  1 chris  wheel      0 Jun 16 16:03 /tmp/stderr.out
> -rw-rw----  1 chris  wheel  40960 Jun 16 16:03 /tmp/stdin.out
> -rw-rw----  1 chris  wheel   8192 Jun 16 16:03 /tmp/stdout.out

Hi Christian,

I was expecting that stdin.out would be 8k, not the 40k shown.   

Since 40k is far short of the 333,309 bytes in the problem message, I
conclude this is a MacOS specific problem -- possibly a maildrop issue,
but more likely a pipe issue.

Of course as I wrote the above, another possibility occurred. Within
the bogofilter debug output, the problem manifests itself in yyinput.
This is part of the grammar generated by flex.  Past experience shows
that flex isn't perfect. Bogofilter's build includes patches for some
old versions of flex, though none are needed for v2.5.35 and later
versions.  Possibly flex has an issue...

Upon further reflection, a flex issue wouldn't cause the stdin.out size
problem, and I return to the problem being with maildrop or pipes.

HTH,

Regards,

David