[maildropl] error writing to filter (bogofilter)

[David Relson]

On Sun, 16 Jun 2013 11:44:06 +0100
Christian Ebert wrote:

> * Matthias Andree on Sunday, June 16, 2013 at 11:53:08 +0200
> > It may also help to show a trace of
> > 
> > cat offending-message.msg | maildrop -V2
> > (note that this does *not* reproduce how Postfix calls maildrop,
> > because your postfix uses maildrop -d chris, but -d is incompatible
> > with -V according to my maildrop manpage).
> 
> $ cat testbogomsg | maildrop -V2
> maildrop: Changing to /Users/chris
> Message start at 52 bytes, envelope sender=f.pelzer at wp-pelzer.de
> maildrop: Attempting .mailfilter
> maildrop: Filtering through xfilter /usr/local/bin/bogofilter -u -e -p
> maildrop: error writing to filter.
> maildrop: Unable to filter message.
>  
> > As well as the truss.txt you get from:
> > 
> > cat offending-message.msg | truss -aef -o truss.txt maildrop -d
> > chris
> 
> Attached is dtruss.txt
> https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/dtruss.1m.html
> from
> $ cat testbogomsg | sudo dtruss -aef maildrop -V2 -d chris 2>
> dtruss.txt
> 
> The -o option does something different, so I redirected stderr.
> 
> Christian
> 
> -- 
> Die Wolke Wolfgang
> Eine Kindergeschichte mit Bildern. Von Michael Weber.
> Das Buch   -->> http://www.blacktrash.org/baustellen#wolkewolfgang
> Online     -->> http://www.blacktrash.org/wolkewolfgang

Of note is that Christian's stdout.txt file is of size 8192.  This
indicates that bogofilter didn't process the complete message
(which is approx 350k).  The question becomes is this a bogofilter
issue or an issue in passing the message to bogofilter?

I've created a maildroprc file that runs a script that saves
stdin, stdout, and stderr.  Here they are:

#### BEGIN /etc/maildroprc ####
# Global maildrop filter file

#DEFAULT="$HOME/.maildir/"

xfilter "/tmp/bogofilter.sh -u -e -p -vvv" 
#### END /etc/maildroprc ####

#### BEGIN bogofilter.sh ####
tee /tmp/stdin.out bogofilter $* -x amlt 2> /tmp/stderr.out | tee \
/tmp/stdout.out 
#### END bogofilter.sh ####

Then I compared maildropbogo/stderr.txt (Christian's) with stderr.out
(mine).  As mail clients tend to wrap lines, I've attached an edited
part of the diff.  Here are my observations:

The textblock.c lines have different addresses due to MacOS and Linux
differences -- not a problem.  Some of these lines have been snipped
(because there are lots of them).

Then at after "@@ -1079,32 +1078,39 @@" the diff becomes interesting.
Function yyinput (part of the lexer) reports 327915 bytes received by
Linux but only 8192 bytes received by MacOS.  My diagnosis is that
MacOS has a problem.

Christian, as a test for you, modify maildroprc to use bogofilter.sh
and see if stdin.out is big (350k) or small (8k).

I'm looking forward to hearing what happens!

Regards,

David
-------------- next part --------------
--- maildropbogo/stderr.txt	2013-06-16 06:21:15.000000000 -0400
+++ stderr.out	2013-06-16 08:48:33.000000000 -0400
@@ -1,5 +1,4 @@
-(tc) tcbdbopen( /Users/chris/.bogofilter/wordlist.tc, 2 )
-textblock.c:38  0x7fdcb1404320 0x7fdcb1407d50  40 *ini* cur: 40, max: 40, tot: 40
+textblock.c:38  0x198e030 0x198e1a0  40 *ini* cur: 40, max: 40, tot: 40
 *** mime_reset
 *** mime_push. stackp: 0
 **** MIME stack is:
@@ -7,7 +6,7 @@
 --- set_tag(Header) -> prefix=head:
 BEGIN INITIAL
 ***  0 hi 37 Return-Path: <f.pelzer at wp-pelzer.de>
-textblock.c:64  0x7fdcb1407d50 0x7fdcb14084b0  37 *add* cur: 101, max: 101, tot: 101
+textblock.c:64  0x198e1a0 0x1992f40  37 *add* cur: 101, max: 101, tot: 101
 *** yyinput("Return-Path: <f.pelzer at wp-pelzer.de>
 ", 0, 8192) = 37
 --- set_tag(Return-Path:) -> prefix=rtrn:

... < snip > ...

@@ -1079,32 +1078,39 @@
 head:ClamAV
 ClamSMTP
 head:using
-*** 92 hi  1 
-textblock.c:64  0x7fdcb140bd30 0x7fdcb140a060   1 *add* cur: 7642, max: 7642, tot: 7642
+*** 92 hi 23 Content-Length: 327915
+textblock.c:64  0x19b3f50 0x19b3f70  23 *add* cur: 7664, max: 7664, tot: 7664
+*** yyinput("Content-Length: 327915
+", 39, 8192) = 23
+--- set_tag(Header) -> prefix=head:
+Content-Length
+head:ClamSMTP
+*** 93 hi  1 
+textblock.c:64  0x19b3f90 0x19b3fe0   1 *add* cur: 7689, max: 7689, tot: 7689
 *** yyinput("
-", 39, 8192) = 1
+", 23, 8192) = 1
 *** end of header
 
 
-*** 93 bt 44 This is a multipart message in MIME format.
-textblock.c:64  0x7fdcb140bd50 0x7fdcb140bd70  44 *add* cur: 7710, max: 7710, tot: 7710
+*** 94 bt 44 This is a multipart message in MIME format.
+textblock.c:64  0x19b4000 0x19b4020  44 *add* cur: 7757, max: 7757, tot: 7757
 *** yyinput("This is a multipart message in MIME format.
 ", 1, 8192) = 44
 This
-head:ClamSMTP
+head:Content-Length
 is
 a
 multipart
-head:ClamSMTP
+head:Content-Length
 message
-head:ClamSMTP
+head:Content-Length
 in
 MIME
-head:ClamSMTP
+head:Content-Length
 format
-head:ClamSMTP
-*** 94 bt  1 
-textblock.c:64  0x7fdcb140bda0 0x7fdcb140be60   1 *add* cur: 7735, max: 7735, tot: 7735
+head:Content-Length
+*** 95 bt  1 
+textblock.c:64  0x19b4060 0x19b4130   1 *add* cur: 7782, max: 7782, tot: 7782
 *** yyinput("
 ", 44, 8192) = 1
 *** got_mime_boundary:  stackp: 0, boundary: '------=_NextPart_000_0001_01CE6816.3B3807A0