A trick with perl

[Chris Fortune]

Hello Andrew,

You said :
"
  > You're heading for headaches because a pipe can only buffer so much
  > input, so your program will hang when you have a long message.
"
But I did some tests and was able to make several large FIFO files, the
largest being over 105Mb with perfect data integrity, and without hanging.
I haven't experienced any problems in production, with emails of 1-200kb in
size.  The average size of an email is < 5kb, so I don't forsee any buffer
size problems in the future.  At least not on Linux RH8 with 512Mb RAM.  I
should measure speeds comparing FIFO access to regular file disc access
times: my textbook tells me it should be a lot faster.

You are correct about the ^D character, I did some more tests and discovered
it was not necessary because a FIFO file stream terminates with the EOF
character automatically.  This appeared to be necessary with SpamAssassin
(and was necessary on the command line), but SA was hanging not because of a
missing EOF char, it was hanging because it was just so darn slow!

Cheers!

----- Original Message -----
From: "Andrew Pimlott"
To: <bogofilter at aotto.com>
Sent: Saturday, January 03, 2004 9:02 PM
Subject: Re: A trick with perl


> On Sat, Jan 03, 2004 at 04:51:09PM -0800, Chris Fortune wrote:
> >         # Loop for each email
> >             open NP, ">$RAM_file.FIFO" or die $!;
> >             print NP "$head\n$body\n^D"; # ^D is EOF, just in case
> >             close NP;
>
> You're heading for headaches because a pipe can only buffer so much
> input, so your program will hand when you have a long message.
> Using Open2 has the same problem.  The simplest solution is to use a
> temporary file at one end:
>
>     open FILTER, "| prog > result";
>     print FILTER "$head\n\n$body";
>     close FILTER;
>     open RESULT, "<result";
>
> Also, that ^D is bogus.  ^D only means end of file when pressed in a
> terminal; it does not appear in the file stream and can only confuse
> the filter.
>
> Andrew
>
> ---------------------------------------------------------------------
> FAQ: http://bogofilter.sourceforge.net/bogofilter-faq.html
> To unsubscribe, e-mail: bogofilter-unsubscribe at aotto.com
> For summary digest subscription: bogofilter-digest-subscribe at aotto.com
> For more commands, e-mail: bogofilter-help at aotto.com
>
>