OT - duplicate emails

[Tom Allison]

David Relson wrote:
> On Tue, 07 Sep 2004 14:37:22 -0700
> .rp wrote:
> 
> 
>>>If we are talking about simple duplicated messages, then I would
>>>like to suggest a procmail ruleset of:
>>>
>>># remove duplicate emails
>>>:0 Wh: msgid.lock
>>>| formail -D 2048 msgid.cache
>>>
>>
>>How does this work? We often get dictionary attacks that SA or BF
>>pickoff, but if we can limit the work using this ...
> 
> 
> Run "man formail" for an explanation of the "-D" option.
> 
> Note:  The procmail snippet is incomplete since it doesn't check
> formail's return code.
> 

man procmailex and I got the snippet from there, so I hope it's complete 
(sorry to sound defensive).  It seems to work OK.  I dropped the size 
since I have a low volume server I didn't think I needed 8K and most 
duplicates are right on top of each other.

to quote:

If  you are subscribed to several mailinglists and people cross-post to
some of them, you usually receive several  duplicate  mails  (one  from
every  list).   The following simple recipe eliminates duplicate mails.
It tells formail to keep an 8KB cache file in which it will  store  the
Message-IDs  of  the most recent mails you received.  Since Message-IDs
are guaranteed to be unique for every new mail, they are ideally suited
to  weed  out  duplicate mails.  Simply put the following recipe at the
top of your rcfile, and no duplicate mail will get past it.

               :0 Wh: msgid.lock
               | formail -D 8192 msgid.cache