SPAM processing server

[Ramesh Natarajan]

I am planning to implement a distributed spam processing messaging
application. My application server(s) will monitor chat/text messages
exchanged in conversations and tag messages as spam if any inbound message
is a spam. I am planning to use bogofilter to identify if the conversation
is a spam or not.  These messages are not email messages but text messages
you will typically see in a typical chat or conversation.

I am expecting these servers to process thousands of conversation per
sec. When I tried using bogofilter and fork off a shell/process to classify
a message the performance is quite unacceptable. I was able to get decent
performance using batch mode and may be if i work around the process per
message limitation, this would be a viable solution.

Since bogofilter is written in C, I am wondering if I can  launch a tcp
based persistent server processes that directly call the bogofilter
routines to classify the message. I would probably need to have options to
delimit messages and support concepts of transaction/session e.t.c so
multiple sessions can be run in parallel.

Does anyone have any thoughts on this?

Thanks
Ramesh