bogofilter-static-1.0.0-1 RPM problem (CentOS 3.6)

[Matthias Andree]

bogo at escom.com writes:

> Since I've been the cause of a lot of traffic on this list recently,
> I wanted to clarify why I'm asking all these bizarre questions,
> and why it may seem that I can't make up my mind what version of 
> Linux I want to run.

Al,

"Bizarre" is something else than asking portability questions :-)

> The reason is that I'm asking mostly on behalf of ESCOM's customers,
> of which more than half have Windows backgrounds, and who may be
> running anything from Red Hat 7.1 to this year's Red Hat Enterprise 
> Linux.  And asking our Windows customers to download your source and 
> compile it is just something they don't have the time, inclination,
> or training to do.  Even one ISP customer, who runs Linux on everything,
> says he doesn't have time to keep up with all this since he has a LOT 
> of other things he has to do, every day.

Well, the process itself is as simple as
"rpmbuild --rebuild bogofilter-1.0.0-1.src.rpm" - but I'd think people
would have to know which other RPMs to install beforehand, and this list
is going to be different with every system...

> There's another problem.  We wrote HOWTO files on installing various
> Red Hat versions to support our software, and we told them they
> didn't need the Development RPMs. I doubt if any of our customers have
> compilers on their servers, and the Microsoft customers may not even
> have another computer in their organization that's running Linux.

...and if they don't have staging computers with similar hardware to the
servers, they may be torn between "want to compile source" and "don't
ease privilege escalation, don't provide compilers on servers".

I believe I see the problem now.

> But we won't be successful in getting our customers to use Bogofilter
> unless we compile it here and give them a source+binary ISO to install.
> But to do that, since some customers are running RH 7.1, I had to build
> it on a 7.1 system and then bundle Sleepycat's libdb-3.1.so with it.  
> Yeah, I know, not an optimal solution.

Perhaps it's just that, a local optimum: 

1. you can control the Berkeley DB compilation parameters in this case,
   such as which mutex implementation to use (see the Berkeley DB
   manual, docs/ref/build_unix/conf.html in the source tarball), which
   architecture to compile for and the like.

2. you can control that bogofilter uses that particular Berkeley DB by
   installing your Berkeley DB under /opt/db-3.1.17-for-bogofilter and
   using -rpath as linker option when compiling bogofilter (for
   instance, you'd use -Wl,-rpath -Wl,/opt/db-3.1.17-for-bogofilter/lib -ldb
   in LIBS at ./configure time)

   Unless db-3.1 has been widely deployed yet, I'd suggest to use a more
   recent Berkeley DB, 4.1 or newer.

> That's why I would like to have a bogofilter-static RPM that my
> customers could download and that they could install using scripts
> that I would provide.
>
> Matthias Andree's message of a couple days ago goes a long way to 
> explaining why this may not be possible: 
>
>> What Chris is saying is that although the executable may have been
>> linked statically, it may still have inherited dependencies on the
>> processor type, mutex support or kernel of the build machine through the
>> statically linked Mandrake Berkeley DB.
>> 
>> Remember the problem that transactions wouldn't work on some
>> Fedora/processor combinations because only private environments were
>> supported?  I have forgotten the details, the grand picture was that
>> their Berkeley DB had compiled NPTL (next-generation POSIX thread
>> library) stuff in, which worked only for the latest processor
>> generation.
>
> Matthias' points also indicate that I might even expect problems with
> my lowest-common-denominator distributions.

My statement was based on two assumptions: 1. the binaries are compiled
on David's system usually, which is a pretty recent Mandriva
distribution on i686 (thus unfit for Pentium MMX and K6 for instance,
Pentium Pro and Slot-A Athlon is the oldest that might work), 2. current
code doesn't work well (if at all) with older libc and older kernels.

If you have the chance to ship your custom-made libdb-bogocompat or
however you'd call the package, it's easier. A package compiled on a
i586 RH 7.1 machine should work as long as all required libraries are
there, if the libdb goes out of its way depending on the threading
libraries for mutexes.

Berkeley DB can use gcc-assembly (test-and-set) mutexes on many machines
(among them MIPS, ARM, x86, x86_64, ia64, perhaps sparc), run

       egrep '^[-_0-9a-zA-Z/]+\)' dist/aclocal/mutex.??

in the Berkeley DB source directory to find out which mutex kinds the
particular DB version supports. Avoid the "private" and "intra-process"
variants though, they won't work as we need a "public" or "shared" or
"inter-process" mutex.  I cannot give recommendations which
mutex to use though.

Experimenting with these, you may find the one configuration that works
on all systems.

> And so we are left with some sites that are willing to use Bogofilter,
> but only if it is pre-built for them, and all kinds of reasons why
> it may not be possible to pre-build a binary distribution.  This is
> not meant to be negative in any way regarding Bogofilter -- it just 
> may not be possible.

Another possibility that I'd like to suggest evaluating is a bogofilter
version that uses SQLite 3.2.6 or newer.  It may be a bit slower than
Berkeley DB and a bit more CPU bound (it's not a groundshaking
difference though), but it goes out of the way of all the mutex, log
file and environment hassles that bogofilter's doc/README.db warns
about.

-- 
Matthias Andree