Problem compacting databases (again!)

[Matthias Andree]

"Juan J. Martinez" <reidrac at blackshell.usebox.net> writes:

> Hello,
>
> It happened again:
>
> # bogoutil -d wordlist.db | bogoutil -l wordlist.db.new
> # bogoutil: Unexpected input [d'informÃ] on line 25173. Expecting 
> whitespace before count.
>
> It's the same bug last time (the same word also!).

Juan,

does this patch help?

If it does, please file an OpenBSD bug report stating that isspace()
falsely considers 0xa0 a space in the POSIX locale.

Index: src/bogoutil.c
===================================================================
RCS file: /cvsroot/bogofilter/bogofilter/src/bogoutil.c,v
retrieving revision 1.172
diff -u -c -r1.172 bogoutil.c
*** src/bogoutil.c	17 Jan 2005 03:00:47 -0000	1.172
--- src/bogoutil.c	23 Jan 2005 22:19:22 -0000
***************
*** 109,119 ****
  }
  
  #define BUFSIZE 512
  
  static byte *spanword(byte *t)
  {
!     while (isspace(*t)) t += 1;		/* skip leading whitespace */
!     while (*t && !isspace(*t)) t += 1;	/* span current word	   */
      if (*t)
  	*t++ = '\0';
      return t;
--- 109,122 ----
  }
  
  #define BUFSIZE 512
+ const char POSIX_space[] = " \f\n\r\t\v";
  
  static byte *spanword(byte *t)
  {
!     /* skip leading whitespace */
!     t += strspn(t, POSIX_space);
!     /* span current word */
!     t += strcspn(t, POSIX_space);
      if (*t)
  	*t++ = '\0';
      return t;

-- 
Matthias Andree