README.ext3

[Matthias Andree]

Greg Louis <glouis at dynamicro.on.ca> writes:

> On 20030201 (Sat) at 1612:36 +0100, Matthias Andree wrote:
>
>> We'd better get the performance issues fixed, or if there's a bug, we'd
>> better get that reported. ext2 is way inferior to ext3 in terms of
>> consistency, recovery or robustness.
>
> Recovery I will buy; can you supply pointers to evidence for the other
> two?

ext2 performs unordered writes of any dirty data in buffers it finds at
the time. This easily breaks your file system -- files ending up in
lost+found are commonly seen on ext2 after a crash, as well as
freshly-created files with incomplete data. This also happens on
reiserfs, not just ext2. I'd call that consistency and robustness.

No, I don't have documentation handy beyond mount(8) of a recent system
that explains what data=ordered does (and that this is the default).

>>, so it might really be a tuning issue or an
>> issue with the kernel version that you're using.
>
> Yes, that is true, it might.  We need to know.  What kernel version are
> _you_ using? -- I have one machine I could borrow to check that.

SuSE's k_athlon-2.4.19-167, for SuSE Linux 8.1. However, given the
recent results on the ext3-users list, I don't think there's much for
the kernel to tune. We need to get our overwrite rate down from the 99%
you're observing. If we pass like 2 GB down to the drive to get 20 MB
out, that's expensive. I think I've taken Andrew Morton's point here.

>> Plus, priming the data
>> base with some training data is an operation that isn't performed very
>> often, so we can live with that.
>
> The email to which you were replying mentioned that with ext3 a
> _classification_ takes me four times as long as if the db files are on
> ext2.  Maybe _you_ can live with that...

It totally missed that point -- unless you're using -u, the read
performance of ext3 and ext2 should be quite similar. I can however
imagine other application's writes get in the way.

> Yes, I agree with you here.  Some sort of external recovery strategy is
> definitely needed if the db files are on ext2 or ext3/writeback.  As
> in, "he who laughs last probably made a backup."  Probably the warning
> I gave is worded too gently.

There's nothing short of regularly dumping the data bases that makes the
whole beast really solid. We'd need to use transactions if we wanted
mostly-complete robustness.

> Of data=ordered.  Writeback is only slightly slower than ext2.  Just a
> typo, no doubt.  I too was interested that journal is almost twice as
> fast as ordered, but I don't know enough about db internals to turn
> that to advantage.

I can't estimate how much data=journal helps -- it likely just
accelerates the terminal fsync() operation because the kernel can just
dump the whole buffer stuff into the journal, report completion, and
trickle the data from the journal back to the standard locations in the
actual file system.

-- 
Matthias Andree