Обсуждение: Lessons from using mmap()

This blog, and the blogs it links to, explains the complexities of using
mmap() for database data/index file I/O.

    https://www.symas.com/post/are-you-sure-you-want-to-use-mmap-in-your-dbms

The blog starts by stating:

    There are, however, severe correctness and performance issues
    with mmap that are not immediately apparent. Such problems make it
    difficult, if not impossible, to use mmap correctly and efficiently
    in a modern DBMS.

The remainder of the article makes various arguments that such mmap use
is _possible_, but ends with a reasonable conclusion:

    Ultimately, the answer to the question "are you sure you want
    to use mmap in your DBMS?" should be rephrased - do you really
    want to reimplement everything the OS already does for you? Do
    you really believe you can do it correctly, better than the OS
    already does? The DBMS world is littered with projects whose
    authors believed, incorrectly, that they could.

I think we have come to the same conclusion in the past, but I thought
it would be good to share someone else's research, and it might be
helpful if we ever revisit this idea.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

Hi!

On Wed, Feb 21, 2024 at 1:21 AM Bruce Momjian <bruce@momjian.us> wrote:
> I think we have come to the same conclusion in the past, but I thought
> it would be good to share someone else's research, and it might be
> helpful if we ever revisit this idea.

I read this blog post before. In my personal opinion it is an example
of awful speculation.  If your DBMS needs WAL, then your first
question about using mmap() for your data is how to enforce WAL to be
really ahead of writing the data pages.  As I know there is no
solution for that with mmap() (or at least a solution with acceptable
portability).  The blog post advertises LMDB, and LMDB is really good.
But LMDB uses copy-on-write instead of WAL to ensure durability.  And
it supports a single writer at a time.  This is just another niche of
solutions!
The blog post makes an impression that developers of non-mmap()
DBMS'es are idiots who didn't manage to use mmap() properly.  We're
not idiots, we develop DBMS of high concurrency! :-)

------
Regards,
Alexander Korotkov