Storing files: 2.3TBytes, 17M file count

Hi,

PostgreSQL is rock solid and one of the most reliable parts of our toolchain.

    Thank you

Up to now, we don't store files in PostgreSQL.

I was told, that you must not do this .... But this was 20 years ago.


I have 2.3TBytes of files. File count is 17M

Up to now we use rsync (via rsnapshot) to backup our data.

But it takes longer and longer for rsync to detect
the changes. Rsync checks many files. But daily only
very few files really change. More than 99.9% don't.

Since we already store our structured data in postgres, I think
about storing the files in PostgreSQL, too.

What is the current state of the art?

Is it feasible to store file in PostgreSQL?

Are there already projects which use PostgreSQL as storage backend?

I have the hope, that it would be easier to backup only the files which changed.

Regards,
    Thomas Güttler


Related question at rsnapshot mailing list:
https://sourceforge.net/p/rsnapshot/mailman/rsnapshot-discuss/thread/57A1A2F3.5090409@thomas-guettler.de/



--
Thomas Guettler http://www.thomas-guettler.de/