Re: Database versus filesystem for storing images

2007/1/6, Maurice Aubrey <maurice.aubrey@gmail.com>:
> Clodoaldo wrote:
> > But the main factor to push me in the file system direction is the
> > HTTP cache management. I want the internet web clients and proxies to
> >  cache the images. The Apache web server has it ready and easy. If
> > the images where to be stored in the DB I would have to handle the
> > HTTP cache headers myself. Another code layer. Not too big a deal,
> > but if Apache give me it for free...
>
> There's a hybrid approach which has worked well for us.
>
> You store the binary data in the database along with a signature.
>
> On the Apache side, you write a 404 handler that, based on the request,
> fetches the binary from the database and writes it locally to the
> filesystem based on the signature (using a multi-level hashing scheme
> possibly as detailed in previous posts).
>
> When a request comes in to Apache, if the file exists it is served
> directly without any db interaction. OTOH, if it's missing, your 404
> handler kicks in to build it and you get a single trip to the db.
>
> You get the benefits of keeping the data in the db (transaction
> semantics, etc.) but also get the scalability and caching benefits
> of having the front-end webservers handle delivery.
>
> If you lose the locally cached data it's not an issue. They'll be
> faulted back into existence on demand.
>
> With multiple webservers, you can just allow the data to be cached on
> each machine, or if there's too much data for that, have your load
> balancer divide the requests to different webserver pools based on the
> signature.
>
> As an extension, if you need different versions of the data (like
> different sizes of an image, etc.), you can modify your URLs to indicate
> the version wanted and have the 404 handler take that into account when
> building them. You only store the original content in the database but
> could have any number of transformed versions on the webservers. Again,
> losing those versions is not an issue and do not require backup.

Very interesting approach. And I think it is also original as I have
not seen any mention of it. Thanks for sharing it.

--
Clodoaldo Pinto Neto