Re: ML archives caching 404 results

On Tue, Oct 7, 2014 at 4:03 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Magnus Hagander <magnus@hagander.net> writes:
>> On Tue, Oct 7, 2014 at 3:52 PM, Marti Raudsepp <marti@juffo.org> wrote:
>>> When adding my messages to CommitFest, I noticed that when I'm fast enough
>>> to click on the resulting link, I get a 404 page from mailing list archives,
>>> I guess if archives loader hasn't processed my message yet. This 404 result
>>> gets cached for a long time, so my message is not viewable even after links
>>> to it appear in archives.
>
> FWIW, I've been annoyed by that too ...
>
>>> Is there a default expiration time set in Varnish somewhere? Maybe the
>>> solution is as easy as setting a lower TTL for 404 results in Varnish:
>>> http://www.garron.me/en/bits/avoid-varnish-cache-404-error-page.html
>
>> Yes we cache the 404 pages, and that's definitely intentional.
>
> It might be intentional, but is it really useful?  How much traffic do
> we get to nonexistent pages?

From broken search engines running amok every now and then - quite a
bit. During normal operations, not much.

But we can probably easily drop it to say 5 minutes or so for 404's -
would that be enough?

-- Magnus HaganderMe: http://www.hagander.net/Work: http://www.redpill-linpro.com/