Re: PostgreSQL passes MySQL for Freshmeat Downloads

On Fri, Jun 13, 2008 at 5:19 PM, Josh Berkus <josh@agliodbs.com> wrote:
>
> It would be way keen.  I'm pretty sure WWW will be happy to host it on
> community servers.
>

Great; I'll do it! Hosting is not a problem for me; I have a server of
my own colocated on a reliable network, and an account on another
machine in a carrier hotel. But if it's easy to get it hosted on a
PostgreSQL-controlled server, all the better.

> Do you think you could make it extensible?  So that we could have it check
> SF.net, Wikipedia, etc.?

Wikipedia is doable for sure. The "edit this page" URL never changes
and it's very easy to separate out the wiki content in the textarea
from the surrounding HTML.

sf.net is probably also fine, but it's fairly dynamic and there's no
clear delineation between content and boilerplate. However, just
looking for the string "8.3.3" in the HTML of the download page turns
up 5 matches and no false positives. The moral is, "boneheaded web
scraping works far better than you would hope or expect." :)

-Jonathan