Re: Heroku early upgrade is raising serious questions

On Mon, Apr 8, 2013 at 06:58:57PM -0400, Jonathan S. Katz wrote:
> In an ideal world - yes - everyone should have access to critical
> security bugfixes at identical times.  However, due to the sensitivity
> of certain systems (I do not think any of us would want to see a
> hospital's system go down due to a security exploit), there does need
> to be an early access list.  The way keep it fair for that is to have
> a committee of various PGDG members with different backgrounds to
> review applications and vote to decide who should have early access
> (where access is not even necessarily the source - access could even
> be just to the binaries).  I know one of the goals is to keep that
> list small, and I do agree with that, but more importantly, there
> should be guidelines on how to maintain membership to that list.

Good analysis --- let me add a few things.  First, as others have
pointed out, Heroku did nothing wrong.  They followed the rules set by
the core/security teams --- if they had been given different rules, they
would have acted differently --- so any blame on the outcome has to be
assigned to the core/security teams.

Second, not only is Heroku uniquely exposed to the bug, they don't
supply source or even binaries to users, hence an early deployment had
little risk of a security leak.  (They also produce a custom build with
additional features, e.g. JSON.)  Also, they did a rolling deployment
and provided daily reports to the security team about their progress and
lack of problems, so their early deployment had some testing benefit to
the community.  (The buildfarm did no testing of the security patches
due to the git mirror blackout.)

I think the unfortunate net effect is that, in this case, distributors
who had to provide source or release notes were actually hampered in
providing early deployment binaries to users.

If you look at each issue individually, allowing binary-only,
no-release-note distributors to deploy early actually makes sense, but
seen in a larger context, it looks more doubtful.  There was some sense
that this was such an unusual release that _safety_ was given such great
importance that some other criteria were overlooked --- perhaps rightly,
perhaps wrongly.

As Josh Berkus already mentioned, the core team is still debriefing all
these issues and how this was handled and trying to improve its
processes.  This is only my personal analysis, not that of the core team
as a whole.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +