On setting and meeting goals, and statistical fun

While NMUing plays an important part in getting us to a release, as shown by the thousand or so fixed-in-NMU RC bugs that we recently cleaned up the BTS state on, it's evident after doing this whole release-managing thing for a while that there is a very high (non-linear) correlation between the age of an open RC bug and the likelihood that a release manager is going to have to touch it at some point in order to get it fixed. By and large, maintainers really do a good job of fixing their own packages, which is probably something we don't say enough - maybe 75% of all RC bugs, I'd guess, will (or would) get fixed by maintainers in the first week without anyone needing to poke them about it, and of the remainder, probably another 80-90% get taken care of by BSPers and serial-NMUers like bug-a-day mascot Steinar Gunderson. That's why I'm not bothered overly much by the idea of full-archive searches for hidden RC bugs, like Lucas's piuparts tests, affecting the release timeline, because I know that 95-98% of those bugs are going to be gone two weeks after they're filed anyway, especially when they're no-brainer fixes to maintainer scripts or dependencies.

But then that leaves the other 2-5%, which are the ones nobody else has fixed. And why hasn't anyone fixed them? Because they're the hard ones, of course...

So although I follow the debian-bugs-rc list and occasionally swat a few non-bugs closed to save the maintainers time, and also provide some nudges when it looks like a package is causing RC bugs to be filed on other packages, my goal for the last full week of my Dunc-Tank contract was to get the count of etch RC bugs unfixed in unstable that are 20 days old or older down from around 50 to under 30. This seemed a reasonable goal, since it had been at 80 the week before, and it would go a long way towards helping the real blockers on the release. Initially I was aiming to bring down the count of RC bugs unfixed in testing that are 20 days old or older to this number, but quickly realized that this wasn't going to be realistic because it depended too much on the autobuilders all operating optimally and there wasn't time to both help fix 20 of the hardest RC bugs and handhold 60 RC bugfixes into testing in one week.

In the end, we did miss this lesser target by a day or two. Still not a bad week's work, given that I'm pretty sure I fixed about 20 of these bugs myself in that time period (through NMUs, removals, and analysis+downgrades) and they seem to average 3 hours of work for a fix. And we've been holding at or below 30 on this list of bugs since then, so we're successfully keeping up with new bugs aging past the 20-day mark, which is also a good sign.

Unfortunately, I seem to have left myself to fix the alpha Xorg 7.1 bugs on my own time. :)

By now the announcement has made the rounds that the release team doesn't think we can hit the Dec. 4 target date because more time is needed for a final release candidate of the installer. Does that mean that RC bugs were the wrong thing to be focusing on for the past weeks? Well, no; if you want one place in Debian where Brooks' Law holds, I think you'll find that getting a d-i release out is it, and we still have a hard road ahead to make sure we do get our other RC issues sorted out in the same time frame. The installer just happens to put a lower limit on a release date, we still have an opportunity to screw it up from there. ;)

At least the statistics suggest that we're going in the right direction.

Only 13% of RC bugs fall into the "20 days or older and unfixed" category. Half of all RC bugs over 10 days old have a fix of some kind in unstable, and 64% of all reported RC bugs are less than 10 days old.

We'll see whether this graph backs me up on this in a couple more days.