Fear and Loathing in Les Rambles

bzr-git case study

Robert Collins did a very nice writeup not so long ago of creating a Debian package with bzr-builddeb, but he starts from the assumption that upstream also uses bzr. As we all know, there are many projects that have their upstream sources in a VCS, but don't use bzr. Can we still get reasonable results from bzr-builddeb, including the ability to merge new upstream releases direct from the VCS, if, say, upstream is using git?

Yes, we can!

Of course, if you're happy with using git as the VCS for your Debian packaging already, there's probably no compelling reason for you to switch to bzr. But if you prefer to use bzr and have been frustrated at having to choose between being able to use the VCS client of your choice for your packages and being able to use your VCS client to access upstream revision history, read on.

Thanks to the fine work of Jelmer Vernooij and friends, bzr-git has been usable for a while if you just want to track the default git branch. But what if you care about tracking multiple upstream branches?

Enter bzr git-import, which we'll use to set up a bzr repo from scratch for our cifs-utils package, with a full import of all the branches of upstream's git tree.

First we do some work to set up a shared bzr repository. As long as we're going to the effort of mirroring all the git branches, we probably want to publish them for other people to use, so we make sure our bzr repository is reasonably configured for sharing data between branches:

$ bzr init-repo --no-trees cifs-utils.server
Shared repository (format: 2a)
Location:
  shared repository: cifs-utils.server
$ cd cifs-utils.server

Then we use bzr git-import, provided by the bzr-git plugin, to do a full import of the upstream branches into a cleverly named subdirectory:

$ bzr git-import git://git.samba.org/cifs-utils.git upstream
[/                   ] Counting objects: 181, done.

Now we create our own local 'trunk' for the package, by branching from the upstream tag matching the release version we're going to package.

$ bzr branch -r tag:cifs-utils-4.0rc1 upstream/HEAD trunk
Branched 42 revision(s).

Do a little more prep work for future branches...

$ mkdir branches

And that's it. Now we have a bzr repo locally that we can push out to our hosting server of choice (N.B.: we could do this all directly on the server, but alioth doesn't currently have bzr-git installed):

$ rsync -az . alioth.debian.org:/bzr/pkg-samba/cifs-utils/
$

Now that we have a bzr repository, it's time to get ourselves a working directory and do some packaging. We're probably going to work with multiple branches locally as well, so we create another shared repository for local use, and get a copy of the trunk branch we created before.

$ bzr init-repo cifs-utils
Shared repository with trees (format: 2a)
Location:
  shared repository: cifs-utils
$ cd cifs-utils
$ bzr co bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/trunk

Now we grab the upstream tarball, and whip up some quick packaging with (what else?) debhelper 7:

$ wget ftp://ftp.samba.org/pub/samba/cifs-utils/cifs-utils-4.0rc1.tar.bz2
$ ln -s cifs-utils-4.0rc1.tar.bz2 cifs-utils_4.0~rc1.orig.tar.bz2
$ cd trunk
$ mkdir -p debian/source
$ echo '3.0 (quilt)' > debian/source/format
$ echo 7 > debian/compat
$ cp /usr/share/doc/debhelper/examples/rules.tiny debian/rules
$ dch --package cifs-utils --versio 4.0~rc1-1 --create 'Initial package'

Create debian/control by hand, generate an initial source package, and import it into bzr with bzr-builddeb.

$ debuild -uc -us -S -i
$ rm -r debian
$ bzr import-dsc ../*.dsc
$

Since we want to continue tracking upstream development, we need to periodically sync our bzr import of the git tree.

$ bzr git-import git://git.samba.org/cifs-utils.git bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/upstream
$

And when we find a new upstream version in that import that we want to package, bzr-builddeb makes this a snap, too.

$ cd cifs-utils
$ wget ftp://ftp.samba.org/pub/samba/cifs-utils/cifs-utils-$version.tar.bz2
$ cd trunk
$ bzr merge-upstream --v3 --version $version_with_epoch \
    ../cifs-utils-$version.tar.bz2 -r tag:cifs-utils-$version \
    bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/upstream/HEAD
No distribution specified, and no changelog, assuming 'debian'
Committing to: /tmp/tmpbzOVMs/upstream/
Committed revision 2.
All changes applied successfully.
The new upstream version has been imported.
You should now review the changes and then commit.
$ bzr diff
<review changes>
$ bzr commit -m "merge upstream $version"
Committing to: bzr+ssh://bzr.debian.org/bzr/pkg-samba/cifs-utils/trunk
modified debian/changelog
Committed revision 50.
$

And that's it! It's exciting to see signs of real interoperability between DVCSes at last. Thanks to everyone who's helped make it possible!

Posted Tue Jun 1 13:39:56 2010

bus factor

I am very glad to know that the Debian train was not hit by a bus on its way back to Madrid.

Heat + a head cold are not a good mix. But after a night of fitful sleep, I seem to be doing better and may actually make it out to see some things in the city today (instead of wandering out for dinner, whining at the heat and immediately returning to my room). I'm sorry to not have been able to meet up with people yesterday - apparently Madrid is swarming with DDs right now - but there's always next year in New York!

Tentative plans for the day: this, then this.

Posted Sat Aug 1 01:53:04 2009

DebConf9 and the joys of Spanish rail

Am I going to DebConf9? Yes.

Do I know how I'm getting there? No.

$ TZ=Europe/Madrid date
Tue Jun 16 23:50:43 UTC 2009

FF CC HH de PP.

Posted Tue Jun 16 17:11:31 2009

Configuring-an-old-Debian-system

Chris,

Please note that the method you recommend for activating libpam-cracklib only applies to lenny and earlier; it's obsolete for the upcoming squeeze release (and in Ubuntu 8.10 and later), where libpam-cracklib is integrated with pam-auth-update and will be autoconfigured. Running your sed command on a squeeze system will break pam-auth-update integration, and cause the admin headaches going forward if they ever have occasion to use anything other than pam_unix for authentication.

So for a squeeze system, this should be:

apt-get install libpam-cracklib

... and you're done!

Actually, the package defaults in squeeze are stronger than what you've listed: libpam-cracklib will set minlen=8 by default (instead of the uselessly-weak minlen=6), and coming soon, pam_unix should default to sha512 instead of md5 for password hashing.

Posted Thu Jun 4 13:18:27 2009

There are no words to describe

Sign on the lunch buffet the other day in Berlin:

Antipasti von Aubergine, Zucchini und Champignon

Posted Sun Feb 8 12:18:16 2009

First snowfall of the next ice age

Pshaw, and they say it only ever rains in Portland.

Posted Sun Dec 21 19:06:22 2008

Palin is great

Like Jaldhar, I too am thrilled at the news that McCain has chosen Palin as his running mate. Finally, a candidate who embodies the truly American spirit of the Republican party!

Haven't we had enough political circus in Washington? Isn't it time to give Flying Circus a chance?

Posted Tue Sep 2 15:43:31 2008

A Portland, la ciudad de Mejores Aires

I'm back (for several days now) from Argentina and DebConf 8 to Portland. Too long a journey and too short a trip, as always. It was great to see everybody again as well as to meet a number of new folks from the local community. Here's hoping that DebConf 8 inspires more involvement in Debian from Argentina!

It was my first time to this wonderful South American country, which made it even more melancholy to have to leave after such a short time and to be making this trip without Patty.

Random observations about Argentina, for posterity:

• The 6-hour bus ride from Buenos Aires to Mar del Plata looks an awful lot like Iowa. Except for the occasional palm tree and the fact that it's 20°C in winter.
• Well, ok... Iowa also tends not to have quite so many cows. Or country grills (parrillas) right next to pastures...
• All the towns along the route are named after generals or colonels?
• All the police on the highway wear OSHA green vests.
• Following the German with the voluminous guidebook is an effective way to acquire generous portions of bife de chorizo.
• People from Bahía Blanca seem to have a somewhat more paranoid opinion of what constitutes a "dangerous neighborhood" than I do. :)
• "Buenos Aires" appears, I'm sad to say, to be a marketing lie. There was nothing bueno about the aire on the autopista on the ride to the airport. But the air in Mar del Plata was quite nice, and the people were the best!

As always it was a pleasure to take part in another annual DebConf, an event which is such a positive force for rejuvenating the Debian community.

And BTW, Biella, NYC is not the only US city entertaining a bid for DebConf 10.

Posted Sat Aug 23 17:21:53 2008

bad security is worse than no security

Pierre,

"Why on earth is https with an untrusted certificate less secure than http ?."

Apparently you thought this was a rhetorical question. I disagree. https with an untrusted certificate is less secure than http, because whether or not most users understand the meaning of https, there are a significant number of users who do understand what it means, or at least think they do. You can't seriously tell me you that you think all users always look for the lock icon instead of looking at the URL to know whether they're protected, can you? Users, on the whole, are very good at cargo-culting where technology is concerned. Just because many users look for the icon does /not/ mean that there aren't other users who know just enough of the definition of "protocol" to be a danger to themselves.

An https URL is a declaration on the part of the server (or the party linking to it) that the resource in question should be secure. This creates an expectation on the part of those users who know what https is that if the connection succeeds, it is secure, which means they'll think it's ok to do the kinds of things that they normally do over secure connections but won't do over unsecured ones.

It therefore certainly is important to give users feedback that an https connection has failed. If you try to connect to an https resource, and your browser can't verify the certificate, something is wrong. Either the server operator is a stooge for Intel trying to drive up the client CPU requirements for web connectivity so that they can sell more chips, or the server operator has failed to establish a chain of trust to you in the appropriate manner, or someone really has compromised the connection with a man-in-the-middle attack. Any of these conditions are something that ought to be brought to the user's attention, not ignored.

If you don't like seeing cumbersome security warnings for insecure https connections, how about not using https when what you really want is http in the first place?

Posted Thu Jun 26 23:08:32 2008

Did you hear that etch is out?

One of the reasons I tend not to blog very much is that I don't have much to say to the Planet Debian audience (the only place I know my blog is syndicated) that hasn't been said a dozen times before. But the world's best release party (the one good enough to cross an ocean for) gave me food for thought, and for once I'm inspired in the true spirit of blogging to write even if nobody cares what I have to say. Even if it has taken me until two months after the release to write it...

Folks within the Debian community have been asking for years whether it's really still worthwhile for Debian to do stable releases, or if we're better off focusing on our "core competency" of testing and unstable and leaving stable releases to derivatives. For me, the answer to this question is an unqualified: yes, it's worthwhile.

Of course that's the answer you'd expect from a release manager. Who would want to believe after committing as much time as the release team does that their efforts had been wasted? But the investment I've made in the etch release wasn't done without reflection; I recognize that the level of involvement required to be a release manager has come at some personal cost to me, as it does for each member of the release team, and I have made the investment anyway because I believe in Debian's philosophy, its goals, and its community.

Even outside of a stable release, Debian packages really are great; in spite of the bugs and library transitions that we as release managers spend our time on in unstable and testing, and figures showing that within a month of the etch release we already had some 500 RC bugs identified for lenny, both unstable and testing do a great job of meeting the needs of many of our users. And package maintainers are constantly improving their packages in unstable, fixing bugs and adding new features. Unstable is a flowing river, vibrant and vital, never the same twice -- and only rarely are people dragged to their deaths by the undertow. ;)

Maintainers often don't get to choose the version of their package that goes into stable. When it comes to a stable release, the bug you know is quite often better than the bug you don't. But although the release team is charged with chasing out the release-critical issues in preparation of a stable release, it isn't just the release team who deserves credit for etch. Etch is a monument to the work of all Debian maintainers, who have each helped shape the release with their contributions. "Stable" doesn't mean "perfect", but it does mean "lasting": in etch we have not only created what I think is the highest-quality OS I've ever had the privilege to be involved in, we have created something permanent that will be burned to thousands or millions of disks, something that users will continue to use and install for years to come. Far from standing in opposition to unstable, it is a perfect complement, letting us share everything that's great about Debian with users for whom tracking a constantly-changing testing or unstable simply isn't an option.

I had decided well before etch was finished that it was time for me to move on from serving as release manager, but I still look forward to great things from Debian stable for lenny and releases beyond. With last weekend's release team meeting in Juelich, and the wonderful collaboration that's sure to take place this week at DebConf 7, I think we're off to a great start. The grumbling on IRC about broken packages and missing builds and uncoordinated library transitions is already in full swing for lenny -- and I'm no small contributor to that myself even now -- but we shouldn't let that blind us to what a great achievement etch has been for the Debian community, and what a great achievement lenny will be when it's released exactly on schedule next year.

Posted Sat Jun 16 08:49:47 2007