The Eclipse Common Build Infrastructure

Creating an Common Build Infrastructure (CBI) at Eclipse has been one of our holy grail pursuits over the years. My main goal in this effort has been to get Eclipse committers out of the build business so that more time and energy can be focused on actually writing software (which is what most software developers would rather spend their time doing).

When we talk about “build”, we mean the process of taking source code and turning it into a form that adopters and users can download and consume. Building software in general–and building Eclipse plug-ins/OSGi bundles in particular–is a relatively hard problem. The original CBI at Eclipse used a combination of PDE Build, black magic, cron, and manual intervention in the form of pulled hairs.

Our first attempt at trying to make this easier for Eclipse projects resulted in the Athena Common Build. Athena added a layer on top of PDE Build that made it quite a lot easier to build Eclipse plug-ins, features, and update sites. The community owes a debt of gratitude to Nick Boldt for all the hard work he did to implement Athena, and his tireless efforts help projects to adopt it.

Around the same time that Athena came into being, we brought Hudson into the picture to provide proper build orchestration.

Builds at Eclipse continued to evolve. The Buckminster project, which focuses on assembling artefacts, waded into the build space. The B3 project, a third-generation build based on Modeling technology, was created. At one point, a new project at Eclipse had lots of different choices for build.

Then the push to Maven started. For years many vocal members of the community complained that Maven couldn’t be used to build Eclipse bundles. It was Eclipse’s fault. It was Maven’s fault. There were a many false starts. But everything changed with Tycho. Tycho makes Maven understand how to build Eclipse plug-ins. Tycho facilitates a couple of useful things: first, it allows you to do “manifest-first” builds in which your Maven pom leverages the OSGi dependencies specified by an Eclipse bundle; second, it enables Maven to resolve dependencies found in p2 repositories. It does more than this, but these are the big ones.

Unfortunately, we haven’t found a good way to track the rate of migration. But in my estimation, Eclipse projects and many others in the adopter community are flocking to it.

The combination of Hudson, Maven, and Tycho seems to be delivering the holy grail. I managed to get up and running on Maven/Tycho in a matter of minutes and haven’t thought about the build since. For projects delivering a handful of features and bundles, it’s dirt-easy to get started and maintain the build. There are a still few rather large corner cases that need to be addressed. For example, we have a team working on moving the Eclipse project’s build over to the CBI. The Eclipse project’s build is about as hard as it gets.

The CBI has evolved and will continue to evolve. Our noble Webmaster recently added a new interface for signing build artefacts with the Eclipse certificate. We have ongoing work to develop a standard “parent pom” for Eclipse projects, and even a Maven repository where Eclipse projects to push their build results for dissemination to the general public.

So the CBI at Eclipse seems to be stabilizing around these technologies. But, I have no doubt that it will continue to evolve, especially as more and more projects start to consider implementing continuous build strategies combining Gerrit and Hudson.

This entry was posted in Project Management. Bookmark the permalink.

6 Responses to The Eclipse Common Build Infrastructure

  1. vogella says:

    I hope that CBI will allow local builds for the Eclipse platform. I believe that will attract more contributors.

  2. milestparker says:

    Thanks for the great update, Wayne. I wanted to join you in acknowledging Nick Boldt and the rest of the CBI folks for their tireless work on Athena. Even if it was basically supplanted, I think the time spent there wasn’t entirely wasted. If nothing else helped us to identify gaps and begin to get a handle on all of the artifact and process issues.

    I must admit that I’m more than a little disappointed that buckminster hasn’t had more pickup. A lot of the modelling tools adopted it including XText, XPand, AMP, etc.. in the AMP projects case, which has particularly nasty dependencies — there just aren’t the resources to migrate to yet another build structure. It certainly had its issues as well, but in many ways is a much richer and more powerful environment. Perhaps Buckminster can be used to create the Maven artifacts. (Yikes.)

    I must admit that I’m still not so sure about Maven; it has some ugly artifacts, overly verbose artifacts and I think the Eclipse tooling needs quite a bit more love. I’m particularly concerned that over time the Maven way of doing things will begin to pollute Eclipse plugin structure and tooling and that the xml ugliness will grow. But all the approaches had a fair shot, and it looks like the community has voted with their feet. That’s the way it should work.

  3. holmes70 says:

    I also love Buckminster a lot and I’d like to point out that it’s really more than “just” a build system. We use it to materialize our developer workspaces, too, and I wouldn’d like to miss it anymore.

    • milestparker says:

      Yeah, we do exactly the same thing. It’s just send someone a link to the cspec and they get everything they need, TP w/ all plugin dependencies, workspace projects, other artifacts. Very cool stuff.

  4. thomas says:

    The original intent with Buckminster was to serve as an integration platform for other build techniques, including Maven. The collaboration with the Maven community got a bad start though, partly because of legal issues. The work on getting the maven jars approved by the EF commenced but it took forever. I think 3-4 years went by without much action from the people involved (no shadow over the Eclipse legal team here, they responded quickly and provided lots of feedback and help). Another reason was that some of the companies that we tried to cooperate with turned out to be mostly interested in pushing their own agenda and not the least interested in making Maven one of several possible technologies to choose from.

    Over time, we’ve had several committers form the Maven community in Buckminster. None of them provided a single line of code. Projects like m2e and Tycho was started without any questions asked on our forum. Our objective hence failed and instead we eventually became a Maven competitor. Needless to say, that’s not a good situation for us and it was the direct opposite to what we originally intended.

    That said, I wish everyone good luck with the choices made and I hope that the people behind Tycho and m2e are up to the task of supporting the Eclipse Foundation.

  5. Pingback: The Eclipse Common Build Infrastructure | All of Java

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s