Project Activity Metrics, Charts, and Stuff

Yesterday we were reminded that it is difficult to provide good activity metrics. Certainly Git commit statistics are a good means of indicating that there is activity, but different styles of development can leave very different impressions: a lot of very small commits leaves the impression that there is more activity than a smaller number of large commits. That is, when you just count commits, a one line contribution is equal to a 1,000 line contribution (skipping merge commits is a no-brainer).


I’ve tinkered with using diff data to quantify individual commits, but even that can be misleading (e.g. generated code; removing vast chunks of code is often a net positive contribution). It’s made even harder when you consider that real software developers can spend a lot of time to work out a solution to a problem that results in a relatively small contribution of code.

Code talks at Eclipse, so making an initial assessment of liveliness by looking at the Git commit record makes a lot of sense. But, when I assess a project for activity I usually start my investigation with commit metrics and dig in from there. Bugzilla, mailing list, and forum activity are good places to look for evidence of daily activity (the Dashboard is a good source for this information). I also look to see if the project is making regular releases, or generating milestone builds. Finally, I actually try to communicate directly with the project team. It’s amazingly easy to write a quick “how’s it going?” note if I’m concerned that I don’t see enough evidence of activity.

Our open source projects information site, the so-called Project Management Interface, includes some handy charts intended to provide quick insight into project Git commit liveliness. They can, however, be a bit difficult to understand if you are not familiar with our arcane project structure. So I’ve made a change.

The query behind the Git commit charts that we display for a project now includes data from all subprojects recursively. So, for example, the charts for the Tools top-level project include data from all Tools projects. Likewise, the the charts for JDT include the subprojects (Core, Debug, and UI). I quite like the change as it makes it much easier to get an understanding of both the real project activity and diversity of the committer base.

But even with this change, it’s difficult to fully assess activity based on just one project’s commit metrics. Many people in the community equate JDT with “Eclipse IDE” and while JDT is what makes Eclipse a Java IDE, it relies heavily on frameworks implemented by the Platform project. So, activity in the Platform is an indication of activity in the Java IDE. This is made more challenging when you add the projects that leverage the extensibility of the platform, e.g. Web Tools, Mylyn, Code Recommenders, m2e (Maven), Buildship (Gradle), Git (EGit/JGit). A collection of activity charts including all of these projects would be interesting…

We don’t have an activity problem in our Java IDE. But we do have an opportunity to do a better job of communicating what we do and how we do it.

If you’re curious about ongoing work in Java 9 support, check out the beta.

Posted in Community, Eclipse 101, Java | Leave a comment

Platform and Java development tools at the EclipseCon Europe Hackathon

We have a handful of committers from the Eclipse Platform and Java development tools (JDT) projects ready to help you fix bugs at the EclipseCon Europe 2015 Hackathon.


Dani Megert (Eclipse Platform and JDT), Noopur  Gupta (JDT UI committer), and Olivier Prouvost (Eclipse Platform UI) have all agreed to attend office hours during the Hackathon. Others have agreed in principle (we expect confirmation soon), so there will be great representation if you have bugs to report or want to contribute fixes for “IDE” features.

Hackathon office hours run during the conference sessions. We have two sessions on Tuesday (15:00-16:20, and 16:45-18:05) and three on Wednesday (10:30-11:50, 13:45-15:05, and 16:15-17:35). The Hackathon area will be open beyond these time blocks, so you can plan to make great contributions to Eclipse well into the evening.

We’ll have Mikael Barbero around to help with Common Build Infrastructure (CBI), Eike Stepper and Ed Merks to help with Oomph and all things Modeling, Denis Roy to answer all your questions about Eclipse Foundation IT infrastructure and webmastery-sorts-of-things, and Markus Tiede there to help you contribute to Jubula (or use Jubula to build GUI tests). Of course, Eclipse Foundation staff will be on hand to help with processy-sorts of things, contributor license agreements (CLA), and more.

EclipseCon Europe 2015

Posted in EclipseCon, Hackathon | Leave a comment

Common Build Infrastructure at the EclipseCon Europe 2015 Hackathon

The Common Build Infrastructure (CBI) is an Eclipse project that provides a collection of build technology, infrastructure, and practices for building Eclipse software. If you’re building Eclipse plug-ins, this is something that you need to investigate.

Hudson Orchestration

CBI leverages Maven and Tycho to build project artifacts from source, and Hudson to orchestrate builds. It provides scripts to invoke signing using the Eclipse Foundation certificate (required by all projects that participate in the simultaneous release, and strongly recommended for all projects in general).

CBI was created to help Eclipse projects build, but the technology stack is generally useful and can be leveraged à la carte. So it’s worth investigating for use in closed source/internal projects as well.

If you’re interested in CBI, you’re in luck. Build meister Mikaël Barbero will be hanging out at the EclipseCon Europe 2015 Hackathon. Mikaël’s skill set has considerable breadth: he can also help you with work related to the Eclipse Platform and numerous other Eclipse projects.

You may also want to attend Mikaël’s Empower DevOps with Hudson Instance Per Project (HIPP) session on Wednesday.

EclipseCon Europe 2015

Posted in Hackathon | Tagged , , , | Leave a comment

Create your Project Import Set up at the EclipseCon Europe Hackathon

Eclipse Mars has this great new feature that you may have missed: the Oomph Project Importer will configure your Eclipse workspace with everything you need to hit the ground running quickly.

Oomph Project Importer

You can open this by selecting the Projects into Workspace importer via the File > Import… menu. The importer will guide you through the process of cloning Git/Gerrit repositories, constructing build targets, pulling tasks into Mylyn, and adding required plug-ins. In short, it will build a complete ready-to-get-started-coding environment with minimal effort in a matter of minutes.

Creating and maintaining an Oomph set up is critical if you want to lower the entry barriers for your potential contributors. You can also create set ups for your internal projects: get new employees up and running quickly!

But how to do create a set up for your project?

Step one: register for EclipseCon Europe

Step two: attend Eike Stepper and Ed Merk’s, Oomph: Eclipse the Way You Want It tutorial on Tuesday morning.

Step three: drop by the Hackathon area during designated office hours and get help to build your set up directly from the masters themselves.

If you want to learn more about Oomph and project set ups, there’s lot of useful information in the Eclipsepedia wiki.

Posted in Hackathon, Mars, Screenshots | Tagged | Leave a comment

Candidate bugs for the EclipseCon Europe 2015 Hackathon

Alternate title: What I saw at the EclipseCon Europe Hackathon made my jaw drop!

I’m pretty excited about the hackathon we’re running at EclipseCon Europe.

Hacker's Keyboard

In past hackathons, we’ve let attendees pick the bug that they want to work on. We’re going to encourage this sort of thing this time around, but want also to provide some help for those potential contributors who want to help, but don’t already have an idea.

We need some candidate bug reports.

It occurred to me that we have many open bugs reports annotated with helpwanted, indicating that committers feel that the issue addressed by the report is valid, but that they have no plans to resolve it. Some of these reports, however, are relatively large problems that could day hours, days, and weeks to resolve.

We also have many bug reports annotated with bugday. This keyword was originally used to flag reports that were candidates for a Bug Day event, but has grown to represent issues that committers feel can be addressed by a new contributor with modest effort (e.g. one or two hours).

The bugday bugs seem like a reasonable subset of candidates for the hackathon. As of this moment, there are 54 open bug reports annotated with bugday. That number drops to 40 if you include helpwanted in the search (i.e. there are 40 helpwanted bugday bugs). Both of these numbers are a little large, but are a pretty good starting point. I’m thinking that a list of 20 would be ideal.

I’m hopeful that committers will step up and annotate more bugs. Should an embarrassment of riches result, we’ll have to sort out a reasonable means of whittling this list down to a manageable size (e.g. an age limit).

I’m curious to hear if anybody has thoughts on how we select good candidates.

The EclipseCon Europe 2015 Hackathon runs from noon on Tuesday, November 3rd until the end of the day on Wednesday, November 4th. We’re setting up office hours where you can meet and work with Eclipse committers (more on this later).

Posted in EclipseCon, Hackathon | Leave a comment

Introducing the EclipseCon Europe 2015 Hackathon

Alternate title: I Went to EclipseCon Europe and What I Saw There Blew My Mind…

EclipseCon has great tutorials and sessions. The speakers are second-to-none, and the networking opportunities are fantastic. if you’re looking for it, you’ll find software developers doing amazing things together. For EclipseCon Europe 2015, we’re going to formalize this a bit. At very least, we’re going to give it a nice home and a lot of love.

10553673244_84713e4157_z We’re extending our usual notion of a hackathon: instead of just having a single evening session, we’re going to host a hacking area for two full days during the conference. We’ll have a few things happening in this hacking area.

There will be multiple tables arranged in perfect order for pair programming. The tables will have wired connections so that participants don’t have to wrestle with the wifi. We’re going to set up a server with mirrors of our Git and Mars p2 repositories so you don’t have to make the long trip back to our servers in Ottawa to get started (once you’ve cloned, you can connect to the master repository to get updates and push patches).

We’ll have Eclipse Foundation staff and others there to help. We can help with things like creating Eclipse Foundation accounts, setting up Contributor License Agreements (CLA), configuring development environments, understanding the development and intellectual property (IP) processes, and more. We can even help you find a problem to solve and a partner to solve it with (if you need that sort of help). We’ll have a little presentation area set up where we’ll run short how to contribute sessions throughout the day.

You are most certainly welcome to work on your own problem. Or, you can select from our short list of issues that we feel can be solved with up to two hours of effort: pick something from the list and we’ll help you get started and sort through the development process. If you’re not quite ready to submit patches, you can help us test by downloading one of the packages, the installer, or a feature; we’ll help you create bug reports for issues that you find. Or we can help you just create a bug report for some aspect of Eclipse that’s been driving you batty.

We’re going to set up some office hours: specific times set aside when we’ll have subject matter experts on hand to help with specific sorts of problems. Use the office hours to connect with experts on topics such as building Oomph configurations, testing your plug-ins, or using the Common Build Infrastructure; and committers from specific Eclipse projects.

Eclipse committers are, of course, welcome to use the hacking area to collaborate with fellow committers that you don’t normally get to work with directly.

I think that you’ll find that we have a little something for everybody.

EclipseCon Europe 2015

Posted in EclipseCon, EDP, Hackathon | Leave a comment

Technology Behind the new Eclipse Project Handbook

I’ve been asked about the technology used in the new Eclipse Project Handbook.

Editing the Eclipse Project Handbook

The content itself is specified in Asciidoc format and rendered into HTML using Asciidoctor. I started off using Asciidoc for rendering, but changed after I ran into a couple of bugs and problematic licensing.

The content is represented as a collection of documents that are combined via master documents for each of the separate forges; I decided to do this to make the content more manageable and easier for contribution, and make it more easy to mix and match content. Forge-specific parameters are specified in the master document.

Many of the images are specified using Graphviz and rendered as hierarchical images using dot. For this style of graphic, the very simple text format permitted by Graphviz means that I have very good version control for these images. I also really hate pushing pixels around.

To render the documents as PDF, I’m using Asciidoctor to render the content in Docbook format and Apache FOP to take it the rest of the way. I think that my challenges with the graphics in the rendered PDFs are a issue with FOP, but haven’t quite concluded my investigation. As I mentioned yesterday,  I consider the PDF versions a nice-to-have, so I haven’t made sorting this out a very high priority.

I’m pulling all the pieces together using a make build. Given the nature of the project, this was, frankly, the easiest solution. It works well and is easy to maintain (there are, however, a few hard-coded file paths that I’m not at happy about).

The sources are all in the /projects Git repository if you’d like to take a closer look. I’m thinking that I will move these to a separate repository to make it easier for folks who want to contribute patches (so that you don’t need to clone all the other junk stuff in that repository).

Naturally, I’m using Eclipse to edit content and invoke the makefile as an external tool. Adding Mylyn Docs Wikitext support for Asciidoc to my Eclipse configuration is on my to-do list this month.

I’m pulling all of the tools from the Fedora software repository.

Posted in Other | 3 Comments