Eclipse TLP Contributors by Quarter

Alexander posted an interesting tweet the other day.

I decided to take a harder look at the data to see if he’s right. It turns out that he’s not. According to my data, the Eclipse Top Level Project (TLP) had the largest number of contributors, 120, in 2014Q1.

eclipse_contributors_quarterly

Still, 70 in 2018Q4 is pretty good. And the quarter isn’t even over yet. Those spikes in the chart notwithstanding, the healthy upward contribution trend is encouraging.

Note that my numbers are a little different than Alexander’s. His are based on the last three months worth of contributions as of the day that he parsed the chart data. My data is based strictly on traditional quarter boundaries (so his number includes some commits made in September that mine disregards).

Normally, I restrict the date range on these sorts of queries, but I decided to run against the full history. You’ll notice that it looks like we had no committers before 2005: we didn’t keep committer records in our database before the Eclipse Foundation was created in 2005.

These numbers represent only commits pushed to Git repositories owned by Projects that fall under the Eclipse Top Level Project, which only a fraction of the activity that occurs Eclipse Foundation open source projects.

Advertisements
Posted in Uncategorized | Leave a comment

Eclipse Development Process 2018

On December 7/2018, the Eclipse Foundation’s Board of Directors approved a new edition of the Eclipse Development Process (EDP) which introduces one particularly significant change, along with a small number of other useful changes. We rolled this new edition out on December 14/2018.

Note that project teams who are familiar with the previous edition of the EDP can just keep doing what they’ve always done: all changes in this new edition are backwards compatible.

The big change is the introduction of Progress Reviews and the recast of Release Reviews as a type of Progress Review. Progress Reviews are basically the same as Release Reviews with the exception that they can occur at any point in the project lifecycle (while the EDP doesn’t explicitly make any specific requirements, Release Reviews are generally accepted to be timed near the end of a release cycle). The basic idea is that a project team can opt to time a Progress Review at their convenience.

Progress Reviews are not in and of themselves interesting. What is interesting is that a project team can declare as many official releases as they’d like for an entire year following a successful Progress Review or Release Review. This should be especially interesting and valuable to projects that need to make very frequent releases.

lifecycle

The new release process looks a little something like this.

This change in the nature of how we do releases, required that the statement that “the IP Policy has been followed and all approvals have been received” be moved from being a requirement of a Release Review to being a requirement of a release. That is, the change in the nature of releases does not change the requirement the the Eclipse IP Policy be followed and that projects engage in the full Eclipse IP Due Diligence Process.

Intellectual property must be properly accounted for and tracked at all times. The project team must engage in the Eclipse IP Due Diligence Process on an ongoing basis. The IP Log review and approval that occurs at the time of either a Release Review or Progress Review should be regarded as a means of confirming that intellectual property is being properly managed and not as a trigger to engage in a last minute clean up.

Only intellectual property that has been been approved (or license certified) by the Eclipse IP Team may be included in a release.

In addition to the IP Policy, project teams are expected to continue to engage in the usual sorts of community building activities like open and transparent release planning.

The new edition also includes the following changes:

  • A long-missing Terms and Definitions section has been added;
  • The Releases section has been restructured and is (hopefully) more easily parsed;
  • The content is captured in Asciidoc and rendered using Asciidoctor; and
  • All anchors in the document are self-referential links (which make it easier to get links into the document).

A more comprehensive list of the changes is captured in the change log.

I’ll push out updates to the Eclipse Project Handbook that support these changes later this week.

Posted in Eclipse 101, EDP | Leave a comment

Eclipse Foundation Specification Process, Part III: Creation

The Eclipse Foundation Specification Process (EFSP) includes an image that provides an overview of what goes into creating a new Specification Project. By creating, we mean the process of taking a Specification from an initial idea or concept through to the point where the necessary resources and permissions are in place to do real development. It’s the same basic process that we follow when creating a regular Eclipse Project with an extra Specifications-specific approval added in.

creation

The first couple of steps of the project creation process usually happen via private email exchanges between the people involved with the project and the Eclipse Management Organization (EMO) (this is one of the few things that we do in private at the Eclipse Foundation). With this communication, we take the idea behind a Specification Project and create a Proposal that describes and defines it. The Proposal includes important information like a statement of Scope, a description, and the list of the initial Specification Team (Project Leads, Committers, and Mentors). Other useful information is contained the Proposal, including background information, descriptions of content that already exists, and initial plans.

In the steady state (i.e. after the Specification Project has been created), all roles are assigned via demonstrations of merit and votes from the existing Committers. Since we have no existing Committers before the project is created, the creation process serves as the display of merit for the community and vote.

The EMO works with the proposers to whip the Proposal into shape. When the EMO decides that the content is complete, it is delivered to the Executive Director of the Eclipse Foundation (EMO(ED)) for approval to post for community review.

Once we have EMO(ED) approval and the Proposal is posted for the community to review, it is said—according to the Eclipse Development Process (EDP)—to have entered the Proposal Phase. We often refer to this as the community review period. During this period, other developers may step up and volunteer to become Committers, community members may ask questions, the Scope and other aspects of the Proposal may be tweaked, and more. Crucially, during this period, Member Companies of the Eclipse Foundation have an opportunity to express concerns and even (possibly) reject the Project Proposal (this has never actually happened). A Specification Project must stay in the Proposal Phase for a minimum of two weeks to give the various stakeholders enough time to review and respond.

The EMO uses the Proposal Phase to ensure that we can reasonably assert ownership of the Specification Project’s name as a trademark on behalf of the community (or at least that the project name does not infringe on somebody else’s trademark). If a Specification Project’s name is already used by the Specification Team (or an organization that’s involved with the Proposal), then EMO will work with them to transfer ownership of the trademark and associated Internet domains to the Eclipse Foundation. In parallel, the EMO will also work with the Eclipse Architecture Council to identify a Mentor, and seek approval from the Project Management Committee (PMC) of the target Top-level Project. It’s not shown on the image, but we’d use this time to inform the corresponding Specification Committee that the new Specification Project is coming.

At the Eclipse Foundation, we organize all Projects and Specification Projects hierarchically. Top-level Projects, which sit at the the top level of the hierarchy, are generally not used for actual development work. It is in the Projects (also referred to as Subprojects) under the Top-level Projects where the real work happens. All Top-level Projects have a PMC who—as part of the Project Leadership Chain for all projects that fall under the Top-level Project—are responsible for ensuring that the Specification Teams work as good open source projects (as defined by the EDP).

When all of these separate threads are resolved, we move to the Creation Review. During this period, no further changes are allowed and the Proposal is locked down. The EMO uses this time to ensure that the process was followed. The Creation Review lasts for a minimum of one week. We schedule all reviews—Creation Reviews included—based on the date that they conclude; reviews are scheduled to conclude on the first and third Wednesday of every month (we may schedule additional reviews in exceptional cases).

To successfully complete a Creation Review for a Specification Project, the EFSP requires that we have Super-majority approval of the Specification Committee (Super-majority is defined as two thirds of the Specification Committee members). The EFSP is silent regarding the specifics of how such a vote is executed, but is generally accepted that votes run for a minimum of a week (a Specification Committee may decide that more or less time is required). The factors that the Specification Committee must consider when casting their votes are also not specified. In general though, the Specification Committee will be looking at whether or not new Specification Project Proposals are viable and are a good fit with the corresponding Working Group.

Upon successful conclusion of the Creation Review, the Proposal is sent to the Eclipse Webmaster Team for provisioning. The provisioning process starts with Committer Paperwork: once we have complete paperwork for one Committer, the Webmaster will create resources including websites, Git repositories, Issue trackers, etc. for the Specification Team.

With provisioned Committers and resources, the Specification Project is said to be in the Incubation Phase (or “in incubation”); it will stay in this phase until the Specification Team engages in a Graduation Review (which is generally combined with the project’s first or second Release and corresponding Release Review). The first action of the Specification Team with their new Specification Project is to submit their Initial Contribution (when a Specification comes to the Eclipse Foundation with existing intellectual property) to the Eclipse IP Team for review; with their approval of the Initial Contribution, that content can be added to the project’s Git repository and the Specification Team can start to weave their magic.

For more background, please see Part I, and Part II of this series.

Posted in Eclipse 101, EDP, EFSP | Leave a comment

Eclipse Foundation Specification Process, Part II: the EFSP

The Eclipse Foundation Specification Process defines a general framework for developing specifications in open source at the Eclipse Foundation; it extends the Eclipse Development Process (EDP) by adding a few extra checks and balances. In the first installment of this series, we described the EDP; in this second installment, we introduce the Eclipse Foundation Specification Process (EFSP) with a focus on how it extends the EDP. We’ll provide more detail and tackle why it’s implemented as it is in future installments.

Note that we’re using the same form of pseudo-legal style capitalization of defined terms that we use in the EDP, EFSP, and other policy documents in this post.

Like regular Eclipse open source projects, a Specification Project starts life as a Proposal with a description, scope, list of committers, and more; goes through an iterative development cycle that produces one or more Milestone builds; and then engages in a release process.

efsp_lifecycle

Unlike a regular Project, a Specification Project must be aligned with exactly one Eclipse Foundation Working Group. The Working Group designates a Specification Committee that maintains and manages the specification process on the Working Group’s behalf (when we talk about the Specification Committee in the context of a Specification Project, we mean the Specification Committee of the Working Group with which the Specification Project is aligned).

A Specification Project must get approval from the corresponding Specification Committee to pass key project lifecycle events. In this regard, the role of the Specification Committee is very similar to the governance role of the Java Community Process’ (JCP) Executive Committee.

It’s worth noting that the EFSP provides the general framework for doing specifications in open source at the Eclipse Foundation and that we expect multiple Working Groups to leverage it for that purpose. We also fully expect that a Specification Committee will augment the process as it’s defined. We expect, for example, that we will have a Jakarta EE Specification Process that uses the EFSP as its foundation.

The Specification Committee needs to approve of the creation of a Specification Project from a Proposal by taking a role in the Creation Review. The expectation is that the Specification Committee members will consider the details of the proposed Specification Project (with particular focus on the Scope) before making their decision. In addition to the requirements defined by the EDP, a Super-majority affirmative vote of the entire Specification Committee is required to approve a Creation Review.

Following successful creation and provisioning of project resources, the Specification Project begins development. During the development cycle, the project team must produce at least one Milestone build of the specification’s content (documentation and technical artifacts) to solicit feedback, and at least one of the Milestone builds must serve as a trigger to engage in a Progress Review.

Progress Reviews, a new addition to the EDP (introduced with the 2018 version), are roughly equivalent to Release Reviews, but with the intent of ensuring that the Specification Project is progressing in a manner that will ultimately result in a successful release. A Specification Committee and Project Leadership may compel a Specification Project to engage in additional Progress Reviews.

For a Progress Review, the Project Management Committee (PMC), and Eclipse Management Organization (EMO) validate that the Project Team is following the EDP and EFSP, and that the Eclipse Foundation’s Intellectual Property Policy is being correctly implemented. The EFSP further requires that the Specification Committee approve a Progress Review by Super-majority vote.

At the end of every release cycle, the project team must produce a Release Candidate build that we label as a Specification Version and then engages in a Release Review. For a Release Review, the PMC, EMO, and Specification Committee all engage in the same sorts of activities that they do for a Progress Review, but with the understanding that approval results in the ratification of the specification and promotion to an official status. The EFSP requires that the Specification Committee approve a Release Review by Super-majority vote.

Following a successful Release Review, the final release version of the Specification Artifacts are considered Ratified and morph into what the process refers to as a Final Specification. It is the Final Specification that must be used to build Compatible Implementations.

Following a successful first release (and every subsequent release thereafter), and before engaging in any further development of the specification, the project team must assemble and present their Plan for review by the Specification Committee via Plan Review. The notion of a Plan Review is specific to the EFSP (since Plan Reviews are not part of the EDP, no formal involvement from the PMC is required). A Plan Review provides the Specification Committee with an opportunity to ensure that the plan for the next Specification Version is in-scope, fits within the overall vision of the Working Group, and is otherwise charting a path to eventual ratification and release. The EFSP requires that the Specification Committee approve a Plan Review by Super-majority vote.

After the Plan is approved, the Project Team engages in Development as before.

We’ll discuss the relationship between Specification Projects, Working Groups, and Specification Committees; what happens when the Specification Committee does not approve a vote; the relationship between a Specification Version and Final Specification; and more in future posts.

Mike and I delivered a talk about this at EclipseCon Europe 2018, titled “Introducing The Eclipse Foundation Specification Process” (or go directly to the video on our YouTube Channel).

Posted in Eclipse 101, EDP, EFSP | 1 Comment

Eclipse Foundation Specification Process, Part I: The EDP

The Eclipse Foundation Specification Process (EFSP) was authored as an extension to the Eclipse Development Process (EDP). With this in mind, before we can discuss the EFSP, we’ll start with a quick EDP primer.

At a high (and very simplified) level, the EDP looks a little something like this:

edp_lifecycle

 

All open source projects at the Eclipse Foundation start life as a proposal. A proposal literally proposes the creation of a new open source project: the proposal document suggests a name for the new project, and defines many things, including a description and scope of work. The proposal also serves as the nomination and election of all project committers and project leads.

The proposal is posted for community feedback for a minimum of two weeks; during that time, the Eclipse Foundation staff works behind the scenes to ensure that the project’s name can be claimed as a trademark, a mentor has been identified, the licensing scheme works, and more. The community feedback period ends with a creation review which lasts for a minimum of one week. The creation review is the last opportunity for the community and the members of the Eclipse Foundation to provide feedback and express concerns regarding the project.

After successful completion of the creation review, and the project resources have been provisioned by the Eclipse Webmaster team, the project team engages in development. Project committers push code to into the project’s source code repositories, and produce and disseminate milestone (snapshot) builds to solicit feedback as part of an iterative development process.

When the time comes to deliver a formal release, the project team produces release candidates and engages in a release review. A release review provides an opportunity for the project team to demonstrate to their Project Management Committee (PMC) that their content is ready for release, work with the Eclipse Intellectual Property Team to ensure that all of the required IP due diligence has been completed successfully, and give the community and membership a final opportunity provide feedback and express concerns. Following a successful release review, the project team will push out their final (GA) build and announce the official release to their community via established channels.

The proposal serves as the first plan for the new open source project. Subsequent releases start with the creation of some sort of plan before reengaging in the development (release) cycle. The level of formality in the planning process varies by project. For many projects, the plan is little more than an acknowledgement that further development is needed. But for some projects, planning is a well-defined open process by which the committers work with their communities to identify themes and issues that will be addressed by the release.

In my next post, I’ll discuss how this process is extended by the the EFSP. Then, I’ll start digging into the details.

You can find the community draft of the Eclipse Foundation Specification Process here.

Posted in EDP, EFSP | 2 Comments

Eclipse IP Process: Service Releases of Third Party Content

The primary role of the Eclipse IP Team is to reduce the risks associated with adopting open source software. In broad terms, they ensure that the licenses on content are compatible, that provenance is clear, and that content otherwise unencumbered from a legal point-of-view (strictly speaking, the team does all of this only for Type B requests). In other words, they do the sorts of things that every software project really needs to do (especially those projects that care about wide scale adoption), but software developers hate doing.

It’s impossible to remove all risk. The IP Due Diligence process is all about risk mitigation.

Project committers do play an important role in this work. The Eclipse IP Team does the heavy investigative work, but it is the committers who must bring intellectual property matters to the IP Team for their review. This takes the form of creating a contribution questionnaire (CQ) and then providing assistance where necessary to our analyst to investigate, and identify and resolve issues.

Experience has demonstrated that service releases of third party content are very low risk. By their nature, service releases include bug fixes only, and so don’t tend to include a lot of new intellectual property. Our experience is that bug fix releases generally change or add a few lines of code here and there.

Based on this experience, the Eclipse IP Due Diligence Process gives service releases of third party content a pass: project committers do not need to create a CQ or otherwise engage with the Eclipse IP Due Diligence Process for any service release of third party content that has already been approved.

That is, if a version of some bit of third party content has been approved by the IP Team, then service releases based on that approved version do not require any review. Just drop ’em into your build and have at it (e.g. if version 3.2 has been approved for use, a project can just use version 3.2.n without formal review).

Of course, if you suspect shenanigans or otherwise lack confidence in the status of the content, you can bring the service release to the IP Team in the usual manner. In fact, if you do suspect that maybe something labeled as a service release isn’t actually service release, please do engage the IP Team.

This and many other topics are covered by the Eclipse Project Handbook.

I’m at EclipseCon Europe. If I’m not in a session, I’ll be in the registration area. Ask me questions!

ece_2018_logo

Posted in Eclipse 101, EDP, Intellectual Property | Leave a comment

Jakarta EE @EclipseFdn: Getting it done

To say that bringing Jakarta EE to life at the Eclipse Foundation was a massive undertaking is a gross understatement.

We started this effort in early 2018. Today, we have 39 open source projects (as part of the EE4J Top Level Project) representing 14% of our overall project space. These projects collectively have 99 Git repositories (most of which are hosted in the eclipse-ee4j GitHub organization; a small number are hosted on Eclipse Foundation infrastructure) representing 8% of the repositories created and managed by the Eclipse Foundation’s IT team. We have 162 committers working on these projects, representing 10% of our total active committer community. These projects have made a combined 440 requests to the Eclipse Foundation’s IP Team to review intellectual property, representing 17% of the work done by the team. The Jakarta EE projects have 53 distinct CI instances running (or in the planning stages to run) on Eclipse Foundation build infrastructure, representing 30% of our work there (on the topic of build, if you have some spare cycles and want to contribute, we can use your help).

Added to the all of the project-related work, we’ve also been doing a lot of other work to  stand up the new Jakarta EE Working Group, bootstrap the new EE4J PMC, manage related trademarks and various legal agreements, recruit participants, build websites, improve infrastructure, hire and train new staff, and more.

By way of comparison, the work that we do with Eclipse IoT is at about the same scale, but grew more organically (one open source project at a time) over six years. Jakarta EE happened all at once.

While we engaged in this enormous chunk of work, we still did everything that we’ve always done. In just the first nine months of 2018, for example, the Eclipse Intellectual Property Team has already received and processed 30% more requests for reviews than they did in all of 2017! And the number of new project proposals, committers, and contributions (not including those that are related to Jakarta EE) that we’ve added in 2018 are on an upward trend compared with past years.

This has been an all-hands-on-deck sort of effort for the Eclipse Foundation staff. Even those of us who have not been directly involved in the work, have helped by taking on extra burdens so that those of use who were directly involved could stay focused.

This is not to say that we’ve done this alone: this has been a ginormous shared effort. We’ve had a lot of help from the Eclipse Foundation member companies that have contributed many talented people to the migration effort, and—of course—the members of our community who have stepped up.

I really want to name names, but the probability of me missing somebody is so high that I dare not. If you want to meet some of the key players, join us at EclipseCon Europe!

Home

Posted in Eclipse 101, Intellectual Property, Jakarta EE | Leave a comment