Eclipse Foundation Specification Process, Part I: The EDP

The Eclipse Foundation Specification Process (EFSP) was authored as an extension to the Eclipse Development Process (EDP). With this in mind, before we can discuss the EFSP, we’ll start with a quick EDP primer.

At a high (and very simplified) level, the EDP looks a little something like this:

edp_lifecycle

 

All open source projects at the Eclipse Foundation start life as a proposal. A proposal literally proposes the creation of a new open source project: the proposal document suggests a name for the new project, and defines many things, including a description and scope of work. The proposal also serves as the nomination and election of all project committers and project leads.

The proposal is posted for community feedback for a minimum of two weeks; during that time, the Eclipse Foundation staff works behind the scenes to ensure that the project’s name can be claimed as a trademark, a mentor has been identified, the licensing scheme works, and more. The community feedback period ends with a creation review which lasts for a minimum of one week. The creation review is the last opportunity for the community and the members of the Eclipse Foundation to provide feedback and express concerns regarding the project.

After successful completion of the creation review, and the project resources have been provisioned by the Eclipse Webmaster team, the project team engages in development. Project committers push code to into the project’s source code repositories, and produce and disseminate milestone (snapshot) builds to solicit feedback as part of an iterative development process.

When the time comes to deliver a formal release, the project team produces release candidates and engages in a release review. A release review provides an opportunity for the project team to demonstrate to their Project Management Committee (PMC) that their content is ready for release, work with the Eclipse Intellectual Property Team to ensure that all of the required IP due diligence has been completed successfully, and give the community and membership a final opportunity provide feedback and express concerns. Following a successful release review, the project team will push out their final (GA) build and announce the official release to their community via established channels.

The proposal serves as the first plan for the new open source project. Subsequent releases start with the creation of some sort of plan before reengaging in the development (release) cycle. The level of formality in the planning process varies by project. For many projects, the plan is little more than an acknowledgement that further development is needed. But for some projects, planning is a well-defined open process by which the committers work with their communities to identify themes and issues that will be addressed by the release.

In my next post, I’ll discuss how this process is extended by the the EFSP. Then, I’ll start digging into the details.

You can find the community draft of the Eclipse Foundation Specification Process here.

Advertisements
Posted in EDP, EFSP | Leave a comment

Eclipse IP Process: Service Releases of Third Party Content

The primary role of the Eclipse IP Team is to reduce the risks associated with adopting open source software. In broad terms, they ensure that the licenses on content are compatible, that provenance is clear, and that content otherwise unencumbered from a legal point-of-view (strictly speaking, the team does all of this only for Type B requests). In other words, they do the sorts of things that every software project really needs to do (especially those projects that care about wide scale adoption), but software developers hate doing.

It’s impossible to remove all risk. The IP Due Diligence process is all about risk mitigation.

Project committers do play an important role in this work. The Eclipse IP Team does the heavy investigative work, but it is the committers who must bring intellectual property matters to the IP Team for their review. This takes the form of creating a contribution questionnaire (CQ) and then providing assistance where necessary to our analyst to investigate, and identify and resolve issues.

Experience has demonstrated that service releases of third party content are very low risk. By their nature, service releases include bug fixes only, and so don’t tend to include a lot of new intellectual property. Our experience is that bug fix releases generally change or add a few lines of code here and there.

Based on this experience, the Eclipse IP Due Diligence Process gives service releases of third party content a pass: project committers do not need to create a CQ or otherwise engage with the Eclipse IP Due Diligence Process for any service release of third party content that has already been approved.

That is, if a version of some bit of third party content has been approved by the IP Team, then service releases based on that approved version do not require any review. Just drop ’em into your build and have at it (e.g. if version 3.2 has been approved for use, a project can just use version 3.2.n without formal review).

Of course, if you suspect shenanigans or otherwise lack confidence in the status of the content, you can bring the service release to the IP Team in the usual manner. In fact, if you do suspect that maybe something labeled as a service release isn’t actually service release, please do engage the IP Team.

This and many other topics are covered by the Eclipse Project Handbook.

I’m at EclipseCon Europe. If I’m not in a session, I’ll be in the registration area. Ask me questions!

ece_2018_logo

Posted in Eclipse 101, EDP, Intellectual Property | Leave a comment

Jakarta EE @EclipseFdn: Getting it done

To say that bringing Jakarta EE to life at the Eclipse Foundation was a massive undertaking is a gross understatement.

We started this effort in early 2018. Today, we have 39 open source projects (as part of the EE4J Top Level Project) representing 14% of our overall project space. These projects collectively have 99 Git repositories (most of which are hosted in the eclipse-ee4j GitHub organization; a small number are hosted on Eclipse Foundation infrastructure) representing 8% of the repositories created and managed by the Eclipse Foundation’s IT team. We have 162 committers working on these projects, representing 10% of our total active committer community. These projects have made a combined 440 requests to the Eclipse Foundation’s IP Team to review intellectual property, representing 17% of the work done by the team. The Jakarta EE projects have 53 distinct CI instances running (or in the planning stages to run) on Eclipse Foundation build infrastructure, representing 30% of our work there (on the topic of build, if you have some spare cycles and want to contribute, we can use your help).

Added to the all of the project-related work, we’ve also been doing a lot of other work to  stand up the new Jakarta EE Working Group, bootstrap the new EE4J PMC, manage related trademarks and various legal agreements, recruit participants, build websites, improve infrastructure, hire and train new staff, and more.

By way of comparison, the work that we do with Eclipse IoT is at about the same scale, but grew more organically (one open source project at a time) over six years. Jakarta EE happened all at once.

While we engaged in this enormous chunk of work, we still did everything that we’ve always done. In just the first nine months of 2018, for example, the Eclipse Intellectual Property Team has already received and processed 30% more requests for reviews than they did in all of 2017! And the number of new project proposals, committers, and contributions (not including those that are related to Jakarta EE) that we’ve added in 2018 are on an upward trend compared with past years.

This has been an all-hands-on-deck sort of effort for the Eclipse Foundation staff. Even those of us who have not been directly involved in the work, have helped by taking on extra burdens so that those of use who were directly involved could stay focused.

This is not to say that we’ve done this alone: this has been a ginormous shared effort. We’ve had a lot of help from the Eclipse Foundation member companies that have contributed many talented people to the migration effort, and—of course—the members of our community who have stepped up.

I really want to name names, but the probability of me missing somebody is so high that I dare not. If you want to meet some of the key players, join us at EclipseCon Europe!

Home

Posted in Eclipse 101, Intellectual Property, Jakarta EE | Leave a comment

Lightening the Release Review Burden

The Eclipse Architecture Council is in the process of making a change to how the Eclipse Development Process (EDP) defines the Reviews that Eclipse open source projects are required to engage in. Foremost on our minds is the nature of Release Reviews which the EDP current requires ahead of all major and minor releases (service releases are excused from the requirement).

The current thinking is that we will decouple Releases from Reviews and instead require a regular (probably annual) equivalent to the Release Review.

This is actually not as big a departure from the current process as you might think. The EDP describes the Release Review as an opportunity “to summarize the accomplishments of the release, to verify that the IP Policy has been followed and all approvals have been received, to highlight any remaining quality and/or architectural issues, and to verify that the project is continuing to operate according to the principles and purposes of Eclipse.” That is, it is far less about the current release than it is about ensuring that the process is being followed and that the project is doing the right sorts of things to attract and grow community.

Likewise, the purpose of an IP Log is not to accurately represent the contents of any particular release, but rather to provide a checkpoint to ensure that the project is correctly following the Eclipse IP Due Diligence Process (so project teams can continue to receive contributions or engage in other activities that might change the IP Log between the point in time when it is reviewed and approved, and the project makes a release).

Eclipse committers are required to observe the Eclipse IP Policy and IP Due Diligence Process at all times, and so our open source projects must always be in correct state with regard to intellectual property management.

For those of you who have made it this far, it would be great if you could weigh in on Bug 534828 which includes an effort to more precisely define “Release”. We’re tracking all of our plans to update the EDP via Bug 484593. Input is welcome.

 

Posted in Community, Eclipse 101, Project Management | Leave a comment

The Eclipse Committer Election Workflow

In the world of open source, Committers are ones who hold they keys. Committers decide what code goes into the code base, they decide how a project builds, and they ultimately decide what gets delivered to the adopter community. With awesome power, comes awesome responsibility, and so it’s no mistake that the Open Source Rules of Engagement described by the Eclipse Development Process, puts Meritocracy on equal footing with Transparency and Openness: becoming a committer isn’t necessarily hard, but it does require a demonstration of commitment (committer… commitment… see what I did there?)

There’s two ways to become an Eclipse Committer. The first way is to be listed as an initial committer on a new project proposal. When projects come to the Eclipse Foundation we need them to actually start with committers, and so we include this as part of the bootstrapping. As part of the process of community vetting a new project proposal, the committers listed are themselves vetted by the community. That’s why we include space for a merit statement for every committer listed on a proposal (in many cases, the merit statement is an implied “these are the people who worked on the code that is being contributed”). In effect, the project proposal process also acts as a committer election that’s open to the entire community.

The second way to become a committer is to get voted in via Committer Election. This starts with a nomination by an existing committer that includes a statement of merit that usually takes the form of a list various contributions that the individual has made to the project. What constitutes a sufficient demonstration of merit varies by project team and PMC. Generally, though, after an individual has made a small number of high quality contributions that demonstrate that they understand how the project works, it’s pretty natural for them to be invited to join the team.

There’s actually a third way. In cases where a project is dysfunctional, the project leadership has an option to add and remove committers and project leads. In the rare cases where this option is exercised, it is first discussed in the corresponding Project Management Committee‘s (PMC) mailing list.

Last week, we rolled out some new infrastructure to support Committer Elections.

Every project page in the Project Management Infrastructure (PMI) includes a block of Committer Tools on the right side of the page. From this block, project committers can perform various actions, including the new Nominate a Committer action.

Screenshot from 2018-02-06 10-34-43

Committer Tools

Clicking this will bring up the nomination form where the existing committer will provide the name and email address of the nominee along with the statement of merit.

Screenshot from 2018-02-06 10-35-15

What the committer sees when they nominate a new committer.

When you click the Nominate button, the Committer Election begins by sending a note to the project mailing list inviting existing project committers to vote. Committers visit the election page to cast their vote and—since this is a transparent process—everybody else can watch the election unfold.

According to our election rules, an election ends when either everybody votes in the affirmative or seven days has passed. If at the end of the election we have at least three affirmative votes and no negative votes, the vote is considered successful and it passed on to the PMC for approval (note that when a project has fewer than three committers, success is declared if everybody votes in the affirmative). The PMC will validate that the merit statement is sufficient and that the election was executed correctly, and either approve or veto it. PMC-approved elections get passed into the next piece of the workflow: Committer Paperwork.

Regardless of how a developer becomes a committer (by vote, by proposal, or by appointment), they are required to complete legal paperwork before we can grant them write access to project resources. The Eclipse Foundation needs to ensure that all committers with write access to the code, websites, and issue tracking systems understand their role in the intellectual property process; and that we have accurate records of the people who are acting as change agents on the projects. Committers must provide documentation asserting that they have read, understood, and will follow the committer guidelines; and must gain their employers consent to their participation in Eclipse Foundation open source projects.

Our Commmitter Paperwork process is initiated whenever a developer joins us as a new committer, or—since paperwork is tied to a specific employer—when a committer changes employers.

Screen Shot 2018-02-07 at 11.54.35 AM

The exact nature of the type of paperwork required varies based on the individual’s employment status and the Eclipse Foundation membership status of their employer. Again, a full discussion of this is out-of-scope for this post, but we need to have either an Individual Committer Agreement or a Member Committer Agreement on file for every committer. The workflow guides the new committer through the options.

Note that we’ve just gotten approval on an update to the Individual Committer Agreement that eliminates the need for the companion Eclipse Foundation Committer Employer Consent Form. This should make it easier for new committers to get started. We’re rolling the new version out now.

We owe this great new implementation of this workflow to the tireless efforts of the entire Eclipse IT Team, and especially Eric, Chris, and Matt. Big Thanks!

Posted in Eclipse 101, EDP, Intellectual Property, Project Management | Leave a comment

Decoding the Eclipse IP Policy: Third Party Dependencies

The term Intellectual Property (IP) refers to any sort of creative work, be it literature, art, or software. In the realm of open source software, artifacts like source code, documentation, and images are considered intellectual property. Unless otherwise stated, intellectual property is the property of its creator, who may grant permission for others to use that intellectual property by providing a license.

The ease with which software can be copied and combined makes it challenging to know with confidence if content can be used without running into legal issues. Any sort of serious software development effort must be accompanied by a well-defined IP Due Diligence process that can ferret out issues and mitigate the risk of leveraging the work of others. IP Due Diligence is a time consuming process that requires specialized skills and a keen eye for detail.

There are different kinds of content (source code, documentation, images, …) to consider. Project Code is content that is produced and maintained by the open source project committers and contributors. Third Party Content generally takes the form of libraries (modules, components, …), source files, images, or other forms of IP that are produced and maintained outside of the scope of the open source project. To mitigate the risk associated with adopting open source in products, the Project Code and Third Party Content that it leverages need to be vetted to ensure that the copyrights expressed are correct, licensing is valid and compatible, and other issues have been uncovered and properly investigated.

The Eclipse Foundation has a well-defined IP Policy, corresponding IP Due Diligence Process, and a dedicated team of professional IP specialists who perform the heavy lifting in the due diligence process. Committers, the software developers who decide what will become Project Code and how an Eclipse open source project will leverage Third Party Content, are responsible for bringing IP issues to the attention of the Eclipse IP Team.

Most of the Project Code produced by committers can just be pushed into a project repository without any sort of legal review. However, at least in some cases, the IP Team needs to be engaged to review Project Code that comes from contributors (who are not committers); and there are some cases where even the work of committers needs to be reviewed. I’ll discuss Project Code with more detail in a future post.

The sort of effort that the Eclipse IP Team brings to bear on Third Party Content varies depending on the type. The Guidelines for the Review of Third Party Dependencies defines three different types: Prerequisite, Exempt Prerequisite, and Works With Dependency.

The simplest form of Third Party Content is Prerequisite. Prerequisites are required by the Eclipse project content to provide core functionality. Prerequisite content is not generally stored in an Eclipse project’s source code repositories, but is likely included in build scripts and referenced as runtime dependencies. Since adopters of Eclipse project content are compelled to adopt the Prerequisite content, that content must also be vetted by the IP Team. The vetting requirement applies recursively: the entire transitive closure of a Prerequisite’s dependencies needs to reviewed (the dependencies of a Prerequisite are themselves Prerequisites).

prereq

The transitive closure requirement only applies when an Eclipse project makes a direct reference to Third Party Content (the Eclipse Project Handbook provides some examples of what constitutes a direct reference). In the case where an Eclipse project references code from a second Eclipse project that itself references Prerequisites, no further vetting of that chain of Prerequisite content is required (the IP Team will have already vetted it on behalf of the second project team). Eclipse project teams should take care to only reference release versions of other Eclipse projects in their own releases to ensure that the IP process has been completed.

project

When you follow this all the way to the bottom, the entire runtime environment including virtual machines and the operating system are included in the transitive closure of dependencies. Clearly, having the IP team vet virtual machines and operating systems is not a great use of time, and—in the case of closed source operating systems—just not be possible.

The Eclipse IP process guidelines provide for a notion of Exempt Prerequisite dependencies, which are not subject to review. According to the guide, content may be considered exempt if it “is pervasive in nature, expected to be already on the user’s machine, and/or an IP review would be either impossible, impractical, or inadvisable.” The Eclipse IP Team does not review the source code associated with an Exempt Prerequisite.

One of the key aspects of an Exempt Prerequisite is that the user or adopter is typically the one that actually installs the software and so is the one who must agree to the licensing terms. Content that is declared an Exempt Prerequisite should never be directly distributed by an Eclipse project or otherwise made available without some explicit action from the consumer. Exempt Prerequisites must be approved by the Eclipse Foundation’s Executive Director.

The Eclipse IP process guidelines also define the notion of a Works With Dependency (commonly referred to as a “works with”) that applies in two different cases. Third Party Content may be declared a Works With Dependency when:

  • the functionality of Eclipse project content is enhanced by the presence of the software, but is otherwise functional and useful without it; or
  • there are multiple choices and vetting all of them is impractical or impossible.

A Works With Dependency is, literally, a dependency that the Eclipse Project Code will work with when it is available. The fundamental requirement is the Eclipse Project Code must be useful and adoptable without the Works With Dependency. That is, either the Project Code provides useful functionality without the Works With Dependency or the Works With Dependency is a suitable alternative for a Prerequisite.

It’s enough to just register the use of Works With Dependency without seeking IP Team approval for its dependencies. The consumer is responsible for making that content available and otherwise agreeing to the terms for that content.

workswith

As with an Exempt Prerequisite, a Works With Dependency should never be directly distributed or otherwise made available without some explicit action from the consumer. Works With Dependencies must be approved by the Eclipse project’s Project Management Committee.

As an example, if Eclipse Project Code will make use of a specific database implementation when it is available, but otherwise provides useful functionality when that database is not present, that database may be a Works With Dependency. If, however, a specific database must be available for the Eclipse Project Code to provide any sort of core functionality, then that database is likely a Prerequisite which must be fully vetted. Or, if the Project Code must have access to some database, then the IP process requires that at least one specific database be fully vetted as a Prerequisite and other options can be declared as Works With Dependencies.

Eclipse committers engage the Eclipse IP Process is by filing a Contribution Questionnaire (which is commonly referred to as a CQ). Committers can find a link to create a CQ on their project page in the Eclipse Project Management Infrastructure (PMI).

As always, if you’re just not sure… ask. We’re here to help.

Posted in EDP, Intellectual Property, Project Management | Leave a comment

Running a Successful Open Source Project

This post is based on a talk that Gunnar Wagenknecht and I delivered at the Open Source Leadership Summit 2017 and Devoxx US 2017. This content was recently published in the All Eyes on Open Source issue of JAX Magazine.

Running a Succesful Open Source Project - Slides

Running an open source project is easy. All you have to do is make your source code available and you’re open source, right? Well, maybe. Ultimately, whether or not an open source project is successful depends on your definition of success. Regardless of your definition, creating an open source project can be a lot of work. If you have goals regarding adoption, for example, then you need to be prepared to invest. While open source software is “free as in beer”, it’s not really free: time and energy are valuable resources and these valuable resources need to be invested in the project.

So, how do you invest those resources?

Define success. Before you can consider running a successful open source project, you need to have a clear definition of success. There are many factors to consider. Is it enough to just get some code into a publicly accessible repository or do you want more for your project? Is collaboration and adoption important to you? Are you just trying to build your reputation as as software developer? Does your definition of success long include long term viability? Do you want to grow a community around the project? Do you care about commercial adoption? Your answer to these questions can help you decide how many of the rest of our recommendations you’ll need to adopt.

Be transparent. Transparency is pretty simple to understand: make it so that the community can watch and understand what you’re doing: use a publicly accessible source code repository that’s easy to find, use public facing issue tracking software, post your release plans where the community can find them, and capture meeting minutes in public forums (e.g. mailing lists with archives).

Be open. For a lot of open source projects, “transparency” and “openness” mean the same thing, but the terms really are quite different. Being open is more than just being “open book” (which is essentially the same thing as transparency). For many, the “open” in open source means open to new ideas, or open to participation. The rules for participating in an open source project should be the same for everybody (“level playing field”): it’s not enough to just accept a few patches, you have to be open to new ideas. In short, you have to let others in and give up absolute control of the project.

Keep the “Playing Field Level”.  This doesn’t necessarily mean that you have to let just anybody join the project, but rather that you ensure that the same set of rules apply to everybody (the playing field may be level, but you still have to earn your way onto the field). Meritocracy is about earning your way in.  Some projects implement meritocracy, for example, by requiring that developers make some number of contributions to demonstrate that they understand the code, rules, and culture of the project before inviting them to join the project team. Make sure that the process for adding new developers to your project are well known and that the process is operated transparently (e.g. a public vote).

Be vendor neutral. In order to be truly open, people need to feel welcome to contribute. This is easier if the project is vendor neutral. A vendor neutral project is not dominated by any organization or organizations; meritocracy should be based on the contributions of an individual, not the goals or hiring practices of any specific organization. Hosting at vendor neutral foundation is one way to achieve this.

Have well defined and documented standards. Be sure to document your project’s code formatting rules (make code formatter preferences easily accessible), expectations with regard to test coverage, development methodology, software and tools required, channels to connect with the project team, and other important information for potential contributors. Capture all of this information and make it as easy as possible to find. It’s a good practice to include a contribution guide in the root of your projects source code repositories (with DVCS, it’s entirely possible that potential contributors will find a copy of a copy of a copy of your repository; having the contribution guide in the repository will make it easy for potential contributors to find their way home).

Ensure that the project code is always buildable. Include build scripts and instructions with the project code. Make it as easy as possible to successfully build and test the project code.

Connect with your user community. The user community is that group of people who use the products of your open source project. The user community rarely contributes anything directly to the project code, but does tend to ask a lot of questions. Make sure that those questions get answered. A healthy user community feeds an adopter community.

Connect with your adopter community. An obvious sign of success for an open source project is that other groups start to use your open source project in their own products, or build extensions. This community is more willing to give back to the project and will be the project’s best source of contributions. Some number of those contributors will be great candidates to join your project’s team. Development of an eco-system of adopters and extenders is a great way to ensure the longevity of your project.

Connect with your development community. The development community is comprised of your project’s team members and contributors. Provide well-known channels for communication within this community. Having clear lines of communication will help developers collaborate.

Have a plan. It’s easy to lapse into a pattern of just letting software development happen, but like any process (especially a software development process), having  some method to the madness is critical. Make sure that your project employs a development methodology and make sure that somebody owns the process (e..g a project lead). Having a plan helps developers know where they can contribute the most value and makes it easier for adopters and extenders to implement their own plans (and thereby be successful). Treat your open source project like any other software development project.

Manage your brand. Your project will have a brand. The project’s name is its identity; as is the project logo, along with the names of any products (it’s typical that an the products of an open source project share the name of the project, but some projects produce more than one product). Claim the project’s brand as a trademark and consider registering that trademark. Establish trademark usage guidelines so that adopters know how to use your brand. This is an area where working with an open source software foundation can add value. A foundation can hold and defend the project’s trademarks on behalf of the community. This avoids letting any particular individual or organization hold the project’s name hostage (this happens).

Manage Intellectual property and copyright. The code, documentation, and other artifacts contributed to the project is intellectual property. Who owns that intellectual property? Do the authors retain their ownership, or do they assign it to another entity? Make sure that the rights and responsibilities of contributors are understood by all contributors. Consider having contributors sign a developer certificate of origin (DCO) or a contributor license agreement (CLA). Ensure that copyright notices are included with the source code and in notices.

Note that it is unlikely that the project itself is a legal entity that can hold copyrights. This is another way in which an open source foundation can provide a valuable service.

Pick an OSI-approved open source license. Don’t create your own custom license; that will just add legal hurdles for anybody who wants to use your code. Make sure that the license that you choose is compatible with the manner in which you intend for the code to be used. Further, ensure that the license is compatible with any third party content (e.g. libraries) that your project code needs. Include the SPDX code for your license in the headers for all source files.

Move your project to an open source foundation. We’ve mentioned foundations a few times already. A foundation can first and foremost help you to keep your project vendor neutral, which will help adoption: a bit part of the appeal of open source software is that adopters can avoid being beholden to a particular organization. A foundation can hold onto and defend the project’s trademarks, establish a governance model, help you manage your brand, provide intellectual property management services, and just generally provide assistance and advice for operating a successful open source project. Being a part of an open source foundation provides a value feedback loop. The foundation provides credibility for your open source project which in turn provides credibility to the foundation.

Running an open source project is a lot of work. But, as we’ve suggested, how much work it takes really depends on your definition of success. Fall back on the core principles of open source development: transparency, openness, and meritocracy. Everything else comes from that.

Posted in Community, Eclipse 101, Intellectual Property | Leave a comment