Eclipse Infrastructure Support for IP Due Due Diligence Type

The Eclipse Foundation’s Intellectual Property (IP) Policy was recently updated and we’re in the process of updating our processes and support infrastructure to accommodate the changes. With the updated IP Policy, we introduced the notion of Type A (license certified) and Type B (license certified, provenance checked, and scanned) due diligence types for third-party dependencies that projects can opt to adopt.

With Type A, we assert only that third-party content is license compatible with a project. For Type B third-party content, the Eclipse Foundation’s IP Team invests considerable effort to also assert that the provenance is clear and that the code has been scanned to ensure that it is clear of all sorts of potential issues (e.g. copyright or license violations). The type of due diligence applies at the release level. That is, a project team can decide the level of scrutiny that they’d like to apply on a release-by-release basis.

For more background, please review License Certification Due Diligence and What’s Your (IP Due Diligence) Type?

By default all new projects at the Eclipse Foundation now start configured to use Type A by default. We envision that many project teams will eventually employ a hybrid solution where they have many Type A releases with period Type B releases.

The default due diligence type is recorded in the project’s metadata, stored in the Project Management Infrastructure (PMI). Any project committer or project lead can navigate to their project page, and click the “Edit” button to access project metadata.

project_edit.png

In the section titled “The Basics” (near the bottom), there’s a place where the project team can specify the default due diligence type for the project (it’s reported on the Governance page). If nothing is specified, Type B is assumed. Specifying the value at the project level is basically a way for the project team to make a statement that their releases tend to employ a certain type of due diligence for third-party content.

project_dd_type.png

Project teams can also specify the due diligence type for third-party content in the release record. Again, a project committer or project lead can navigate to a release record page, and click “Edit” to gain access to the release metadata.

release_dd_type.png

As for projects, the metadata for IP due diligence type is found in the section titled “The Basics”. The field’s description is not exactly correct: if not specified in the release record, our processes all assume the value specified in the project metadata. We’ll fix this.

When the time comes to create a request to the Eclipse Foundation’s IP Team to review a contribution (a contribution questionnaire, or CQ), committers will see an extra question on the page for third-party content.

create_cq.png

As an aside, committers that have tried to use our legacy system for requesting IP reviews (the Eclipse Developer Portal) will have noticed that we’re now redirecting those requests to the PMI-based implementation. Project committers will find a direct link to this implementation under the Committer Tools block on their project’s PMI page.

We’ve added an extra field to the record that gets created in our IP tracking system (IPZilla), Type, that will be set to Type_A or Type_B (in some cases, it may be empty, “–“). We’ve also added a new statelicense_certified, that indicates that the licenses has been checked and the content can be used by the project in any Type A release.

Any content that is approved can be assumed to also be license certified.

There are many other questions that need to be answered, especially with regard to IP Logs, mixing IP due diligence types, and branding downloads. I’ll try to address these topics and more in posts over the next few days.

Posted in Intellectual Property | 1 Comment

What’s Your (IP Due Diligence) Type?

Long-time Eclipse Committer, Ian Bull initiated a interesting short chat on Twitter yesterday about one big challenge when it comes to intellectual property (IP) management. Ian asked about the implications of somebody forking an open source project, changing the license in that fork, and then distributing the work under that new license.

We can only surmise why somebody might do this (at least in the hypothetical case), but my optimistic nature tends toward assuming that this sort of thing isn’t done maliciously. But, frankly, this sort of thing does happen and the implications are the same regardless of intent.

Even-longer-time Eclipse Committer, Doug Schaefer offered an answer.

The important takeaway is that changing a license on intellectual property that you don’t own is probably bad, and everybody who touches it will potentially be impacted (e.g. potentially face litigation). I say “probably bad”, because some licenses actually permit relicensing.

Intellectual property management is hard.

The Eclipse Foundation has a dedicated team of intellectual property analysts that do the hard work on behalf of our open source project teams. The IP Team performs analysis on the project code that will be maintained by the project and for third-party libraries that are maintained elsewhere. It’s worth noting that there is no such thing as zero risk; the Eclipse IP Team’s work is concerned with minimising, understanding, and documenting risk. When they reject a contribution or third-party library use request, they do so to benefit of the project team, adopters of the project code, and everybody downstream.

In yesterday’s post, I introduced the notion of Type A (license certified), or Type B (license certified, provenance checked, and scanned). The scanned part of Type B due diligence includes—among many other things—the detection of the sort of relicensing that Ian asked about.

Since we don’t engage in the same sort of deep dive into the code, we wouldn’t detect this sort of thing with the license certification process that goes with Type A. That is, of course, not to say that it’s okay to use inappropriately relicensed third-party code in a Type A release, we just wouldn’t detect it via Type A license certification due diligence. This suggests a heightened risk associated with Type A over Type B to consider.

Type B due diligence is more resource intensive and so potentially takes a long time to complete. One of the great benefits of Type A, is that the analysis is generally faster, enabling a project team to get releases out quickly. For this reason, I envision a combination approach (some Type A releases mixed with less frequent Type B releases) to be appealing to many project teams.

So project teams needs to decide for themselves and for their downstream consumers, what sort of due diligence they require. I’ve already been a part of a handful of these discussions and am more than happy to participate in more. Project teams: you know how to find me.

It’s worth noting that Eclipse Foundation’s IP Team still does more due diligence review with Type A analysis than any other open source software foundation and many commercial organisations. If a committer suspects that shenanigans may be afoot, they can ask the IP Team to engage in a deeper review (a Type A project release can include Type B approved artifacts).

April wrapped up the Twitter conversation nicely.

Indeed. Kudos to the Eclipse Intellectual Property Team.

If you want to discuss the differences between the types of due diligence, our implementation of the Eclipse IP Policy changes, or anything else, I’ll be at Eclipse Converge and Devoxx US. Register today.

Eclipse Converge

Posted in Intellectual Property, Uncategorized | 2 Comments

License Certification Due Diligence

With the changes in the Eclipse Intellectual Property (IP) Policy made in 2016, the Eclipse Foundation now offers two types of IP Due Diligence for the third-party software used by a project. Our Type A Due Diligence involves a license certification only and our Type B Due Diligence provides our traditional license certification, provenance check, and code scan for various sorts of anomalies. I’m excited by this development at least in part because it will help new projects get up to speed more quickly than they could have in the past.

Prior to this change, project teams would have to wait until the full application of what we now call Type B Due Diligence was complete before issuing a release. Now, a project team can opt to push out a Type A release after having all of their third-party libraries license certified.

A project team can decide what level of IP Due Diligence they require for each release. Hypothetically, a project team could opt to make several Type A releases followed by a Type B release, and then switch back. I can foresee this being something that project teams that need to engage in short release cycles will do.

We’ve solicited a few existing projects to try out the new IP Due Diligence type and have already approved a handful of third-party libraries as Type A. The EMO has also started assuming that all new projects use Type A (license certification) by default. As we move forward, we expect that all new projects will employ Type A Due Diligence for all incubation releases and then decide whether or not to switch to Type B (license certification, provenance check, and code scan) for their graduation. There is, of course, no specific requirement to switch at graduation or ever, but we’re going to encourage project teams to defer the decision of whether or not to switch from Type A until that point.

After graduation, project teams can decide what they want to do. We foresee at least some project teams opting to issue regular multiple Type A releases along with an annual Type B release (at this point in the time, there is no specific requirement to be Type A or Type B to participate in the simultaneous release).

We’ve started rolling out some changes to the infrastructure to support this update to the IP Due Diligence process. I’ll introduce those changes in my next post.

Update: Based on some Tweets, I changed my intended topic for the next post. Please see What’s Your (IP Due Diligence) Type?

Eclipse Converge

Posted in Intellectual Property | 2 Comments

Sleepwalk, So Fast Asleep

One of my favourite literary quotes comes from Canadian author, Hugh MacLennan.

“But that night as I drove back to Montreal, I at least discovered this: that there is no simple explanation for anything important any of us do, and that the human tragedy, or the human irony, consists in the necessity of living with the consequences of actions performed under the pressure of compulsions so obscure we do not and cannot understand them.”
― Hugh MacLennan, The Watch that Ends the Night

I think that this describes our industry pretty well…

This passage was the basis for the the song Courage by the Tragically Hip, arguably one of (if not the) most Canadian bands in existence (seriously, the entire country basically shut down for the night while a third of the population dropped everything to gather in groups to watch their last concert on August 20/2016).

Anyway, this is the long way of saying that I have no recollection of why I selected the username wayninator for my primary Google account many years ago. But I’m apparently stuck with it.

Posted in Cranky Old Programmer, Other | 2 Comments

The source for the Eclipse Project Handbook is in AsciiDoc format (rendered using Asciidoctor via Maven), and includes embedded Graphviz graph descriptions (which are rendered into embedded images). Proofing those images is a bit of a pain: it requires that I actually build the handbook and use a browser to view the results.

A few years ago, I wrote an Image Util plug-in, primarily as an excuse to learn about how adapters work. I pulled that old plug-in into my current Eclipse IDE (Neon edition) and was quite happy to learn that it still works. The implementation is simple enough: it provides an Image Preview view that—when possible—adapts the current selection into an ImageProvider that (curiously enough) provides an image to display. It works brilliantly for file selections and I even made it work for selections in the Remote Systems Explorer.

So, frustrated with the round-trip process for testing images (after spending some quality time hunting down an existing solution), I created a plug-in that adapts an arbitrary text selection into an image provider that invokes the Graphviz dot command to render the selection into an image. I was a bit stunned when my first attempt at implementing it just worked. It works in every text editor that I’ve tested it with (including the handy AsciiDoc WikiText editor).

screenshot-from-2016-10-06-17-20-54

Anyway… it’s a quick and dirty hack (albeit, a hack with some potential) that requires the command-line tool be installed, but I think that it demonstrates some of the power that comes when your platform has well-documented and stable APIs (FWIW, the implementation has about seven meaningful lines of code).

Note that the implementation is currently only available as source code. Sorry, there’s no software site yet. Maybe that’s something we can work on at the EclipseCon Europe 2016 Hackathon!

EclipseCon Europe 2016

Posted in Eclipse 101, Examples, Hackathon | 4 Comments

Replacing Bugzilla with Tuleap

Bugzilla has served the Eclipse Community well for many years, easily scaling to serve the needs of over 500 open source projects, servicing a community of thousands of software developers generating half of a million issue reports over close to two decades (I’m taking some liberties with that last one: it’s been about 17 years). When I say “easily”, I mean that our world-class IT team has made it look easy.

While generally robust, Bugzilla isn’t sexy, and it’s missing some valuable features. I’m also a little concerned that the team responsible for maintaining Bugzilla doesn’t seem to have the resources necessary to move Bugzilla forward (though, I’ll admit that I have only limited insight).

I’ve been investigating Tuleap, which is billed as “software development and project management tool”, as a potential successor to Bugzilla. Any effort to make Tuleap a first-class service for Eclipse projects will include the deprecation and eventual retirement of Bugzilla. Much like our migration from CVS to Git, I expect that project teams will start slowly adopting the new technology. Much like that other migration, the pace will likely pick up quickly as project teams see the benefits being reaped by the earlier adopters. This is how it will have to be: with project teams migrating themselves completely off Bugzilla and into the new system.

The criteria for picking a Bugzilla successor is pretty straightforward. Any new issue tracking software we adopt must:

  1. be able to import our existing issue (bug) data;
  2. be open source;
  3. have a strong community behind it; and
  4. provide equivalent functionality to what Bugzilla provides today.

Tuleap can import existing Bugzilla data directly, so that’s a huge win. We’ve discussed potential for synchronising data across systems, but that’s a big and expensive challenge that I’d really like to avoid. Once a team decides to move to Tuleap, the data will be moved and they’ll be off. This is an important consideration for project teams that might be considering participating in the ongoing proof of concept: we do have a means of moving your existing data from Bugzilla to Tuleap, but currently don’t have a means of making the return trip should we decide to move in a different direction.

Requiring that the replacement be open source is, I think, obvious. All of our public-facing systems are open source today and this will continue into the future.  Tuleap is 100% open source.

Having a strong community is important and ties into the open source requirement. Replacing Bugzilla is going to be a massive undertaking and we need to have confidence that the replacement that we choose has some staying power. Tuleap appears to have some large enterprise organizations adopting the technology, which is encouraging. We’ll expect to contribute to the open source project, but we need to be part of a community that plans to contribute and continue to evolve the product.

I think that it would be a mistake to look for a feature-to-feature replacement for Bugzilla. If we’re going to make this change, then we need to do some dreaming. Having said that, with my experimentation so far, I believe that Tuleap provides most of the fundamental functionality that we require.

A small number of our projects have started to independently evaluate the potential to use Tuleap. I’m certainly not an expert, but I’ve engaged with both the Tuleap project itself and several the Eclipse projects testing Tuleap via Tuleap as a user, so I’m starting to get a pretty good feel for it. So far, my experience has shown that it’s a pretty awesome bit of software and the projects that are currently using it seem to be moving along nicely.

Tuleap is more than an issue tracker. What I think sells a lot of project teams on Tuleap is the ability to engage in a full Scrum or Kanban process, but Tuleap provides all sorts of other services, including a wiki, document management, Git hosting, Gerrit integration, and more. It is relatively easy for a project team to create additional Git repositories from directly within the Tuleap interface. There is a notion of a project landing page that likely overlaps (at least a little) with the PMI; it may be worth investigating integration options (I’m not particularly keen on project teams being required to manage multiple landing pages on different services).

Tuleap provides fine-grained permissions which would make it relatively easy to, for example, give a project lead the ability to grant access to non-committer triagers (I believe that our various policies permit this sort of thing).

It is possible to add custom fields into a project team’s bucket to capture information that may be specific to the project. My understanding, however, is that the user experience for this sort of administrative task is not as refined as the user-facing interface. Customisation can be templated, which may make it possible to share some set of common customisations. But customisation introduces a troubling problem: since the source and target may have different customisation, moving issues from one team’s bucket to another team’s bucket is not currently supported: moving issues must be easy (especially considering the number of bugs that are incorrectly opened against JDT). Tuleap also does not currently provide an easy means of marking duplicates. According to the project team, these important features just haven’t been implemented yet. I’ve opened an artifact to track the need requirement to easily move issues.

Unfortunately, I believe that the abilities to move an issue from one project team to another, and to quickly and easily mark duplicates are critical to our adoption and so are show stoppers for the moment.

I have been trying to think a bit outside of the box. One of the bigger drivers for the need to be able to move issues is that the JDT team is the target for any and every issue that comes up in any variant of an Eclipse IDE that includes Java development tools. Maybe we can use adoption of a new issue tracking system as an opportunity to help our users find the right place to report issues without having to navigate our project structure. That’s going to require a lot of thought and energy from the community as a whole.

I think that Tuleap has a lot of potential, but we’re not quite ready for widespread adoption by Eclipse projects. I’ve managed to identify two show stoppers, but they’re point-in-time problems that will be addressed. I believe that we’re setting up a BoF at EclipseCon Europe 2016; if you’re using Tuleap now, or are interested in being a part of the decision making process for widespread adoption, please attend that session (and, of course, please do feel free to connect with me one-on-one at the conference). Please also to interact with the projects currently using Tuleap and let me know if your project is interested in participating in the experiment.

Finally, please join the discussion on Bug 497333.

EclipseCon Europe 2016

Posted in Community | 7 Comments

Java 9 module-info Files in the Eclipse IDE

Note that this post is not intended to be a status update; it’s just a quick update based on some experimenting that I’ve been doing with the beta code.

It’s been a while, but I’m back to experimenting in Java 9 support in the Eclipse IDE.

For testing purposes, I downloaded the most recent Oxygen (4.7) integration build (I20160914-0800) from the Eclipse Project downloads the latest  Java 9 JRE build (135).

I configured the Eclipse IDE to run on the Java 9 JVM. This still requires a minor change in the eclipse.ini file: to launch successfully, you must add --add-modules=java.se.ee to the vmargs section (I expect this to be resolved before Java 9 support is officially released; see Bug 493761 for more information). I used and used the Install new software… dialog to pull in updates from the BETA_JAVA9 SDK builds repository (see the Java9 Eclipsepedia page for more information).

I created a very simple Java application with a module-info.java file. Content assist is available for this file.

screenshot-from-2016-09-14-14-27-24

Note that there is an error indicated on the import of java.awt.Frame. This error exists because the module info file does not provide visibility to that class (AWT is not included with java.base).

If we change that requires statement, the visibility issue is resolved and the compiler is happy. Well, mostly happy. Apparently not using declared variables gets you a stern warning (this is, of course, configurable).

screenshot-from-2016-09-14-14-27-51

The Eclipse Project is planning to ship support as part of an Eclipse Neon update release that coincides with the official release date of Java 9. I’ll be talking a bit about this during my JavaOne talk and demonstrating this (and more Java topics) at the Eclipse Foundation’s booth.

Conference: JavaOne
Session Type: Conference Session
Session ID: CON6469
Session Title: Developing Java Applications with Eclipse Neon
Room: Hilton—Continental Ballroom 6
Date and Time: 09/19/16, 11:00:00 AM – 12:00:00 PM

The call for papers for Devoxx US is open. Devoxx is a community conference from developers for developers. Submit your proposal now.

Posted in Java, Neon, Screenshots | Leave a comment