Service Releases of Third Party Content in the Eclipse IP Due Diligence Process

Some time ago, the Eclipse Foundation’s Board of Directors passed the following resolution.

RESOLVED, that previously approved dependencies of Eclipse projects can be
reviewed and approved by the EMO as follows:
a) Service releases (e.g. x.y., bug fixes, security fixes) will require no review.
b) Minor revisions (e.g. x..) will require a reduced review by the EMO.
c) Major revisions (e.g. ..) will require a full review by the EMO.

At the time the resolution was passed, it was decided that project teams would still be required to register the use of service releases for third party content with the Eclipse Intellectual Property (IP) team (i.e. create a Contribution Questionnaire); it was decided that the resolution really only impacted how the IP Team would process the content: at least in theory, the IP Team would rubber stamp requests for service releases. I say in theory, because the IP Team would still eyeball these requests and sometimes invest some modest effort to ensure that significant new IP hadn’t been introduced.

We’ve reconsidered this position.

Project teams are not required to engage with the Eclipse IP Team to review service releases for third party content, provided that the service release is based on a previously-approved release. This assumes that the providers of the third party content regard service releases in the manner described by Semantic Versioning (i.e. a patch of backwards-compatible bug fixes) and that the service release does not include significant new IP.  Of course, if a project team isn’t sure, they can always just submit the content for review anyway.

We envision a scenario where project teams will take advantage of this for milestone builds, but then get full approval for final versions of their third party content. This isn’t required, but it is an option that’s available for project teams that feel that it’s necessary.

For major and minor releases, the manner in which the the project team interacts with the Eclipse IP Team does not change. Note that it is implied that the reduced review for minor releases follows an approved full review for a corresponding major release and that any service release follows an approved major or minor release.

For more information, to add your thoughts and concerns, or to track our progress with regard to updating our documentation, add yourself in copy to Bug 512465.

Posted in Intellectual Property | Leave a comment

Run Eclipse IDE on One Version of Java, but Target Another

The Eclipse IDE for Java™ Developers (and the other Java developer variants) is itself a Java application that’s used to build Java applications. That relationship can be a bit weird to wrap your brain around.

Written almost entirely in Java, the Eclipse IDE requires a Java Runtime Environment (JRE) to run. A JRE provides just the runtime platform: it doesn’t include the source code and Javadoc for any of the base Java libraries, or any of the development tools that are included in the Java Development Kit (JDK). An Eclipse IDE runs just fine on a JRE.

If you’re building Java applications, however, you really need to have access to a JDK. By default, an Eclipse IDE will configure itself to build applications against the JRE that it was launched on. If that JRE is part of a JDK, then you’ll get access to all the goodies that you need to get useful content assist, documentation, debugging support, etc. If the runtime platform is just a JRE, then a lot of that valuable goodness will be missing (but compiling still works because the Java development tools include the Eclipse Compiler for Java)

Here’s where it gets a bit weird. You can run an Eclipse IDE on a JRE from one version of Java and build applications that target one or more different versions of Java. You can, for example, run your Eclipse IDE on Java 8, but use it to build applications based on basically any earlier version of Java. You can select the default Java version for your workspace in the preferences (on the Java > Compiler page), or individually in the properties for each Java Project (preferred).

JDK Compliance settings in the Preferences (Java > Compiler)

In order to actually build applications on a different version of Java, you need to connect your Eclipse IDE with the corresponding JDK. To do this, first install the JDK, and then tell the Eclipse IDE where to find it via the Java > Installed JREs page in the workspace preferences. With additional JDKs installed, you can configure individual projects to use specific versions of the compiler and runtime.

All of this is a long way of saying that you can configure your Eclipse IDE, Oxygen Edition milestone build to run on a Java 9 JRE, (download a JDK from the JDK 9 Early Access site) but use it to build applications that target earlier versions of Java (i.e. keep doing your day job). Even doing just this and providing feedback will be very helpful to the Eclipse projects involved in the Eclipse IDE. If you actually want to build Java 9 applications, you’ll need to install the Java 9 Support (BETA) for Oxygen from the Eclipse Marketplace and provide feedback to the team.

Note that the JDT product in the Eclipse Foundation’s Bugzilla instance is specifically for reporting bugs that are directly related to the Java development tools. Use the EGit product to report issues with the Git integration, the m2e product to report issues with the Maven integration, the Buildship product to report issues with the Gradle integration, the Platform product for issues with regard to the basic IDE framework, the Web Tools product for issues with enterprise Java and web development, or the EPP (Eclipse Packaging Project) product if you’re not sure.

Many committers from the various Eclipse projects that contribute to the Eclipse IDE, including at least a couple of committers from the Java development tools project in particular, will be at Devoxx US. If you want to learn more about Java 9 support in the Eclipse IDE, you’ll be able to find them at the Eclipse Foundation’s booth (or we’ll be able to help you find them).

I’ll also be doing a demonstration of Test First Java Development using the Eclipse IDE at the booth. Come by and see if I can keep myself within the twenty minute limit…

If you want to learn more about the great features available in the Eclipse IDE, follow @EclipseJavaIDE on Twitter (follow me while you’re at it).

devoxx_black_transparent400

Posted in Community, Conferences, Eclipse 101, Java, Screenshots | Leave a comment

Eclipse IDE, Oxygen Edition; and Java 9

I’ll admit that I dropped the ball for a while, but I’m back to testing the Eclipse IDE with Java 9. Java 9 includes some pretty major changes, especially in the way that it restricts access to internal code, and so there’s a real risk that Java-based applications are going to break. Just confirming that the Eclipse IDE runs on Java 9 is a critical task (even before we talk about leveraging actual Java 9 features).

Today, I downloaded the Java 9 Early Access Build 158 (March 1/2017) and reconfigured my working installation of the Eclipse IDE for RCP and RAP Developers, Oxygen Edition (M5) to use it. I’ve added a few extra features to my installation, including the PHP Development Tools, Data Tools SQL Designer, Mylyn WikiText, TM Terminal, and a handy regular expression helper/tester that I found in the Eclipse Marketplace.

Configuration is relatively straight forward. The nice thing about the early access builds is that you can just decompress them into a directory on your system without installing them (which will have a impact on your system’s configuration). The downside is that you have to configure your Eclipse IDE to find it. Actually, this isn’t really much of a downside in my opinion: I like to have some degree of control over what Java  runtime I’m using to run the Eclipse IDE rather than just grabbing the system default.

I decompressed the Java build and updated my eclipse.ini file to point point to it:

...
-vm
/home/apps/jigsaw-jdk-9-ea+158_linux-x64_bin/jdk-9/bin/
-vmargs
--add-modules=java.se.ee
...

Note that you need to add the --add-modules option to tell Java 9 to load some modules that it wouldn’t normally load by default. I’m hopeful that we’ll have an update to the launcher to automatically add this option in the M6 builds (see Bug 493761); with this launching the Eclipse IDE will just work if you happen to have a Java 9 JVM installed as your system default.

As a sanity check, I opened the About Eclipse dialog and reviewed the Configuration in the Installation Details to confirm that I’m actually running on Java 9. The next step is to actually start writing some code that leverages Java 9 features; for that I’ll need to install the Java 9 Support (BETA) for Oxygen from the Eclipse Marketplace.

I’m looking forward to learning a lot more about Java 9 at Devoxx US, including these sessions:

See you at Devoxx!
devoxx_black_transparent400

Posted in Conferences, Java | 4 Comments

Language Server Protocol Talks at Eclipse Converge and Devoxx US

I’m particularly interested in learning more about Language Server Protocol at the combined Eclipse Converge and Devoxx US conferences at the end of March. We have a handful of projects doing work on the topic, and the opportunity to connect directly with the developers doing the work is too good of an opportunity to miss.

I’m hoping to fit the following talks into my schedule.

Language Server Protocol Explained

Sven Efftinge

The Language Server Protocol (LSP) introduced by Microsoft’s VSCode team has been a hot topic recently. In a nutshell it is an effort to unify how editors communicate with advanced language tooling.

In this session we want to give you an overview of what the LSP is, why it is so important and how you could leverage it. We will also explain what it does and what it doesn’t do, discuss some misconceptions and show some cool demos based on a Java implementation of the protocol.

A sneak peek into the Spring Boot language server

Martin Lippert

Based in the language server protocol and the architectural ideas behind it, big chunks of the Spring Boot tooling for Eclipse are currently re-implemented. This talk provides a sneak peek into the implementation and discusses the early experiences using this approach. We dive into the details and challenges how we implemented tooling for Spring Boot property files (pure properties + yaml), support for Cloud Foundry manifest files, and how we extracted and refactored existing Spring IDE code to run inside of a language server.

Building the JDT Language Server and how can you build one from your own feature

Gorkem Ercan

Eclipse JDT LS (Language Server) project aims to develop a Java Language Server that will make JDT features available to any client that supports the Language Server Protocol. This talk covers the changes, challenges and lessons learned while building the JDT LS. It will also provide insights to using and developing language servers and some tips to converting existing features. There will also be demos of JDT LS features on different editors and their comparision with the Eclipse JDT to give a better understanding of the possibilities and problems of language servers.

Roll your own Development Environment with Eclipse Orion

Bogdan Gheorghe

Eclipse Orion is a cloud IDE that supports JavaScript development out of the box.  Recently, support for LSP (Language Server Protocol) has been added to the mix.  Come and learn how to create three different development environments using Docker, Orion and LSP (Language Sever Protocol).

Introduction to Eclipse Che

Stévan Le Meur and Florent Benoit

Eclipse Che introduces a new kind of workspace that is composed of projects and runtimes. This approach improves agile workflow and allows fast bootstrapping of developers. Eclipse Che can run locally or in the cloud which allow to scale the resources on-demand and benefit from high performances and resources.

In this session, we will explain how to setup a workspace cloud in Eclipse Che, how to create the environments using Docker, configure the tools that you need and register a set of commands to be executed in that workspace. We will show you how you can benefits from the workspace portability to easily share it onto another Che or to the cloud.

This session will also cover how Eclipse Che is providing support and intellisense for all the languages and explain the work did on the Language Server Protocol. The Language Server Protocol is a communication protocol between a tool and a Language Server than run all code analysis and operations

Eclipse Converge devoxx_black_transparent400

Posted in Uncategorized | Leave a comment

Prerequisite Dependencies

All third party content must be taken through the Eclipse Foundation’s Intellectual Property (IP) Due Diligence Process before being used by an open source project hosted by the Eclipse Foundation. This includes all third party content that is incorporated into project code, included in builds, or otherwise required by the project code to provide functionality.

The Eclipse Foundation’s Guidelines for the Review of Third Party Dependencies defines various classifications of third party content and how project teams should engage in the IP process. The classification of third party content most commonly used is the so-called prerequisite dependency: a prerequisite dependency is third party content that the project either needs to distribute along with the project code, or otherwise have present in order to function.

Third party content is likely a prerequisite dependency if:

  • project code accesses APIs;
  • project code includes an import statement for a package from a library;
  • project code uses reflection or other means to reference a library’s APIs and implementation;
  • the Java/OSGi manifest for a project bundles makes a direct reference; or
  • project code invokes a command line tool to access the functionality.

This list is not intended to be exhaustive, but rather to provide common examples. Fundamentally, if project code makes some kind of use of third-party content, then that content is likely a prerequisite. The guidelines provide for so-called works with and exempt prerequisite dependencies, but I’ll save that discussion for a future post.

Project committers can engage in the IP Due Diligence process by creating a contribution questionnaire (CQ) in the Eclipse Foundation’s IPZilla system.

The term contribution questionnaire is a bit misleading when it comes to third party content. Third party content is not a really a contribution to the project, but since the requirements for tracking project code and third party content is very similar, the same mechanism is used for both.

The workflow looks a little something like this:

prereq

screenshot-from-2017-02-08-13-25-01

Committer Tools

There’s an entry point that project committers can use to Create a Contribution Questionnaire in the Committer Tools block that is on every project’s information page (see the image on the right).

In the first step, the committer creates the CQ record with essential information about the third party content and then attaches the source code for that content (this currently happens in two separate steps). The corresponding Project Management Committee (PMC) must sign-off on the CQ before any further processing occurs.

As a general rule, a CQ should be created for each separate library. The process permits for some flexibility with regard to the definition of library. Content that has source in a single source repository that is distributed as multiple JAR files can very likely be regarded as a single library. There has also been cases where the IP Team has accepted a single CQ to do license certification for a collection of many different JavaScript sources. If you’re not sure, check with the IP Team.

License scanning software will be engaged for third party content that’s to be reviewed for license certification (type A). As we roll out this new type of IP due diligence, the IP Team is engaging the tool manually, evaluating the output, and making a determination. Our plan is to automate the invocation of the tool and make a determination automatically where possible (e.g. in cases where the licenses are clearly indicated in the content) and have the IP Team investigate further when necessary.

For requests to perform the more thorough IP due diligence process (type B), the workflow is different. For content that qualifies for parallel IP processing, the IP Team will do a cursory review and—assuming that the content passes muster—grant check in, meaning that the content can be pushed into the project’s source code repository and included in builds. The full due diligence review includes verification of the content’s provenance and a scan for all kinds of other anomalies (more on this in future posts). This process is a lot work that can take quite a lot of time to complete, and very often requires interaction with the committer. When the IP team completes a full review with a successful outcome, they’ll mark the CQ as approved.

All third party content must be either license certified or approved before an project can issue an official release. Any release that includes license certified content must be marked as a type A. A type A release may include type B content; A type B release, however, cannot include type A content.

I’ve thrown out a few concepts in this post without providing explanations. I’ll try and fill in the gaps in future posts, which will all be grouped in the intellectual property category.

Posted in Intellectual Property | Leave a comment

Eclipse Foundation Open Source Project Announcements, February 10/2017

Intellectual Property Policy Changes Implementation

You’ve likely heard about the introduction of a new type of intellectual property (IP) due diligence for third party content. The short version is that our Type A Due Diligence involves a license certification only and our Type B Due Diligence provides our traditional license certification, provenance check, and code scan for various sorts of anomalies. I’ve been blogging about it: take a look at my blog’s Intellectual Property category for more information.

Vulnerability Reporting Process Tweaks

I’ve been working on some updates to our policy and procedures regarding security issues and vulnerability reporting.

Committers should familiarize themselves with the Eclipse Security Policy. The policy describes a means for tracking discussion on sensitive issues without immediately disclosing them to the public via a “committer only” designation in Bugzilla. Unfortunately, GitHub Issues does not have a means of privately discussing issues between committers, so we’ve set up a solution that uses the Eclipse Bugzilla instance. The Eclipse Webmaster created a generic bucket for capturing vulnerability reports and we are putting the pieces together to ensure that issue reports get directed correctly (e.g. assign them to the right project lead).

We’ve included a handy link on the security page to make it easy to create bug reports in the right state (i.e. with the committers only flag turned on). I encourage project teams (especially those working on runtime technology) to consider including a project-specific link for reporting vulnerabilities.

Note that it is our policy that all vulnerabilities eventually get disclosed, so issue privacy should be considered as short term state to give a project team an opportunity to get ahead of a vulnerability.

Google Summer of Code

From the Google Summer of Code Student Manual:

Google Summer of Code (GSoC) is a global program that matches students up with open source, free software and technology-related organizations to write code and get paid to do it! The organizations provide mentors who act as guides through the entire process, from learning about the community to contributing code. The idea is to get students involved in and familiar with the open source community and help them to put their summer break to good use.

Project teams that intend to participate in the Google Summer of Code should visit our Information Page, sign up for the soc-dev mailing list, and add student project ideas to the Ideas Page. You may also consider marking some of your bugs as helpwanted or bugday.

Note that we’re still in the mentoring organization application stage; we’ll let you know when it’s time to sigh up as a mentor or student.

Project Announcements

There are some reviews concluding on February 15, 2017:

We have several proposals open for community review:

Please add your comments either directly on the proposal or in the Proposals forum.

We run reviews ending on the first and third Wednesday of each month. Our next scheduled review dates are March 1, 2017 and March 15, 2017.

For more information about releases and reviews, please see the Eclipse Project Handbook.

Eclipse Foundation Projects Team at Eclipse Converge and Devoxx US

The Eclipse Foundation Projects Team will be at Eclipse Converge and Devoxx US in March. We’ll be there to answer your questions, and help you work through any process-related issues. We’ll be hanging out the Eclipse Foundation’s Booth.  Join us there!

devoxx_black_transparent400

Posted in Announcements, Community | Leave a comment

Eclipse Infrastructure Support for IP Due Due Diligence Type

The Eclipse Foundation’s Intellectual Property (IP) Policy was recently updated and we’re in the process of updating our processes and support infrastructure to accommodate the changes. With the updated IP Policy, we introduced the notion of Type A (license certified) and Type B (license certified, provenance checked, and scanned) due diligence types for third-party dependencies that projects can opt to adopt.

With Type A, we assert only that third-party content is license compatible with a project. For Type B third-party content, the Eclipse Foundation’s IP Team invests considerable effort to also assert that the provenance is clear and that the code has been scanned to ensure that it is clear of all sorts of potential issues (e.g. copyright or license violations). The type of due diligence applies at the release level. That is, a project team can decide the level of scrutiny that they’d like to apply on a release-by-release basis.

For more background, please review License Certification Due Diligence and What’s Your (IP Due Diligence) Type?

By default all new projects at the Eclipse Foundation now start configured to use Type A by default. We envision that many project teams will eventually employ a hybrid solution where they have many Type A releases with period Type B releases.

The default due diligence type is recorded in the project’s metadata, stored in the Project Management Infrastructure (PMI). Any project committer or project lead can navigate to their project page, and click the “Edit” button to access project metadata.

project_edit.png

In the section titled “The Basics” (near the bottom), there’s a place where the project team can specify the default due diligence type for the project (it’s reported on the Governance page). If nothing is specified, Type B is assumed. Specifying the value at the project level is basically a way for the project team to make a statement that their releases tend to employ a certain type of due diligence for third-party content.

project_dd_type.png

Project teams can also specify the due diligence type for third-party content in the release record. Again, a project committer or project lead can navigate to a release record page, and click “Edit” to gain access to the release metadata.

release_dd_type.png

As for projects, the metadata for IP due diligence type is found in the section titled “The Basics”. The field’s description is not exactly correct: if not specified in the release record, our processes all assume the value specified in the project metadata. We’ll fix this.

When the time comes to create a request to the Eclipse Foundation’s IP Team to review a contribution (a contribution questionnaire, or CQ), committers will see an extra question on the page for third-party content.

create_cq.png

As an aside, committers that have tried to use our legacy system for requesting IP reviews (the Eclipse Developer Portal) will have noticed that we’re now redirecting those requests to the PMI-based implementation. Project committers will find a direct link to this implementation under the Committer Tools block on their project’s PMI page.

We’ve added an extra field to the record that gets created in our IP tracking system (IPZilla), Type, that will be set to Type_A or Type_B (in some cases, it may be empty, “–“). We’ve also added a new statelicense_certified, that indicates that the licenses has been checked and the content can be used by the project in any Type A release.

Any content that is approved can be assumed to also be license certified.

There are many other questions that need to be answered, especially with regard to IP Logs, mixing IP due diligence types, and branding downloads. I’ll try to address these topics and more in posts over the next few days.

Posted in Intellectual Property | 1 Comment