Language Server Protocol Talks at Eclipse Converge and Devoxx US

I’m particularly interested in learning more about Language Server Protocol at the combined Eclipse Converge and Devoxx US conferences at the end of March. We have a handful of projects doing work on the topic, and the opportunity to connect directly with the developers doing the work is too good of an opportunity to miss.

I’m hoping to fit the following talks into my schedule.

Language Server Protocol Explained

Sven Efftinge

The Language Server Protocol (LSP) introduced by Microsoft’s VSCode team has been a hot topic recently. In a nutshell it is an effort to unify how editors communicate with advanced language tooling.

In this session we want to give you an overview of what the LSP is, why it is so important and how you could leverage it. We will also explain what it does and what it doesn’t do, discuss some misconceptions and show some cool demos based on a Java implementation of the protocol.

A sneak peek into the Spring Boot language server

Martin Lippert

Based in the language server protocol and the architectural ideas behind it, big chunks of the Spring Boot tooling for Eclipse are currently re-implemented. This talk provides a sneak peek into the implementation and discusses the early experiences using this approach. We dive into the details and challenges how we implemented tooling for Spring Boot property files (pure properties + yaml), support for Cloud Foundry manifest files, and how we extracted and refactored existing Spring IDE code to run inside of a language server.

Building the JDT Language Server and how can you build one from your own feature

Gorkem Ercan

Eclipse JDT LS (Language Server) project aims to develop a Java Language Server that will make JDT features available to any client that supports the Language Server Protocol. This talk covers the changes, challenges and lessons learned while building the JDT LS. It will also provide insights to using and developing language servers and some tips to converting existing features. There will also be demos of JDT LS features on different editors and their comparision with the Eclipse JDT to give a better understanding of the possibilities and problems of language servers.

Roll your own Development Environment with Eclipse Orion

Bogdan Gheorghe

Eclipse Orion is a cloud IDE that supports JavaScript development out of the box.  Recently, support for LSP (Language Server Protocol) has been added to the mix.  Come and learn how to create three different development environments using Docker, Orion and LSP (Language Sever Protocol).

Introduction to Eclipse Che

Stévan Le Meur and Florent Benoit

Eclipse Che introduces a new kind of workspace that is composed of projects and runtimes. This approach improves agile workflow and allows fast bootstrapping of developers. Eclipse Che can run locally or in the cloud which allow to scale the resources on-demand and benefit from high performances and resources.

In this session, we will explain how to setup a workspace cloud in Eclipse Che, how to create the environments using Docker, configure the tools that you need and register a set of commands to be executed in that workspace. We will show you how you can benefits from the workspace portability to easily share it onto another Che or to the cloud.

This session will also cover how Eclipse Che is providing support and intellisense for all the languages and explain the work did on the Language Server Protocol. The Language Server Protocol is a communication protocol between a tool and a Language Server than run all code analysis and operations

Eclipse Converge devoxx_black_transparent400

Posted in Uncategorized | Leave a comment

Prerequisite Dependencies

All third party content must be taken through the Eclipse Foundation’s Intellectual Property (IP) Due Diligence Process before being used by an open source project hosted by the Eclipse Foundation. This includes all third party content that is incorporated into project code, included in builds, or otherwise required by the project code to provide functionality.

The Eclipse Foundation’s Guidelines for the Review of Third Party Dependencies defines various classifications of third party content and how project teams should engage in the IP process. The classification of third party content most commonly used is the so-called prerequisite dependency: a prerequisite dependency is third party content that the project either needs to distribute along with the project code, or otherwise have present in order to function.

Third party content is likely a prerequisite dependency if:

  • project code accesses APIs;
  • project code includes an import statement for a package from a library;
  • project code uses reflection or other means to reference a library’s APIs and implementation;
  • the Java/OSGi manifest for a project bundles makes a direct reference; or
  • project code invokes a command line tool to access the functionality.

This list is not intended to be exhaustive, but rather to provide common examples. Fundamentally, if project code makes some kind of use of third-party content, then that content is likely a prerequisite. The guidelines provide for so-called works with and exempt prerequisite dependencies, but I’ll save that discussion for a future post.

Project committers can engage in the IP Due Diligence process by creating a contribution questionnaire (CQ) in the Eclipse Foundation’s IPZilla system.

The term contribution questionnaire is a bit misleading when it comes to third party content. Third party content is not a really a contribution to the project, but since the requirements for tracking project code and third party content is very similar, the same mechanism is used for both.

The workflow looks a little something like this:

prereq

screenshot-from-2017-02-08-13-25-01

Committer Tools

There’s an entry point that project committers can use to Create a Contribution Questionnaire in the Committer Tools block that is on every project’s information page (see the image on the right).

In the first step, the committer creates the CQ record with essential information about the third party content and then attaches the source code for that content (this currently happens in two separate steps). The corresponding Project Management Committee (PMC) must sign-off on the CQ before any further processing occurs.

As a general rule, a CQ should be created for each separate library. The process permits for some flexibility with regard to the definition of library. Content that has source in a single source repository that is distributed as multiple JAR files can very likely be regarded as a single library. There has also been cases where the IP Team has accepted a single CQ to do license certification for a collection of many different JavaScript sources. If you’re not sure, check with the IP Team.

License scanning software will be engaged for third party content that’s to be reviewed for license certification (type A). As we roll out this new type of IP due diligence, the IP Team is engaging the tool manually, evaluating the output, and making a determination. Our plan is to automate the invocation of the tool and make a determination automatically where possible (e.g. in cases where the licenses are clearly indicated in the content) and have the IP Team investigate further when necessary.

For requests to perform the more thorough IP due diligence process (type B), the workflow is different. For content that qualifies for parallel IP processing, the IP Team will do a cursory review and—assuming that the content passes muster—grant check in, meaning that the content can be pushed into the project’s source code repository and included in builds. The full due diligence review includes verification of the content’s provenance and a scan for all kinds of other anomalies (more on this in future posts). This process is a lot work that can take quite a lot of time to complete, and very often requires interaction with the committer. When the IP team completes a full review with a successful outcome, they’ll mark the CQ as approved.

All third party content must be either license certified or approved before an project can issue an official release. Any release that includes license certified content must be marked as a type A. A type A release may include type B content; A type B release, however, cannot include type A content.

I’ve thrown out a few concepts in this post without providing explanations. I’ll try and fill in the gaps in future posts, which will all be grouped in the intellectual property category.

Posted in Intellectual Property | Leave a comment

Eclipse Foundation Open Source Project Announcements, February 10/2017

Intellectual Property Policy Changes Implementation

You’ve likely heard about the introduction of a new type of intellectual property (IP) due diligence for third party content. The short version is that our Type A Due Diligence involves a license certification only and our Type B Due Diligence provides our traditional license certification, provenance check, and code scan for various sorts of anomalies. I’ve been blogging about it: take a look at my blog’s Intellectual Property category for more information.

Vulnerability Reporting Process Tweaks

I’ve been working on some updates to our policy and procedures regarding security issues and vulnerability reporting.

Committers should familiarize themselves with the Eclipse Security Policy. The policy describes a means for tracking discussion on sensitive issues without immediately disclosing them to the public via a “committer only” designation in Bugzilla. Unfortunately, GitHub Issues does not have a means of privately discussing issues between committers, so we’ve set up a solution that uses the Eclipse Bugzilla instance. The Eclipse Webmaster created a generic bucket for capturing vulnerability reports and we are putting the pieces together to ensure that issue reports get directed correctly (e.g. assign them to the right project lead).

We’ve included a handy link on the security page to make it easy to create bug reports in the right state (i.e. with the committers only flag turned on). I encourage project teams (especially those working on runtime technology) to consider including a project-specific link for reporting vulnerabilities.

Note that it is our policy that all vulnerabilities eventually get disclosed, so issue privacy should be considered as short term state to give a project team an opportunity to get ahead of a vulnerability.

Google Summer of Code

From the Google Summer of Code Student Manual:

Google Summer of Code (GSoC) is a global program that matches students up with open source, free software and technology-related organizations to write code and get paid to do it! The organizations provide mentors who act as guides through the entire process, from learning about the community to contributing code. The idea is to get students involved in and familiar with the open source community and help them to put their summer break to good use.

Project teams that intend to participate in the Google Summer of Code should visit our Information Page, sign up for the soc-dev mailing list, and add student project ideas to the Ideas Page. You may also consider marking some of your bugs as helpwanted or bugday.

Note that we’re still in the mentoring organization application stage; we’ll let you know when it’s time to sigh up as a mentor or student.

Project Announcements

There are some reviews concluding on February 15, 2017:

We have several proposals open for community review:

Please add your comments either directly on the proposal or in the Proposals forum.

We run reviews ending on the first and third Wednesday of each month. Our next scheduled review dates are March 1, 2017 and March 15, 2017.

For more information about releases and reviews, please see the Eclipse Project Handbook.

Eclipse Foundation Projects Team at Eclipse Converge and Devoxx US

The Eclipse Foundation Projects Team will be at Eclipse Converge and Devoxx US in March. We’ll be there to answer your questions, and help you work through any process-related issues. We’ll be hanging out the Eclipse Foundation’s Booth.  Join us there!

devoxx_black_transparent400

Posted in Announcements, Community | Leave a comment

Eclipse Infrastructure Support for IP Due Due Diligence Type

The Eclipse Foundation’s Intellectual Property (IP) Policy was recently updated and we’re in the process of updating our processes and support infrastructure to accommodate the changes. With the updated IP Policy, we introduced the notion of Type A (license certified) and Type B (license certified, provenance checked, and scanned) due diligence types for third-party dependencies that projects can opt to adopt.

With Type A, we assert only that third-party content is license compatible with a project. For Type B third-party content, the Eclipse Foundation’s IP Team invests considerable effort to also assert that the provenance is clear and that the code has been scanned to ensure that it is clear of all sorts of potential issues (e.g. copyright or license violations). The type of due diligence applies at the release level. That is, a project team can decide the level of scrutiny that they’d like to apply on a release-by-release basis.

For more background, please review License Certification Due Diligence and What’s Your (IP Due Diligence) Type?

By default all new projects at the Eclipse Foundation now start configured to use Type A by default. We envision that many project teams will eventually employ a hybrid solution where they have many Type A releases with period Type B releases.

The default due diligence type is recorded in the project’s metadata, stored in the Project Management Infrastructure (PMI). Any project committer or project lead can navigate to their project page, and click the “Edit” button to access project metadata.

project_edit.png

In the section titled “The Basics” (near the bottom), there’s a place where the project team can specify the default due diligence type for the project (it’s reported on the Governance page). If nothing is specified, Type B is assumed. Specifying the value at the project level is basically a way for the project team to make a statement that their releases tend to employ a certain type of due diligence for third-party content.

project_dd_type.png

Project teams can also specify the due diligence type for third-party content in the release record. Again, a project committer or project lead can navigate to a release record page, and click “Edit” to gain access to the release metadata.

release_dd_type.png

As for projects, the metadata for IP due diligence type is found in the section titled “The Basics”. The field’s description is not exactly correct: if not specified in the release record, our processes all assume the value specified in the project metadata. We’ll fix this.

When the time comes to create a request to the Eclipse Foundation’s IP Team to review a contribution (a contribution questionnaire, or CQ), committers will see an extra question on the page for third-party content.

create_cq.png

As an aside, committers that have tried to use our legacy system for requesting IP reviews (the Eclipse Developer Portal) will have noticed that we’re now redirecting those requests to the PMI-based implementation. Project committers will find a direct link to this implementation under the Committer Tools block on their project’s PMI page.

We’ve added an extra field to the record that gets created in our IP tracking system (IPZilla), Type, that will be set to Type_A or Type_B (in some cases, it may be empty, “–“). We’ve also added a new statelicense_certified, that indicates that the licenses has been checked and the content can be used by the project in any Type A release.

Any content that is approved can be assumed to also be license certified.

There are many other questions that need to be answered, especially with regard to IP Logs, mixing IP due diligence types, and branding downloads. I’ll try to address these topics and more in posts over the next few days.

Posted in Intellectual Property | 1 Comment

What’s Your (IP Due Diligence) Type?

Long-time Eclipse Committer, Ian Bull initiated a interesting short chat on Twitter yesterday about one big challenge when it comes to intellectual property (IP) management. Ian asked about the implications of somebody forking an open source project, changing the license in that fork, and then distributing the work under that new license.

We can only surmise why somebody might do this (at least in the hypothetical case), but my optimistic nature tends toward assuming that this sort of thing isn’t done maliciously. But, frankly, this sort of thing does happen and the implications are the same regardless of intent.

Even-longer-time Eclipse Committer, Doug Schaefer offered an answer.

The important takeaway is that changing a license on intellectual property that you don’t own is probably bad, and everybody who touches it will potentially be impacted (e.g. potentially face litigation). I say “probably bad”, because some licenses actually permit relicensing.

Intellectual property management is hard.

The Eclipse Foundation has a dedicated team of intellectual property analysts that do the hard work on behalf of our open source project teams. The IP Team performs analysis on the project code that will be maintained by the project and for third-party libraries that are maintained elsewhere. It’s worth noting that there is no such thing as zero risk; the Eclipse IP Team’s work is concerned with minimising, understanding, and documenting risk. When they reject a contribution or third-party library use request, they do so to benefit of the project team, adopters of the project code, and everybody downstream.

In yesterday’s post, I introduced the notion of Type A (license certified), or Type B (license certified, provenance checked, and scanned). The scanned part of Type B due diligence includes—among many other things—the detection of the sort of relicensing that Ian asked about.

Since we don’t engage in the same sort of deep dive into the code, we wouldn’t detect this sort of thing with the license certification process that goes with Type A. That is, of course, not to say that it’s okay to use inappropriately relicensed third-party code in a Type A release, we just wouldn’t detect it via Type A license certification due diligence. This suggests a heightened risk associated with Type A over Type B to consider.

Type B due diligence is more resource intensive and so potentially takes a long time to complete. One of the great benefits of Type A, is that the analysis is generally faster, enabling a project team to get releases out quickly. For this reason, I envision a combination approach (some Type A releases mixed with less frequent Type B releases) to be appealing to many project teams.

So project teams needs to decide for themselves and for their downstream consumers, what sort of due diligence they require. I’ve already been a part of a handful of these discussions and am more than happy to participate in more. Project teams: you know how to find me.

It’s worth noting that Eclipse Foundation’s IP Team still does more due diligence review with Type A analysis than any other open source software foundation and many commercial organisations. If a committer suspects that shenanigans may be afoot, they can ask the IP Team to engage in a deeper review (a Type A project release can include Type B approved artifacts).

April wrapped up the Twitter conversation nicely.

Indeed. Kudos to the Eclipse Intellectual Property Team.

If you want to discuss the differences between the types of due diligence, our implementation of the Eclipse IP Policy changes, or anything else, I’ll be at Eclipse Converge and Devoxx US. Register today.

Eclipse Converge

Posted in Intellectual Property, Uncategorized | 2 Comments

License Certification Due Diligence

With the changes in the Eclipse Intellectual Property (IP) Policy made in 2016, the Eclipse Foundation now offers two types of IP Due Diligence for the third-party software used by a project. Our Type A Due Diligence involves a license certification only and our Type B Due Diligence provides our traditional license certification, provenance check, and code scan for various sorts of anomalies. I’m excited by this development at least in part because it will help new projects get up to speed more quickly than they could have in the past.

Prior to this change, project teams would have to wait until the full application of what we now call Type B Due Diligence was complete before issuing a release. Now, a project team can opt to push out a Type A release after having all of their third-party libraries license certified.

A project team can decide what level of IP Due Diligence they require for each release. Hypothetically, a project team could opt to make several Type A releases followed by a Type B release, and then switch back. I can foresee this being something that project teams that need to engage in short release cycles will do.

We’ve solicited a few existing projects to try out the new IP Due Diligence type and have already approved a handful of third-party libraries as Type A. The EMO has also started assuming that all new projects use Type A (license certification) by default. As we move forward, we expect that all new projects will employ Type A Due Diligence for all incubation releases and then decide whether or not to switch to Type B (license certification, provenance check, and code scan) for their graduation. There is, of course, no specific requirement to switch at graduation or ever, but we’re going to encourage project teams to defer the decision of whether or not to switch from Type A until that point.

After graduation, project teams can decide what they want to do. We foresee at least some project teams opting to issue regular multiple Type A releases along with an annual Type B release (at this point in the time, there is no specific requirement to be Type A or Type B to participate in the simultaneous release).

We’ve started rolling out some changes to the infrastructure to support this update to the IP Due Diligence process. I’ll introduce those changes in my next post.

Update: Based on some Tweets, I changed my intended topic for the next post. Please see What’s Your (IP Due Diligence) Type?

Eclipse Converge

Posted in Intellectual Property | 2 Comments

Sleepwalk, So Fast Asleep

One of my favourite literary quotes comes from Canadian author, Hugh MacLennan.

“But that night as I drove back to Montreal, I at least discovered this: that there is no simple explanation for anything important any of us do, and that the human tragedy, or the human irony, consists in the necessity of living with the consequences of actions performed under the pressure of compulsions so obscure we do not and cannot understand them.”
― Hugh MacLennan, The Watch that Ends the Night

I think that this describes our industry pretty well…

This passage was the basis for the the song Courage by the Tragically Hip, arguably one of (if not the) most Canadian bands in existence (seriously, the entire country basically shut down for the night while a third of the population dropped everything to gather in groups to watch their last concert on August 20/2016).

Anyway, this is the long way of saying that I have no recollection of why I selected the username wayninator for my primary Google account many years ago. But I’m apparently stuck with it.

Posted in Cranky Old Programmer, Other | 2 Comments