Technology Behind the new Eclipse Project Handbook

I’ve been asked about the technology used in the new Eclipse Project Handbook.

Editing the Eclipse Project Handbook

The content itself is specified in Asciidoc format and rendered into HTML using Asciidoctor. I started off using Asciidoc for rendering, but changed after I ran into a couple of bugs and problematic licensing.

The content is represented as a collection of documents that are combined via master documents for each of the separate forges; I decided to do this to make the content more manageable and easier for contribution, and make it more easy to mix and match content. Forge-specific parameters are specified in the master document.

Many of the images are specified using Graphviz and rendered as hierarchical images using dot. For this style of graphic, the very simple text format permitted by Graphviz means that I have very good version control for these images. I also really hate pushing pixels around.

To render the documents as PDF, I’m using Asciidoctor to render the content in Docbook format and Apache FOP to take it the rest of the way. I think that my challenges with the graphics in the rendered PDFs are a issue with FOP, but haven’t quite concluded my investigation. As I mentioned yesterday,  I consider the PDF versions a nice-to-have, so I haven’t made sorting this out a very high priority.

I’m pulling all the pieces together using a make build. Given the nature of the project, this was, frankly, the easiest solution. It works well and is easy to maintain (there are, however, a few hard-coded file paths that I’m not at happy about).

The sources are all in the /projects Git repository if you’d like to take a closer look. I’m thinking that I will move these to a separate repository to make it easier for folks who want to contribute patches (so that you don’t need to clone all the other junk stuff in that repository).

Naturally, I’m using Eclipse to edit content and invoke the makefile as an external tool. Adding Mylyn Docs Wikitext support for Asciidoc to my Eclipse configuration is on my to-do list this month.

I’m pulling all of the tools from the Fedora software repository.

This entry was posted in Other. Bookmark the permalink.

3 Responses to Technology Behind the new Eclipse Project Handbook

  1. jmini says:

    Thank you for this nice article.

    Separation of sources and publication is important.

    When you use GitHub pages, the separation is done with branches into the same repository. As an example, take the “asciidoctor.org” repository. There is a “master” branch containing the sources and a “gh-pages” branch containing the published content.
    https://github.com/asciidoctor/asciidoctor.org

    In your case, separation in 2 repositories makes more sense, because if I understood you correctly, you create from a single source in the first repository multiple artifacts (html pages) for multiple websistes.

    Since you are using the Asciidoc Format, I recommend you to host this sources on a repo hosted on GitHub. This way you can offer:
    * An “edit me on GitHub” button on your final HTML pages (Xtext does it. Another example is Microsoft. They do it too for the Azure documentation).
    * For occasional users, they will get at GitHub an online Editor with Syntax highlighting and a preview tab. No need for them to check out the complete repository. (The new live edit feature of Gerrit is not as convenient when it comes to editing adoc files)
    In my opinion using the GitHub interface is important: it offers a user experience comparable to a wiki (with all the benefits of a Git repo in terms of commits, review, local edits…).

    Another important point is your build script. Yes a makefile can be good enough, but it requires some installation on a local machine (and on the build server if you do continuous integration – which I also recommend). I am not a maven fan boy, but here is what I appreciate with Maven:
    * It is an industry state of the art component.
    * It is part the CBI effort at the Eclipse foundation.
    * When working with Asciidoctor, it is the only prerequisite (asciidoctor plugins are hosted at maven central).
    * It makes the build reproducible on every machine.

    Finally I wanted to add a word on continuous deployment:
    Because we have this convenient setup: “publishing to a webserver == commit in a git repository”, we can imagine that the final step on the continuous integration build server is to do this commit in the appropriate repo.
    This mechanism has been indirectly discussed in bug 463327 (as I explained, Jekyll is in my opinion complicated to set up and I would prefer something maven based.)

    With the current setup (mix of sources and published artifact in the same repo) you have some border effects: adoc files and build files are published on the web server:
    https://www.eclipse.org/projects/handbook/source/config.adoc
    https://www.eclipse.org/projects/handbook/source/makefile
    This is not really nice…

    By the way if you are interested in collecting “Asciidoctor best practices” knowledge, please tell me. I have described a lot of the choices we made for the Eclipse Scout project (still work in progress – we do not have the Continous Deployment part yet) on the scout blog, scout forum and Bugzilla. I will try to find time to help the Golo project for creating their Eclipse Help based on their Asciidoc sources, and I have submitted a Talk to ECE 2015:
    https://www.eclipsecon.org/europe2015/session/writing-documentation-asciidoctor

    • waynebeaton says:

      Thanks for the input. This is helpful.

      For the first round, I was trying very hard to focus on just getting the content done and avoided getting bogged down with implementation details. I intend keep this on EF infrastructure, but will explore Maven based builds and CI options.

  2. Great stuff! I’d like to add a patch that uses the Mylyn Docs tools to produce an EPUB that can be opened on reading systems such as iBooks, Calibre and Readium.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s