Write Once, Build Anywhere

Dave Jarvis, June 29 2020 RSS Feed

Cross-compiling self-contained Java desktop application launchers.

Backstory

Java’s promise of write once, run anywhere still resonates with me, perhaps because I cut my teeth on JDK 1.0.2b. It was amazing to write cross-platform networking applications without concern for variations in POSIX implementations, or having to worry about nuances between C compilers, or even needing to grok htons. Someone else could wrestle with #ifdefs, shielding others by high-level, standard abstractions. Although Smalltalk had concocted platform-independence punch in 1983, it wasn’t until 1996 that Java’s virtual machine (VM) and subsequent OEM distributions made drinking the mantra of cross-platform development an easy reality.

Strategically—and selfishly—Microsoft smote that future with a sledgehammer: it moved to embrace and extend Java to achieve vendor lock-in, splintering the platform.

Developers can no longer rely on systems having a Java VM available. This implies that end-users must download (and install) two independent programs: a suitable VM and a Java archive. It also means that developers now have choices ahead of them that were once bygone worries. Choices that burden end-users and developers alike. Asking people to download a VM for their operating system and CPU architecture is a barrier to product adoption. What’s worse is that some VMs now support “full” Java while others don’t support JavaFX modules at all.

We’ll address that last point later.

In late 2019, Kevin Rushforth, of Oracle’s Java Team, presented a solution to the packaging problem: jpackage and jlink. During the presentation he listed some of the tooling’s shortcomings, which include:

Let’s not mince words: Java developers could once build applications on their preferred operating system and deploy anywhere; now, they must build installers for every individual platform they intend to target on those very systems.

Was that the punch-line to a two-decade lead up?

Requiring developers to have at least two commercial operating systems (Windows, MacOS X), even if emulated inside of a virtualized container—as opposed to using physical hardware—for a platform-independent programming language is an ironic barrier to entry. Especially for people bereft of bankroll.

Yes, those tools—alongside modules—help produce minimal executables. Who cares? What they don’t do is make it possible to create a set of native, standalone launcher binaries from a single build machine. Let’s fix that by pouring over the big splashes of how to cross-compile application launchers for Java on one build machine. We’ll use Linux because it’s free.

Software Requirements

You will need:

Install git and wget using the package manager for your Linux distro.

Install Warp Packer

The installation instructions for Warp Packer are a bit buried. Install the software on a Linux system as follows (change the version number to suit):

mkdir -p $HOME/bin
cd $HOME/bin
wget -O warp-packer \
  https://github.com/dgiagio/warp/releases/download/v0.3.0/linux-x64.warp-packer
chmod +x warp-packer

If needed, include $HOME/bin in the PATH environment variable by adding the following line to $HOME/.bashrc:

export PATH="$PATH:$HOME/bin"

Apply the new environment settings by opening a new terminal or source’ing the .bashrc file directly.

Warp Packer is installed.

Install Build Template

Copy, move, or download build-template into $HOME/bin.

The build template is installed.

JavaFX

Before we get into building a Java application, a little more history is warranted. Originally, the Abstract Window Toolkit (AWT) came bundled with Java, for the same reasons that having a bundled networking API was useful. The AWT lacked aesthetic, both in appearance and as a library. Swing, the AWT’s successor, made strides to address both, but fell short. JavaFX, which was also initially bundled with Java (versions 8 to 10), is a modern graphical user interface (GUI) library for developing desktop applications, as well as a superb replacement for Swing.

It has been argued that JavaFX should never have been bundled with Java. Nevertheless, the decision to un-bundle JavaFX from Java 11 onwards has led to Java projects that will be stuck with old Java versions for years to come—possibly leading to their abandonment—because it takes a lot of effort to migrate a build process to support these new, un-bundled Java Development Kits (JDKs).

Part of that effort entails re-bundling JavaFX. There are a few ways this can be accomplished:

It’s almost a Hobson’s choice because we have to create launcher binaries for each target platform anyway. Meaning bundling all of the target platform libraries into each native launcher is wasteful; however, bundling them in a single JAR file for those who have a Java VM installed already may be somewhat useful.

Java has introduced the concept of modules, looking to simplify the build process. Sometimes, though, modules cannot be included because of outdated dependencies. Problems can arise when running jlink against old dependencies. Compounding the issue is when the projects have been sundowned or have long release schedules. Nevertheless, for our purposes, we’ll want to download a version of OpenJDK that supports JavaFX.

With that background in mind, let’s go cross-compile some binaries.

Java Application

We’ll create native application launchers for Scrivenvar, a JavaFX-based text editor that I’ve been developing.

Dive in by cloning the latest version as follows:

mkdir -p $HOME/dev
cd $HOME/dev
git clone https://github.com/DaveJarvis/scrivenvar.git
cd scrivenvar

The application is downloaded.

Build Scripts

Let’s review the scripts that help build the application, which include:

Separating the installer script’s responsibilities from build.gradle is not necessary, technically. When running third-party executables (e.g., warp-packer), my first inclination is to use a shell script, rather than gradle’s exec.

The installer script invokes gradle when requested.

Gradle Script

Open up build.gradle to review a few pertinent code blocks.

Target Operating System

The first snippet determines what native JavaFX libraries (i.e., target platform code) to include within the JAR file:

String[] os = ["win", "mac", "linux"]

if (project.hasProperty('targetOs')) {
  if ("windows".equals(targetOs)) {
    os = ["win"]
  }
  else {
    os = [targetOs]
  }
}

Unless instructed otherwise, the build script will create a JAR file that includes JavaFX binaries for Windows, Linux, and MacOS embedded. We can target a specific platform by using the -P command line option when building, such as:

gradle clean build jar -PtargetOs=linux

There’s a catch here in that win and windows are different names. The installer script needs to use windows because that’s part of the JDK file name being downloaded. Arguably, we could move that logic into the installer script, but the point is that the JDK filename and JavaFX refer to Windows using different terminology. The difference must be captured somwhere.

JavaFX Dependencies

The second snippet tells the build system to include JavaFX for a given set of operating systems, as follows:

def fx = ['controls', 'graphics', 'fxml', 'swing']

fx.each { fxitem ->
  os.each { ositem ->
    runtimeOnly "org.openjfx:javafx-${fxitem}:${javafx.version}:${ositem}"
  }
}

The application uses JavaFX controls, graphics, fxml, and notice that it also lists a JavaFX-Swing integration. Using .each is a tidy way of not having to maintain a list of JavaFX-specific imports for each operating system, should either or both need to change.

JavaFX Exclusions

Of lesser importance are exclude groups:

implementation('de.jensd:fontawesomefx-fontawesome:4.7.0-11') {
  exclude group: 'org.openjfx'
}

Without excluding JavaFX from certain JavaFX-related dependencies, the build will fail. Unfortunately, the build fails in such a way that the developer must investigate what package caused the conflict. Ideally, the build failure would pinpoint any collisions for the developer.

Run Application

The last point of note are the following lines:

applicationDefaultJvmArgs = [
    "--add-opens=javafx.controls/javafx.scene.control=ALL-UNNAMED",
    "--add-opens=javafx.controls/javafx.scene.control.skin=ALL-UNNAMED",
    "--add-opens=javafx.graphics/com.sun.javafx.css=ALL-UNNAMED",
]

These lines address permissions issues encountered when running the application using gradle, such as:

gradle clean build run

Now that we understand how this build script works, let’s move on to the installer shell script.

Installer Script

Looking at the big picture, the installer script performs the following steps to create a native launcher binary:

  1. Configure platform-specific parameters for extraction.
  2. Rebuild the überjar for the target platform.
  3. Clean out the distribution directory for a fresh build.
  4. Extract the target JDK into the distribution directory.
  5. Create a platform-dependent launch script to run the application.
  6. Copy the überjar into the distribution directory.
  7. Wrap the distribution into a standalone binary.

These high-level steps can be found in the execute() function.

Let’s take a closer look at how the script works.

Build Template

The build-template script is described in depth in the first three parts of my Typesetting Markdown series. We reuse the script here because it simplifies writing user-friendly shell scripts.

JDK Extraction

Downloading and extracting a requisite JDK is pivotal. JDKs can be downloaded from a few places, but most of them have unworkable drawbacks. Here are my findings:

BellSoft, and the other OpenJDK suppliers, could be improved by offering zip archives for all platforms. We’ll see why in a moment.

The utile_extract_jre function generates a download URL for a particular target platform based on a number of parameters:

local -r url_jdk="https://download.bell-sw.com/java/${jre_version}/bellsoft-jre${jre_version}-${ARG_JRE_OS}-${ARG_JRE_ARCH}-full.${ARCHIVE_EXT}"

Had all the archives been offered in zip format, we wouldn’t need to capture this difference as ${ARCHIVE_EXT} anywhere. Computers abhor exceptions. The more general you can make systems, the less work it is to automate, and less arduous to maintain. There’s this prevalent assumption that Windows systems use zip files and Unix systems use tar.*z files. It’s strange because unzip has been available for Unix systems since 1989 (with updates into 2009). Yes, tar.*z files compress smaller. So what? The difference in size is negligible. Mini-rant over. Forest and trees, forest and trees.

The script downloads the JDK into /tmp with a simplified name based on the Java version, target platform, and architecture. On subsequent builds, if that archive exists, it is re-used rather than re-downloaded.

Launch Scripts

For the native binary bundle to work, it needs to know how to launch the application. Using Java, this often looks something like:

java -jar application.jar

With a native wrapper, it’s conceptually no different, but in practice we need to take into consideration:

These differences are captured in the following functions:

The ARG_JRE_OS variable value, set in utile_configure_target(), determines what type of launch script is created.

Create Launcher

After the JAR file, launch script (e.g., run.sh or run.bat), and platform-specific JDK version are created inside the distribution directory, we can compile a native wrapper.

Before doing so, another subtle difference must be captured. The Warp Packer uses x64 to indicate a 64-bit binary whereas the download filename uses amd64. Having resolved that last issue, the packer is called as follows:

warp-packer \
  --arch "${ARG_JRE_OS}-${ARG_JRE_ARCH}" \
  --input_dir "${ARG_DIR_DIST}" \
  --exec "${FILE_DIST_EXEC}" \
  --output "${APP_NAME}.${APP_EXTENSION}" > /dev/null

You may wish to remove the > /dev/null if you want to see the error messages. For the adventurous, you could redirect to a log file and display its contents depending on the exit value returned from Warp Packer.

The ${APP_NAME} is derived from a Java property file so that the application name can be changed from a single point in the code base. Migrating the install script into the build.gradle script would simplify determining the application name.

Building Binaries

All this machinery allows creation of platform-specific binaries quite simply:

./installer -V -o windows
./installer -V -o linux

Both invocations create standalone binary executables, with no installation required by application users: just download and run. The first generates an exe for Windows; the second a bin file for Linux platforms (only tested on one distro, buyer beware). MacOS remains an exercise for the reader.

Conclusion

The largest waves for building installation-free native binary application launchers for JavaFX have receded. To me, this approach keeps Java’s original tagline in mind, while eyeballing future continuous build processes with respect to desktop applications: write once, run anywhere (yes, or test everywhere).

In this post we covered:

Returning to the theme of history, the remainder of this post discusses how this blog post came into being.

Hard Sci-Fi

I love writing. Having to remember character names, monikers, physical attributes, story locations, event timelines, day names from dates, and so on is a chore. Character sheets come in a variety of forms: spreadsheets, notebooks, web sites, or even software. I started making a spreadsheet for a hard sci-fi novel when it dawned on me that I’m a software developer; I decided to use a YAML document instead.

Cross-platform Markdown Editor

There didn’t seem to be a cross-platform application out there that allowed users to open a hierarchically structured data definition document alongside the text being authored. Moreover, once a hierarchy is in place, have the ability to interpolate those definitions before embedding into the document.

Given that Java is my first language (followed closely by English), led me to seek out a bare-bones Java-based Markdown editor that I could extend with a panel for editing definitions. That is, provide a way to edit and inject document variables easily. One editor stood out: Markdown Writer FX.

Editor Software Architecture

Turns out, many such text editors (of Markdown, AsciiDoc, etc.) rehash the same formula, depicted as the topmost box (labelled “Today”) in the following diagram:

Software Architecture Diagram

A little more up-front effort, a more versatile editor is within reach: the red, bottommost box. In object-oriented terms, the bottom box represents the chain-of-responsibility design pattern. One processor receives a document, performs some transformation upon it, then passes the new version down the line until the final form is generated: an HTML document.

FX Everywhere

Markdown Writer FX has the following main components: RichTextFX, WebView, and Flowless. The Markdown editor is provided by RichTextFX, the HTML preview is rendered by WebView, and the scrollbars are provided by Flowless. Specifically, the VirtualizedScrollPane handles scrolling the WebView. There are a few drawbacks to this approach.

First, WebView is pretty much a complete web browser, adorned with CSS3 parsing and JavaScript execution. Markdown documents typically transform into rudimentary HTML documents that don’t need the full force of CSS3, nor any JavaScript.

Second, and rather brutal, the VirtualizedScrollPane doesn’t provide a simple API to get a handle to its scrollbars. This turned into a thorny problem when synchronizing the text editor and the preview panel scrollbars. Adding such a feature to that class may not be forthcoming because the maintainer has moved on.

Scroll Pains

Without a reference to VirtualizedScrollPane’s scrollbar, synchronizing the editor and preview panes seemed intractible. Without Flowless, the WebView component would no longer be usable. No great loss, though, because WebView is like drinking from an aquarium when sipping from a cup would do.

Replacing a web browser with an HTML viewer isn’t trivial. Ultimately, WebView was replaced with the following components:

The dependency on Apache Batik made the build process incompatible with jlink. Removing the dependency would mean that vector graphics no longer appear. Since the hard sci-fi novel uses vector graphics, I didn’t want to introduce an external step to first rasterize the images.

Besides jlink not being an option, I don’t have a Mac, don’t want one, and would rather not use MacOS X—much less purchase it. That goes double for AIX. Still, it would be nice to have the possibility of cross-compiling standalone binaries for MacOS X, AIX, and other platforms. Thanks to Warp Packer, this can be achieved.

Standalone Binary Launchers

A few early beta testers mentioned that adoption rates may be higher if there was a standalone binary that people could try. This blog post, having originated with my desire to use interpolated strings while writing, has culminated in the following application binaries:

Please add any issues you encounter to the issue tracker.

Contact

About the Author

My career has spanned tele- and radio communications, enterprise-level e-commerce solutions, finance, transportation, modernization projects in both health and education, and much more.

Delighted to discuss opportunities to work with revolutionary companies combatting climate change.