Kommons process execution: Kotlin and Docker

Kommons: Execute Commands, Scripts & Containers the Kotlin Way

May 13, 2021 Development Usability

The process APIs introduced in version 1.5 of the library now known as Kommons let you run command lines and shell scripts, either on your host or in a Docker container, synchronously or asynchronously, with optional logging and process interaction—all with zero boilerplate.

Update: These examples document the original 1.5 release, published as Koodies, and retain its package names and Maven coordinates. The Docker helpers shown below are no longer part of the current API. See the current Kommons documentation and Maven Central for current releases.

Background

The Java’s Development Kit allows developers to execute native OS processes using Runtime.exec since its first release. As of Java 1.5/5.0 the newly introduced ProcessBuilder rendered process handling easier; as did subsequent feature additions like ProcessHandle.

Apache Commons Exec was somewhat a breakthrough as it took away one of the biggest challenges programmers face with processes—reading their standard output and error. Furthermore Apache Commons Exec supports parameter substitution and helps coping with concurrency (e.g. DefaultExecutor, ExecuteWatchdog). Yet, a couple of everyday tasks are still not easily achievable.

ZT Process Executor is a process executor that makes a lot of things better. It has meta logging, which greatly helps at debugging and provides “one-liners” (that are rather “one-line-ishs” because of line length and exception handling boilerplate), as one can see in the following example:

String output;
boolean success = false;
try {
    output = new ProcessExecutor().command("java", "-version")
                  .readOutput(true).exitValues(3)
                  .execute().outputUTF8();
    success = true;
} catch (InvalidExitValueException e) {
    System.out.println("Process terminated with " + e.getExitValue());
    output = e.getResult().outputUTF8();
}

Unfortunately when it comes to script, no simple solutions seem to exist at all. Also libraries taking advantage of Kotlin features are not known.

This is where Kommons Exec jumps in.
The following snippet provides the same functionality as the code from:

val exec = CommandLine("java", "-version").exec()
     .apply { if(state is Failed) println(exitCode) }
val (output, success) = exec.io.out to exec.successful

Features

Execute Command Lines on Host

CommandLine("printenv", "HOME")
  .exec() // .exec.logging() // .exec.processing { io -> … }

Execute Shell Scripts on Host

ShellScript {
  "printenv | grep HOME | perl -pe 's/.*?HOME=//'"
}.exec() // .exec.logging() // .exec.processing { io -> … }

Execute Command Lines in Docker Container

CommandLine("printenv", "HOME")
  .dockerized{ "ubuntu" }
  .exec() // .exec.logging() // .exec.processing { io -> … }

Execute Shell Scripts in Docker Container

ShellScript {
  "printenv | grep HOME | perl -pe 's/.*?HOME=//'"
}.dockerized{ "ubuntu" }
 .exec() // .exec.logging() // .exec.processing { io -> … }

Logging and Interaction

As you can see in the above examples, three execution variants exist:

▶️ executing-only

ShellScript { … }.exec()

📝 logging

ShellScript { … }.exec.logging()

If things go wrong, it’s also logged:

Process {PID} terminated with exit code {…}
➜ A dump has been written to:
  - {WorkDir}/koodies.dump.{}.log
  - {WorkDir}/koodies.dump.{}.ansi-removed.log
➜ The last 10 lines are:
  {…}
  3
  2
  1
  Boom!

🧠 processing

ShellScript { … }.exec.processing { io -> 
  doSomething(io)
}

io is typed; simply use io is IO.Output to filter out errors and meta information.

Asynchronous Execution

Processes are executed synchronously by default. Simply add async to the exec call to run the process asynchronously:

  • ShellScript { … }.exec.async()
  • ShellScript { … }.exec.async.logging()
  • ShellScript { … }.exec.async.processing { io -> doSomething(io) }

Automatically Captured I/O

Whatever variant you choose, life-cycle events, sent input, the process’s output and errors are stored for you:

  • CommandLine(…).exec().io
  • CommandLine(…).exec().io.output
  • CommandLine(…).exec().io.error.ansiRemoved

Typed (Exit) State

  • Access the state with state, which is either an instance of Running, Exited (with the sub states Succeeded and Failed) or Excepted.
  • All states print nicely and provide a copy of all logged I/O, and state-dependent information such as the exit code.
  • By default, processes are killed on VM shutdown, which can be configured.
  • Life-cycle callbacks can be registered.

Ready to run Docker commands

with(tempDir()) {
    SvgFile.copyTo(resolve("koodies.svg"))

    // convert SVG to PNG using command line-style docker command
    docker("minidocks/librsvg", "-z", 5, "--output", "koodies.png", "koodies.svg")

    // convert PNG to ASCII art using shell script-style docker command
    docker("rafib/awesome-cli-binaries", logger = null) {
        """
           /opt/bin/chafa -c full -w 9 koodies.png
        """
    }.io.output.ansiKept.let { println(it.resetLines()) }
}

Output

&kyTTTTTTTTTTTTTTTTTTTTuvvvvvvvvvvvvvvvvvvvvvvvv\.
RR&kyTTTTTTTTTTTTTTTTTvvvvvvvvvvvvvvvvvvvvvvvv\.
BBRR&kyTTTTTTTTTTTTTvvvvvvvvvvvvvvvvvvvvvvvv\.
BBBBRR&kyTTTTTTTTTvvvvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBRR&kyTTTTTvvvvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBBBRR&kyTx}vvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBBBBBRZT}vvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBBBBBQxvvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBBB&xvvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBZzvvvvvvvvvvvvvvvvvvvvvv\.
BBBBZuvvvvvvvvvvvvvvvvvvvvvv▗▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
BBZTvvvvvvvvvvvvvvvvvvvvvv\.▝▜MMMMMMMMMMMMMMMMMMMM
R3vvvvvvvvvvvvvvvvvvvvvv\.   .▝▜MMMMMMMMMMMMMMMMMM
vvvvvvvvvvvvvvvvvvvvvv\.       .▝▜MMMMMMMMMMMMMMMM
vvvvvvvvvvvvvvvvvvvv\.           .▝▜MMMMMMMMMMMMMM
uxvvvvvvvvvvvvvvvvz3x_              ▝▜MMMMMMMMMMMM
▁3uxvvvvvvvvvvvv▁▅&▆▂gx`              ▝▜MMMMMMMMMM
Z▅▁3uxvvvvvvvvz▆WWRZ&▆▂gv.             `▀WMMMMMMMM
WR&▄▁3uxvvvvvuk▀BWWWRZ&▆▂gv.         .\vvz▀WMMMMMM
WWWRZ▅▁3ux▁▂Zg33k▀BWWWRZ&▆▂g}.     .\vvvvvvz▀WMM0W
000WWRZ▅▃▆MM▆▂Zg33k▀BWWWRZ&▆▂g}. .\vvvvvvvvvvx▀BBR
00000WMMMMMMMM▆▂Zg33k▀BWWWRZ&▆▂yxxvvvvvvvvvvvvvx▝▀
0000MMMMMMMMMMMM▆▂Zg33k▀BWWWRZ▆▆▂gTxvvvvvvvvvvvvvx
00MMMMMMMMMMMMMMMM▆▂Zg33k▀BWWWRZ&▆▂gTxvvvvvvvvvvvv
MMMMMMMMMMMMMMMMMMMM▆▂Zg33g▀BWWWRZ&▆▂gTxvvvvvvvvvv
  • All docker commands (docker, ubuntu, busybox, curl, download, …) use the path in the receiver to
    • set the working directory of both the host command and the docker container
    • map the host working directory to the container’s working directory,
    • that is, all files of that directory are equally available in your container instance.
  • Low-level docker commands: start, run, stop, kill, remove, search, image, ps
  • Object-oriented design
    • Docker: engineRunning, info, images, containers, search, exec
    • DockerImage: list, isPulled, pull, tagsOnDockerHub
    • DockerContainer: start, stop, state, kill, remove
  • See ExecutionIntegrationTest.kt and Docker.kt for more examples.

Installation

The 1.5 release is a Kotlin Multiplatform library using the hierarchical project structure and a minimal set of dependencies. Its source is archived in the Kommons repository on GitHub, and the original artifacts remain available on Maven Central.

Gradle

implementation("com.bkahlert:koodies:1.5.0")

Maven

<dependency>
  <groupId>com.bkahlert</groupId>
  <artifactId>koodies</artifactId>
  <version>1.5.0</version>
</dependency>

Tags: