Lastest Articles

Koodies 1.5: Execute Commands, Scripts & Containers the Kotlin-way

Koodies 1.5 is out now and allows you to run command lines and shell scripts, either on your host or in a Docker container of your choice, synchronously or asynchronously, optionally logging and even while interacting with the created process—all of this with zero boilerplate.

Background

The Java’s Development Kit allows developers to execute native OS processes using Runtime.exec since its first release. As of Java 1.5/5.0 the newly introduced ProcessBuilder rendered process handling easier; as did subsequent feature additions like ProcessHandle.

Apache Commons Exec was somewhat a breakthrough as it took away one of the biggest challenges programmers face with processes—reading their standard output and error. Furthermore Apache Commons Exec supports parameter substitution and helps coping with concurrency (e.g. DefaultExecutor, ExecuteWatchdog). Yet, a couple of everyday tasks are still not easily achievable.

ZT Process Executor is a process executor that makes a lot of things better. It has meta logging, which greatly helps at debugging and provides “one-liners” (that are rather “one-line-ishs” because of line length and exception handling boilerplate), as one can see in the following example:

String output;
boolean success = false;
try {
    output = new ProcessExecutor().command("java", "-version")
                  .readOutput(true).exitValues(3)
                  .execute().outputUTF8();
    success = true;
} catch (InvalidExitValueException e) {
    System.out.println("Process terminated with " + e.getExitValue());
    output = e.getResult().outputUTF8();
}

Unfortunately when it comes to script, no simple solutions seem to exist at all. Also libraries taking advantage of Kotlin features are not known.

This is where Koodies Exec jumps in.
The following snippet provides the same functionality as the code from:

val exec = CommandLine("java", "-version").exec()
     .apply { if(state is Failed) println(exitCode) }
val (output, success) = exec.io.out to exec.successful

Features

Execute Command Lines on Host

CommandLine("printenv", "HOME")
  .exec() // .exec.logging() // .exec.processing { io -> … }

Execute Shell Scripts on Host

ShellScript {
  "printenv | grep HOME | perl -pe 's/.*?HOME=//'"
}.exec() // .exec.logging() // .exec.processing { io -> … }

Execute Command Lines in Docker Container

CommandLine("printenv", "HOME")
  .dockerized{ "ubuntu" }
  .exec() // .exec.logging() // .exec.processing { io -> … }

Execute Shell Scripts in Docker Container

ShellScript {
  "printenv | grep HOME | perl -pe 's/.*?HOME=//'"
}.dockerized{ "ubuntu" }
 .exec() // .exec.logging() // .exec.processing { io -> … }

Logging and Interaction

As you can see in the above examples, three execution variants exist:

▶️ executing-only

ShellScript { … }.exec()

📝 logging

ShellScript { … }.exec.logging()

If things go wrong, it’s also logged:

Process {PID} terminated with exit code {…}
➜ A dump has been written to:
  - {WorkDir}/koodies.dump.{}.log
  - {WorkDir}/koodies.dump.{}.ansi-removed.log
➜ The last 10 lines are:
  {…}
  3
  2
  1
  Boom!

🧠 processing

ShellScript { … }.exec.processing { io -> 
  doSomething(io)
}

io is typed; simply use io is IO.Output to filter out errors and meta information.

Asynchronous Execution

Processes are executed synchronously by default. Simply add async to the exec call to run the process asynchronously:

  • ShellScript { … }.exec.async()
  • ShellScript { … }.exec.async.logging()
  • ShellScript { … }.exec.async.processing { io -> doSomething(io) }

Automatically Captured I/O

Whatever variant you choose, life-cycle events, sent input, the process’s output and errors are stored for you:

  • CommandLine(…).exec().io
  • CommandLine(…).exec().io.output
  • CommandLine(…).exec().io.error.ansiRemoved

Typed (Exit) State

  • Access the state with state, which is either an instance of RunningExited (with the sub states Succeeded and Failed) or Excepted.
  • All states print nicely and provide a copy of all logged I/O, and state-dependent information such as the exit code.
  • By default, processes are killed on VM shutdown, which can be configured.
  • Life-cycle callbacks can be registered.

Ready to run Docker commands

with(tempDir()) {
    SvgFile.copyTo(resolve("koodies.svg"))

    // convert SVG to PNG using command line-style docker command
    docker("minidocks/librsvg", "-z", 5, "--output", "koodies.png", "koodies.svg")

    // convert PNG to ASCII art using shell script-style docker command
    docker("rafib/awesome-cli-binaries", logger = null) {
        """
           /opt/bin/chafa -c full -w 9 koodies.png
        """
    }.io.output.ansiKept.let { println(it.resetLines()) }
}

Output

&kyTTTTTTTTTTTTTTTTTTTTuvvvvvvvvvvvvvvvvvvvvvvvv\.
RR&kyTTTTTTTTTTTTTTTTTvvvvvvvvvvvvvvvvvvvvvvvv\.
BBRR&kyTTTTTTTTTTTTTvvvvvvvvvvvvvvvvvvvvvvvv\.
BBBBRR&kyTTTTTTTTTvvvvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBRR&kyTTTTTvvvvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBBBRR&kyTx}vvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBBBBBRZT}vvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBBBBBQxvvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBBB&xvvvvvvvvvvvvvvvvvvvvvv\.
BBBBBBZzvvvvvvvvvvvvvvvvvvvvvv\.
BBBBZuvvvvvvvvvvvvvvvvvvvvvv▗▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
BBZTvvvvvvvvvvvvvvvvvvvvvv\.▝▜MMMMMMMMMMMMMMMMMMMM
R3vvvvvvvvvvvvvvvvvvvvvv\. .▝▜MMMMMMMMMMMMMMMMMM
vvvvvvvvvvvvvvvvvvvvvv\. .▝▜MMMMMMMMMMMMMMMM
vvvvvvvvvvvvvvvvvvvv\. .▝▜MMMMMMMMMMMMMM
uxvvvvvvvvvvvvvvvvz3x_ ▝▜MMMMMMMMMMMM
▁3uxvvvvvvvvvvvv▁▅&▆▂gx` ▝▜MMMMMMMMMM
Z▅▁3uxvvvvvvvvz▆WWRZ&▆▂gv. `▀WMMMMMMMM
WR&▄▁3uxvvvvvuk▀BWWWRZ&▆▂gv. .\vvz▀WMMMMMM
WWWRZ▅▁3ux▁▂Zg33k▀BWWWRZ&▆▂g}. .\vvvvvvz▀WMM0W
000WWRZ▅▃▆MM▆▂Zg33k▀BWWWRZ&▆▂g}. .\vvvvvvvvvvx▀BBR
00000WMMMMMMMM▆▂Zg33k▀BWWWRZ&▆▂yxxvvvvvvvvvvvvvx▝▀
0000MMMMMMMMMMMM▆▂Zg33k▀BWWWRZ▆▆▂gTxvvvvvvvvvvvvvx
00MMMMMMMMMMMMMMMM▆▂Zg33k▀BWWWRZ&▆▂gTxvvvvvvvvvvvv
MMMMMMMMMMMMMMMMMMMM▆▂Zg33g▀BWWWRZ&▆▂gTxvvvvvvvvvv
  • All docker commands (dockerubuntubusyboxcurldownload, …) use the path in the receiver to
    • set the working directory of both the host command and the docker container
    • map the host working directory to the container’s working directory,
    • that is, all files of that directory are equally available in your container instance.
  • Low-level docker commands: startrunstopkillremovesearchimageps
  • Object-oriented design
    • DockerengineRunninginfoimagescontainerssearchexec
    • DockerImagelistisPulledpulltagsOnDockerHub
    • DockerContainerstartstopstatekillremove
  • See ExecutionIntegrationTest.kt and Docker.kt for more examples.

Installation

Koodies is a Kotlin Multiplatform Library, using the hierarchical project structure, a minimal set of dependencies, and is hosted on GitHub with releases provided on Maven Central.

Gradle

implementation("com.bkahlert:koodies:1.4.2")

Maven

<dependency>
  <groupId>com.bkahlert</groupId>
  <artifactId>koodies</artifactId>
  <version>1.5.0</version>
</dependency>

copy-out

copying out the complete file tree from an ext4 formatted partition right on macOS

Mac and Windows users share one thing. Accessing an ext4 formatted partition is virtually impossible. The problem is serious enough so that IoT distributions like Raspberry Pi OS offer workarounds that give Mac and Windows users at least a few opportunities to customize their flashed memory cards before booting the IoT device itself.

Since tools like guestfish are able to mound ext4 partitions why not run such a tool using Docker and access the files this way? I saw the one or the other try but getting it running was a nightmare. Already due to the fact that guestfish logs all of its output as an error which is not forward by Docker. Therefor I developed a script on my own to see if things can’t be easier. And I suppose they can as you can convince yourself on my new gist on copying-out files from ext4 partition resp. any img guestfish can access.

Koodies Banner

Koodies

RANDOM SOFTWARE GOODIES

A few days ago—while working on a larger project—I released my first multi-platform software library for the awesome Kotlin language.

Koodies is a random selection of goodies to make a Kotlin developer’s life even easier. Features range from various generic builders and special purpose builders (like for shell scripts or Docker command lines) over Java NIO 2 extension functions to various utilities for easier dealing with time, graphics, Unicode, etc.

Kaomojis.`(#-_-)o´・━・・━・━━・━☆`

I can be downloaded from Bintray and is hosted on GitHub.

Sketch Notes on a Scrum Master Training

Mastering Scrum does not lose any of its importance for me. Therefor I decided to go for a Scrum Master Certificate months ago. Due to the Corona pandemic my plans relativized, since the next trainings would be online and I wanted in person. But last week it happened: The training with a small group of other finally took place 😁

It was a 3 day training by Timon Fiddike and Anton Skornyakov from agile.coach. Those guys did a great job at coping with the range of experience the participants had. The most unexpected thing for me was—and I guess that was the key driver for that highly valuable training—they actually implemented Scrum elements during the training itself. Having dailies, a retrospective, and most importantly: they listened to what the group wished for. And: they checked if their “corrections” had the intended effect. (That is something so important and far too rarely done.)

Oh and one last thing: I’m so happy I did it. 10min before leaving my apartment I saw some empty sketch books laying around. Unused. So—although I never did that before—I grabbed one and documented everything I considered relevant to myself in it (which is the excuse for being too late on the first day). As I got unexpectedly positive feedback on it, I decided to share it with you. I hope you enjoy!

First Retrospectives

A couple of weeks ago I started my visual facilitator career and started introducing respectively renovating an existing Scrum board to put my newly acquired knowledge into practice.

Now it was time to combine my new skills with my goal to become more proficient in the Scrum process. As I was always a team player who’s goal was to bring people together and treat each other with the most respect possible, it was just natural to volunteer organising retrospectives.

Continue reading…

Online Grid Generator for Photoshop

For those who layout their websites with grids it is very common to use grid systems like grid the 960 GRID SYSTEM.

The bugging thing about it is always to prepare a corresponding photoshop file. But now the GRID CALCULATOR comes to help!

Simply enter your grid definition and the online tool creates a photoshop script you only need to double click. Et voilà: the corresponding guides are created directly in your photoshop file.

CSS preprocessing

I always thought that CSS lacks in expressiveness which is far below the one of XPath. For example you can’t select elements based on their child elements. How nice would it be if you could format anker tags if and only if they contain a single image tag.
For this to work you would simply use the following XPath expression:

//a[count(img)=1]

Continue reading…

Color Wizard

Tired of bright red, green or blue?
Then try out the Color Wizard on colorsontheweb.com.

Given one color the tool gives you 3 variations (hue, saturation and tint&shade) as much as 6 corresponding colors (monochromatic, analogous, triadic, tetradic, complimentary and split complimentary). Give it a shot!

WordPress Web Service

You are an enthusiastic WordPress user?
You want to implement a Flash based WordPress front end / theme?
You want to use WordPress as a back end for you software?

My new WordPress Web Service plugin is your solution!

WPWS allows you to access your WordPress installation via WSDL/SOAP. In contrast to classical RESTful services WSDL/SOAP is type safe meaning that you always get what you expect. RESTful services can change their normally not explicitly defined API from one day to the other and … BANG! You web application crashes.

After you have installed WPWS you can simply open your blog and append ?/wpws to your blog’s url.
My plugin for example is accessible through bkahlert.com/blog/?/wpws.
The following screenshot shows what the plugin’s homepage looks like.

Continue reading…

generic ISOCUBE

Zachary Johnson and Paul Hayes showed some really nice examples of the CSS 2D transformation capabilites (isocube, 3D isocube using 2D CSS).

What I missed was the math behind it. Both (and every other isocube I could find on the net) statically positioned the three rectangles that form the isocube.

So what I did was to implement a generic isocube that renders based on the dimensions w, h and a degree α. The sources can be downloaded as a jQuery Plugin.

The math is mainly based on the Pythagorean theorem. In order to geometrically verify some hypotheses that came up I used Cinderella.

Continue reading…

Osama Bin Laden killed live on a news broadcast! watch the video: …

A worm has started to spread over Facebook promising
the user a video where the killed Osama Bin Laden can be seen.

The link is:

http://www.facebook.com/pages/Osama-Bin-Laden-Killed-Live-on-Video/201198676585608?sk=app_190322544333196&71029

On the corresponding Facebook profile you are asked to copy a javascript line following line into the browser’s location bar which reloads a JavaScript file named “bin.js” from a doubtful source.

Continue reading…

Client-Server Communication for Humanoid Robots

Last month I finished my Bachelor thesis with the German title Client-Server-Kommunikation für humanoide Roboter (engl. “Client-Server Communication for Humanoid Robots”).

Within the framework of this work I implemented an application based on the Eclipse Rich Client Platform (RCP) that allows for the communication with humanoid soccer playing robots – namely FUmanoids – developed at the Freie Universität Berlin.

Continue reading…