Porting a Go CLI tool to Web Assembly

Publish date: Jun 21, 2023

Tags:

In this blog post I’ll show how to run a golang command line application locally with a web assembly (WASM) runtime.

Background

The current release candidate of go 1.21 adds an experimental port of the Web Assembly System Interface (WASI). While the go compiler could compile to WASM before, only the Javascript target was supported, which was targeting browsers.

WASI enables a WASM application to access operating-system features like files and sockets, while being independet of browsers. This allows to compile a go application to WASM and to run it with a WASM runtime like wasirun.

In theory, when your app is compiled to WASM/WASI you only need a WASM runtime on your target system. It’s similar to Java and the JVM - write once, run anywhere. WASM is supported by languages like C/C++, C#, Rust, Go, Zig and many, many more.

While I already used Web Assembly to run C++ code targeted for the Arduino micro controller in the browser (see the JLed example and FastLED example here), in the following, I’ll show how to port a go command line application (my rabtap tool, a RabbitMQ client, ~9.8kLoC) to WASM.

Install go 1.21

First we need go 1.21 installed. As I started this, go 1.21 was not yet released officially so I used gotip. Gotip loads and runs the latest version of go from the development tree. Just use the gotip command where you would otherwise call go.

$ go install golang.org/dl/gotip@latest
$ gotip download
$ gotip version
go version devel go1.21-261e267 Sat Jun 17 19:02:45 2023 +0000 linux/amd64

Compile for WASM/WASI

Next lets try to compile the app for GOOS=waspi1 and GOARCH=wasm:

$ cd cmd/rabtap && GOOS=wasip1 GOARCH=wasm gotip build -o ../../bin/rabtap-wasm
# github.com/sirupsen/logrus
/home/jdelgado/go/pkg/mod/github.com/sirupsen/logrus@v1.9.0/terminal_check_notappengine.go:13:10:
undefined: isTerminal

That did not work. Checking the logrus repo I found out that the version on master has wasip1 support. Let’s update the dependencies:

$ go get -u ./...
...
$ go get github.com/sirupsen/logrus@dd1b4c2e81afc5c255f216a722b012ed26be57df
go: upgraded github.com/sirupsen/logrus v1.9.3 => v1.9.4-0.20230606125235-dd1b4c2e81af

With these changes the code compiles now:

$ cd cmd/rabtap && GOOS=wasip1 GOARCH=wasm gotip build -o ../../bin/rabtap-wasm && cd -
$ ls -lh bin
total 21M
-rwxr-xr-x. 1 jdelgado jdelgado 7.1M Jun 22 22:36 rabtap
-rwxr-xr-x. 1 jdelgado jdelgado  14M Jun 25 21:45 rabtap-wasm

but does not work:

$ wasirun bin/rabtap-wasm -- --api "http://guest:guest@localhost:15672/api" info --verbose
ERROR[0000] failed retrieving info from rabbitmq REST api: Get
"http://guest:***@localhost:15672/api/vhosts": dial tcp: lookup localhost:
Protocol not available

To fix the Protocol not available error, everywhere where we open a socket connection, we need to pass in a WASI compatible Dial function. That is what stealthrocket/net provides.I created two platform dependent files defining a Dial function each, one for GOOS=wasip1 (dial_wasip1.go) and one for all other GOOS':

//go:build wasip1

package rabtap

import (
	"github.com/stealthrocket/net/wasip1"
)

var Dial = wasip1.Dial

And the non-GOOS=wasip1 version (dial.go) which uses the standard net.Dial function:

//go:build !wasip1

package rabtap

import (
	"net"
)

var Dial = net.Dial

Rabtap creates socket connections to open AMQP connections to a RabbitMQ broker and and to speak to the REST service of the RabbitMQ management interface. Everywhere where we create a socket connection, we must explictly use our Dial function, e.g. to create the AMQP connection we use (example from dial_tls.go)

  ...
  return amqp.DialConfig(uri, amqp.Config{
      Heartbeat:       defaultHeartbeat,
      TLSClientConfig: tlsConfig,
      Locale:          defaultLocale,
      SASL:            sasl,
      Dial:            Dial,  // <--- new
      })

similary we specify the Dial function when we create a http.Client:

    ...
    tr := &http.Transport{
        TLSClientConfig:    tlsConfig,
        DisableCompression: false,
        Dial:               Dial}     // <--- new
    client := &http.Client{Transport: tr, Timeout: ... }

Let’s try to run the application again. This time the program just hangs after opening the connection. Some debugging showed that my SIGINT handler seems to not to play nicely with WASM/WASI. Without further investigation, we exclude this code (signal.go) from the WASM build:

//go:build !wasip1

package main

import (
	"context"
	"os"
	"os/signal"
)

func SigIntHandler(ctx context.Context, cancel func()) {
	// translate ^C (Interrput) in ctx.Done()
	c := make(chan os.Signal, 1)
	signal.Notify(c, os.Interrupt) // <--- does not play well with WASI, why?
	select {
	case <-c:
		cancel()
	case <-ctx.Done():
	}

	signal.Stop(c)
}

Let’s try again if we can establish a connection with these changes:

$ RABTAP_APIURI="http://guest:guest@localhost:15672/api"
$ wasirun  bin/rabtap-wasm -- --api "$RABTAP_APIURI" info --verbose
http://localhost:15672/api (broker ver='3.9.9', mgmt ver='3.9.9', cluster='rabbit@30d7f962eccf')
└─ Vhost /
   ├─ amq.direct (exchange(direct), [D])
   ├─ amq.fanout (exchange(fanout), [D])
   ├─ amq.headers (exchange(headers), [D])
   ├─ amq.match (exchange(headers), [D])
   ├─ amq.rabbitmq.trace (exchange(topic), [D|I])
   └─ amq.topic (exchange(topic), [D])

Voilà, reading the REST API works now, excellent. Note that since we can not directly access environment variables, we need to pass the URL with --api option, although rabtap normally uses the environment variable RABTAP_APIURI directly. Let’s also check the AMQP part. We run the following command in the first console (taps to the amq.topic exchange and waits for messages):

$ RABTAP_AMQPURI="amqp://guest:guest@localhost:5672"
$ wasirun   bin/rabtap-wasm -- --uri "$RABTAP_AMQPURI" tap amq.topic:# --verbose
DEBUG[0000] waiting for new session on amqp://guest:xxxxx@localhost:5672/
DEBUG[0000] got new amqp session ...

The AMQP connection could be established. In another console we now publish a message, which we expect to be delivered by RabbitMQ to the first rabtap instance above:

$ echo "hello" | wasirun bin/rabtap-wasm -- --uri "$RABTAP_AMQPURI" pub --exchange amq.topic

Back on the first console we see that the message was received:

$ wasirun bin/rabtap-wasm -- --uri "$RABTAP_AMQPURI" tap amq.topic:# 
------ message received on 2023-06-18T20:12:10Z ------
exchange.......: amq.topic
hello

AMQP works also, another job well done ;)

Summary

WASM is a promising portable binary code format allowing for platform independent programs. Porting my golang rabtap command line application did not take much time. The hardest part was to find the problems that the SIGINT signal handler caused (reasons unknown), the most important change was to use specific Dial functions to establish socket connections. Staring the WASM version of the app takes a noticable higher time (also the size of the WASM binary doubled to 14MB compared to the x86_64 version), the overall performance of the WASM version in comparison to the native version has to be assessed. While the application works, reading environment variables must be enabled explicitly, and detecting if stdout is a terminal (needed to enable colored output) is also not supported, but is not really needed. I’ll release the changes with the upcoming 1.39 version of rabtap.