Writing a Zeebe Exporter - Part One

by Josh Wulf on May 27 2019 in Zeebe Resources.

Exporters allow you to tap into the Zeebe event stream and export selected events to other systems. You can filter events, perform transformations, and even trigger side-effects from an exporter. In this post, we’ll step through implementing an exporter.

In later posts, we’ll look more in depth at configuration and performance - but for now we’ll cover the bare minimum to help you understand how exporters work in Zeebe.

Building an Exporter

Follow along these steps to create a minimal exporter that can be deployed to a Zeebe broker.

Note: Zeebe is under active development, and things may change. I have noted in each step things that are most likely to change.

mvn archetype:generate -DgroupId=io.zeebe \
    -DartifactId=zeebe-exporter-demo \
    -DarchetypeArtifactId=maven-archetype-quickstart \
    -DinteractiveMode=false
<dependency>
    <groupId>io.zeebe</groupId>
    <artifactId>zeebe-exporter-api</artifactId>
    <version>0.17.0</version>
</dependency>

Note: the current version at the time of writing is Zeebe 0.17.0. Check the releases page for the latest version.

import io.zeebe.exporter.api.spi.Exporter;

Note: as of May 28, 2019, there is an open issue to move this interface to io.zeebe.exporter.api.Exporter.

public class DemoExporter implements Exporter {

}
public class DemoExporter implements Exporter {

    public void configure(Context context) {

    }

    public void open(Controller controller) {

    }

    public void close() {

    }

    public void export(Record record) {

    }
}

Exporter Life-cycle

These methods are the life-cycle hooks for an exporter.

Configure

The configure method allows your exporter to read any configuration specified for it in the zeebe.cfg.toml file, using the Context parameter. If your exporter throws in this method, the broker will halt during startup. This prevents the broker from starting if an exporter doesn’t have sufficient configuration to operate.

Open

If your exporter does not throw in the configure method, then another instance is created, its configure method is called, and then the open method is called. In this method you can get a reference to a Controller. The Controller provides an asynchronous scheduler that can be used to implement operation batching (we will look at that in another post), and a method to mark a record as exported.

Close

When the broker shuts down, the close method is called, and you can perform any clean-up that you need to.

Export

Whenever a record is available for export, the export method is called with the record to export. Remember that you must move the exporter record position forward, otherwise the broker will not truncate the event log.

Exporting a record

We’ll make the simplest exporter possible: we’ll write a JSON representation of the record to the console.

We won’t need configure or close, so we can remove them.

public class DemoExporter implements Exporter {
    Controller controller;

    public void open(Controller controller) {
        this.controller = controller;
    }

    public void export(Record record) {

    }
}
public class DemoExporter implements Exporter {
    Controller controller;

    public void open(Controller controller) {
        this.controller = controller;
    }

    public void export(Record record) {
        System.out.println(record.toJson());
        this.controller.updateLastExportedRecordPosition(record.getPosition());
    }
}

Deploy the Exporter

[[exporters]]
id = "demo"
className = "io.zeebe.DemoExporter"

The source code for this exporter is available on GitHub, and a docker-compose configuration for it is available in the exporter-demo folder here.