Zeebe on Camunda Cloud: Getting Started

by Josh Wulf on Sep 11 2019 in Camunda Cloud.

Zeebe is now available as a managed service in the Camunda Cloud.

This means that you can experiment with Zeebe without having to set up and manage the broker. Currently it is in beta, and you can apply here to get an account.

You will be able do both development and go to production without having to concern yourself with deploying and managing the Zeebe broker on Kubernetes - you can leverage the expertise of the Camunda Cloud Engineering team, who do that full-time.

You can still run Zeebe locally or deploy it to the cloud yourself, so this is an additional option rather than a change to anything.

This post is a guide to getting started with the Zeebe service on the Camunda Cloud once you have an account.

Create a Cluster

Once you have your Camunda Cloud account:

Create New Cluster

Create New Cluster Dialog

Cluster Provisioning

Cluster Provisioned

Retrieve Cluster Contact Details

Cluster Contact Details

You will need these details to interact with the cluster.

Create a client

In order to communicate with your cluster, you need to create a client. This creates an id/secret pair that can be used to retrieve a JWT to authorize requests. You can do this while the cluster is provisioning.

Create a new client

Create a new client

This creates a new client with an id and a secret:

Create a new client

Test connection using zbctl

Let’s test the connection to the cluster using zbctl, the command-line client for Zeebe.

npm i -g zbctl

You’ll need Node installed to do this. If you don’t want that, then you need to download the appropriate zbctl binary for your operating system (0.21.0-alpha3 or later) from the releases page.

zbctl --address <Zeebe ContactPoint> --clientId <clientId> --clientSecret <clientSecret> status 

You should see some output like this:

Cluster size: 1
Partitions count: 2
Replication factor: 1
Brokers:
  Broker 0 - zeebe-0.zeebe-broker-service.1cadb038-fcd7-41eb-976d-36cd8eaa068c-zeebe.svc.cluster.local:26501
    Partition 1 : Leader
    Partition 2 : Leader

It’s alive! You now have a Zeebe cluster running on Camunda Cloud.

Using the environment

If you saw Daniel Meyer’s demo at CamundaCon, you may have noticed him using zbctl to communicate with the cluster without any flags. You can do this too by putting your configuration into environment variables. The zbctl binary will use those if they are available. Set your environment by adding this to your ~/.bashrc or ~/.zshrc file, or creating a cluster.env file like this:

export ZEEBE_ADDRESS=<Zeebe ContactPoint>
export ZEEBE_CLIENT_ID=<clientId>
export ZEEBE_CLIENT_SECRET=<clientSecret>

Now source the file, using, for example: source cluster.env. This will add these credentials to your environment, and you can now run zbctl status with no arguments, and it will communicate with your cluster!

The client libs have been crafted to work the same way, allowing you to externalise your configuration. You can inject secrets into Docker containers, for example, and you don’t need to write boilerplate to demarshal the credentials from the environment if you use the well-known environment variables.

Programming Clients to use the cluster

To connect to the cluster in a client application, you need to use the latest Zeebe clients - ones that support TLS and the OAuth flow that the cloud uses to authorize connections. At the time of writing, the 0.21.0-alpha2 version of the Java client, 0.21.0-alpha3 of the Go client, 0.20.4 of the JavaScript client, and 0.12.1 of the C# client support connecting to Camunda cloud.

To connect to your cluster you need to provide your cluster and client credentials to the client library. We have allowed you to do this explicitly in code, and also via the environment.

The Zeebe E-commerce demo repository contains a mixed JavaScript / Java project that you can examine to see a working scenario with a REST front-end and multiple microservice workers working on Camunda Cloud.

Here is how you configure workers with the explicit configuration:

import io.zeebe.client.ZeebeClient;
import io.zeebe.client.impl.oauth.OAuthCredentialsProviderBuilder;
import io.zeebe.client.impl.oauth.OAuthCredentialsProvider;

public class App {
    public static void main(String[] args) {
        final String broker = System.getenv("ZEEBE_ADDRESS");
        final String audience = System.getenv("ZEEBE_TOKEN_AUDIENCE");
        final String clientSecret = System.getenv("ZEEBE_CLIENT_SECRET");
        final String clientId = System.getenv("ZEEBE_CLIENT_ID");

        final OAuthCredentialsProvider cred = new OAuthCredentialsProviderBuilder()
                .audience(audience)
                .clientId(clientId)
                .clientSecret(clientSecret)
                .build();

        final ZeebeClient client = ZeebeClient.newClientBuilder()
                .brokerContactPoint(broker)
                .credentialsProvider(cred)
                .build();
    }
}
    
import { ZBClient } from "zeebe-node";

const clusterId = process.env.ZEEBE_CLUSTER_ID
const clientId = process.env.ZEEBE_CLIENT_ID
const clientSecret = process.env.ZEEBE_CLIENT_SECRET

const zb = new ZBClient({
  camundaCloud: {
    clusterId,
    clientId,
    clientSecret
  }
});
    

A note about where to get these values from:

For the JavaScript client, you need only your Cluster Id, and the Client Id and Client Secret of the client you created.

For the Java client, you need to supply the broker address, which is the “Zeebe ContactPoint” from the Cloud Console, including the port. The ZEEBE_TOKEN_AUDIENCE is the Zeebe ContactPoint without the port.

Now, if you set the appropriate environment variables, you can use a zero-ceremony constructor, and the client will detect these values automatically:

import io.zeebe.client.ZeebeClient;
import io.zeebe.client.impl.oauth.OAuthCredentialsProviderBuilder;
import io.zeebe.client.impl.oauth.OAuthCredentialsProvider;

public class App {
    public static void main(String[] args) {
        final String broker = System.getenv("ZEEBE_ADDRESS");

        final OAuthCredentialsProvider cred = new OAuthCredentialsProviderBuilder()
                .build();

        final ZeebeClient client = ZeebeClient.newClientBuilder()
                .brokerContactPoint(broker)
                .credentialsProvider(cred)
                .build();
    }
}
    
import { ZBClient } from "zeebe-node";

const zb = new ZBClient();
    

To switch the JavaScript client between your local broker and the cloud, simply unset the three enviroment variables for Camunda Cloud. The client will default to localhost with no TLS on the default port - the default configuration of the Zeebe Docker image.

Visibility in the Cluster

The Zeebe service on the Camunda Cloud comes with an instance of Camunda Operate - the visual inspection tool for Zeebe that queries an Elastic Search server. Historical data is exported from Zeebe into Elastic Search (Zeebe’s API is all about the append-only stream, so you cannot query it).

In your Cloud Console you will see this link:

Camunda Operate

Clicking this link will open your Operate instance in another browser tab. Note that Operate views historical data, so it is not real-time, and when your system is under heavy load it will trail the current state.

The HTTP worker

A commonly requested feature is an HTTP worker. Users ask: “We already have so much investment in REST microservice architecture. Do we need to rewrite everything as gRPC polling workers to use this? Or do we need to write, deploy, and maintain a gRPC translation layer for our existing REST services?

The answer now is: “No.”

We have included in the Zeebe Service a generic HTTP Worker that is always running and can be used to wire your “invoke me!” REST endpoints to service tasks in a BPMN model. When it is time for that service task to execute, the HTTP worker will invoke your REST endpoint.

To use it, set the task type in your BPMN model to CAMUNDA-HTTP. This task type will be serviced by the HTTP Worker.

Create an HTTP worker service task

You can inject variables that should not be part of the model, such as API secrets, using a template string in the form ${variableName}:

Set variables for an HTTP worker service task

These templated variables are replaced with the values that you set in the HTTP Worker “Worker Variables” dialog in the Cluster section of the Cloud Console.

The following custom headers have a special meaning for the HTTP Worker task type:

All other headers will be passed to the called webservice as HTTP headers, with the variable substitution for templated strings.

For POST, PUT and PATCH requests, the worker will invoke the outbound webservice with a JSON payload containing all process instance variables by default. The data to be passed on can be configured via input mapping on the BPMN service task. For GET and DELETE requests, no process variables can be passed on to the outbound webservice yet.

The data being returned by the called webservice (which is expected to be JSON) will be passed back to Zeebe as process instance variables. Again, this can be configuring via output mapping on the BPMN service task.

The job in Zeebe will be marked as completed as long as the called webservice responds with the HTTP status code 200. Otherwise the job will be marked as failed.

The Future

This is Stage 0 of the Zeebe Service in Camunda Cloud, and is suitable for early adopters to begin to experiment with Zeebe with much of the deployment and configuration overhead taken care of for you.

The developer ergonomics will improve rapidly as we continue to dogfood the service ourselves and assimilate feedback from the community.

Stay tuned for an upcoming post on using the Zeebe service on Camunda Cloud for Home Automation, using the Zeebe Node-RED client.

Apply for the beta of Camunda Cloud here.

You can give feedback and get support for the Zeebe service on Camunda Cloud in the Zeebe Forum and the #cloud channel in the Zeebe Slack (get an invite).