Zeebe Message Correlation

by Josh Wulf on Aug 14 2019 in Use Cases.

Message correlation is a powerful feature in Zeebe. It allows you to target a running workflow with a state update from an external system, asynchronously.

You can use it to do things like:

This tutorial uses the JavaScript client, but it serves to illustrate message correlation concepts that are applicable to all language clients.

We will use Simple Monitor to inspect the running workflow state. Simple Monitor is a community-supported tool, and is not designed to be used in production - however, it useful during development.

Workflow

Here is the basic example from the Zeebe documentation:

Message Correlation Example

Use the Zeebe Modeler to open the test-messaging file in this project.

Click on the intermediate message catch event to see how it is configured:

Message Properties

A crucial piece here is the Subscription Correlation Key. In a running instance of this workflow, an incoming “Money Collected” message will have a correlationKey property:

zbc.publishMessage({
    correlationKey: "345",
    name: "Money Collected",
    variables: {
      paymentStatus: "paid"
});

The concrete value of the message correlationKey is matched against running workflow instances, by comparing the supplied value against the orderId variable of running instances subscribed to this message. This is the relationship established by setting the correlationKey to orderId in the message catch event in the BPMN.

Running the demonstration

 npm i && npm i -g ts-node typescript
 ts-node start-workflow.ts

This starts a workflow instance with the orderId set to 345:

await zbc.createWorkflowInstance("test-messaging", {
      orderId: "345",
      customerId: "110110",
      paymentStatus: "unpaid"
})

Message Properties

The numbers above the BPMN symbols indicate that no tokens are waiting at the start event, and one has passed through; and one token is waiting at the “Collect Money” task, and none have passed through.

Message Properties

You can see that this workflow instance has the variable orderId set to the value 345.

ts-node workers.ts

Message Properties

Now the token is at the message catch event, waiting for a message to be correlated.

Message Properties

You can see that the broker has opened a message subscription for this workflow instance with the concrete value of the orderId 345. This was created when the token entered the message catch event.

ts-node send-message.ts

Message Properties

The “Message Subscriptions” tab now reports that the message was correlated:

Message Buffering

Messages are buffered on the broker, so your external systems can emit messages before your process arrives at the catch event. The amount of time that a message is buffered is configured when publishing the message from the client library.

For example, to send a message that is buffered for 10 minutes with the JavaScript client:

zbc.publishMessage({
    correlationKey: "345",
    name: "Money Collected",
    variables: {
      paymentStatus: "paid"
    },
    timeToLive: 600000
});

Here is how you can see it in action:

ts-node send-message.ts

Message Properties

ts-node start-workflow.ts

You will see that the message is correlated to the workflow instance, even though it arrived before the workflow instance was started.

Common Mistakes

A couple of common gotchas:

For example, the interrupting boundary message event in the following example will not be correlated on the updated value, because the subscription is opened when the token enters the subprocess, using the value at that time:

Message Properties

If you need a boundary message event correlated on a value that is modified somewhere in your process, then put the boundary message event in a subprocess after the task that sets the variable. The message subscription for the boundary message event will be opened when the token enters the subprocess, with the current variable value.

Message Properties

Summary

Message Correlation is a powerful feature in Zeebe. Knowing how messages are correlated, and how and when the message subscription is created is important to design systems that perform as expected.

And Simple Monitor is a useful tool for inspecting the behavior of a Zeebe system to figure out what is happening during development.

What would you like to see covered in future articles? Let us know via feedback@zeebe.io. Have a question about Zeebe? Drop by the Zeebe Slack channel or Zeebe User Forum.