Using the Zeebe Kafka Connector to Orchestrate, Integrate, and More
The Zeebe team just implemented a Kafka Connect Zeebe connector. This is a feature users have been asking for since the start of the project, so let’s give it a closer look.
What is Kafka Connect?
Kafka Connect is the ecosystem of connectors into or out of Kafka. There are lots of existing connectors, e.g. for databases, key-value stores or file systems. So for example you can read data from a RDMS and push it to Elasticsearch or flat files.Read more…
Coming in Zeebe 0.21: Long-polling workersThe upcoming 0.21 release of Zeebe includes a feature many users have been asking for: long-polling for workers. And make sure you stay tuned to the end to find out what a massive deal it is. Zeebe is a radical re-imagining of the workflow engine for the modern world: it uses event sourcing to interpret workflows over immutable streams. In the Zeebe model, workers are de-coupled from the broker. Conceptually, workers “subscribe” to task types on the broker to service. Read more…
Yet Another License (YAL)? Why We Created The Zeebe Community LicenseIn early July, 2019, I sat down with Daniel Meyer, Camunda CTO and Zeebe Committer #0, to talk about the new Zeebe Community License. This is the transcript of that conversation. Josh Wulf: We have a change to the licensing for Zeebe - probably, people are going to be interested to know more about that. I know you’ve done a lot of work on it - you’ve been speaking with British lawyers most recently about it. Read more…
Zeebe: Workflow Reinvented for Microservices and the Cloud (From Idea To Production Readiness)
Last Wednesday, July 17, we announced the first production ready release of Zeebe, Camunda’s new cloud-native workflow engine for microservices orchestration. Zeebe is a new code base, written from scratch, putting forward a completely new way of architecting workflow engines for microservices and distributed systems.
This blog post is a quick overview of why and how we did that.Read more…
Generating a Zeebe-Python Client Stub in Less Than An Hour: A gRPC + Zeebe Tutorial
Please note that this blog post has not been updated since its original publication to reflect changes to Zeebe’s gateway.proto file.
The general approach for creating a stub that we show here still applies to newer versions of Zeebe, but you may need to make a few modifications depending on which Zeebe version you’re using.
The high points:
- Starting in Zeebe 0.12, Zeebe clients communicate with brokers via a stateless gRPC gateway, with Protocol Buffers used as the interface design language and message interchange format.
- gRPC makes it easy to generate a “client stub” in any of ten supported programming languages; this means it’s possible to use Zeebe in applications written in not only Java and Go–the languages with officially-supported Zeebe clients–but also Python, Ruby, C#, PHP, and more.
- We’ll show you step-by-step how we generated and started prototyping with a Python client stub in less than an hour, and we’ll provide everything you need to follow along.
Event-Driven Orchestration: Message Events and Message Correlation in ZeebeZeebe 0.12 supports message events, making it easy to use Zeebe as part of an event-driven architecture; workflow instances can to respond to events published by external systems. And Zeebe handles all of the heavy lifting related to correlating a message to a workflow instance: buffering incoming messages, opening a subscription to receive messages that match with active workflow instances, and correlating a message (when one exists) to a workflow instance that has arrived at a message event. Read more…
Zeebe, Event-Driven Architectures, and More: Talk Recordings from Kafka Summit SF and CamundaCon
It’s been a busy past month for the Zeebe team here at Camunda. At the end of September, we took the stage at CamundaCon, our annual user conference, and in mid-October, we traveled to San Francisco to give a presentation at Kafka Summit.Read more…
BPMN and Microservices Orchestration, Part 2 of 2: Graphical Models, Simplified Sagas, and Cross-functional Collaboration
This is part 2 in a 2-part series about BPMN and how it’s being applied to new use cases. You can find part 1 here. A sincere thanks to Bernd Rücker for his feedback during the writing of both blog posts.
Welcome back to our discussion of BPMN (Business Process Model and Notation) and its role in emerging use cases such as microservices orchestration. You don’t have to read the posts in order to be able to follow along, but if you’re new to BPMN, you might find it helpful to start with part 1.
To recap, the first post covered:
- An introduction to BPMN
- Why a well-established standard that thrived in the past can thrive in the future, too
- Common orchestration patterns supported by BPMN
- The current state and future plans of BPMN in Zeebe
In this part 2, we’ll:
- Look at examples where using a graphical model instead of a code-based model simplifies workflow definition
- Dive into tooling for building graphical models in BPMN (and other ways to define workflows)
- Reassure you that BPMN’s graphical models are nothing to be afraid of–even if you’ve had a bad experience with graphical models in the past
BPMN and Microservices Orchestration, Part 1 of 2: Flow Languages, Engines, and Timeless Patterns
A sincere thanks to Bernd Rücker for his feedback during the writing of this blog post.
This is part 1 of 2 in a 2-part blog post series. Part 2 is available here.
We’re building Zeebe to be a next-generation workflow engine for emerging use cases such as microservices orchestration–use cases that may require an engine to handle hundreds of thousands (or millions) of new workflow instances per second.
And to do that, we’re using a graphical modeling standard that’s been around for almost 15 years: BPMN (Business Process Model and Notation).Read more…
Benchmarking Zeebe: An Intro to How Zeebe Scales Horizontally and How We Measure It
In the past few weeks, we’ve mentioned Zeebe’s performance in horizontal scalability benchmarks that we run internally, but we haven’t yet explained how exactly we run these benchmarks. We decided we should take it one step further and open up the benchmark to anyone who wants to try it.Read more…