'How Does Zeebe Compare to X?': An Evaluation Framework
We often get questions about how Zeebe is the same as or different from other tools and frameworks that can be used to orchestrate workflows. These “other tools” include:
- “Traditional” transactional, open-source BPMN workflow engines (such as Camunda BPM and Activiti)
- BPM Suites (such as Pega, Software AG)
- Homegrown open-source orchestration tools (such as Netflix Conductor and Uber Cadence)
- Orchestration tools from cloud providers (such as AWS Step Functions and Google Cloud Composer)
- Distributed tracing tools (such as Jaeger)
- Integration tools (such as Apache Camel and Ballerina)
- Data pipeline tools (such as Spring Data Flow and Apache Airflow)
While we haven’t yet built a feature-by-feature comparison table showing Zeebe next to these other systems, we did put together a list of ten criteria you might consider when evaluating Zeebe alongside other tools–plus our (opinionated) input on those criteria for Zeebe.
We built the list of criteria based on a number of conversations we’ve had with early users, forum posts, questions that come up in the Slack channel, and so on. Some users might need to consider all ten of these points when choosing a workflow engine, others will only need to think about a few.
First, here’s a quick list of the criteria in table form. In the rest of the post, we’ll elaborate on each of the points and link to resources with more information.
Zeebe workflows are defined in ISO-standard BPMN 2.0. A desktop modeling tool, DSL, and XML are all options for creating a workflow model.
BPMN is well-established and mature, with many BPMN experts in the workforce. As a flow language, BPMN is powerful and already supports much if not all of the business logic a user would need for orchestration use cases.
It’s also possible to define workflows in YAML, in which case Zeebe will transform the YAML file to BPMN when it’s deployed. But we should note that YAML supports only relatively simple workflows and we don’t have any near-term plans to expand on its functionality.
Visual representation of workflows
In Zeebe, because a BPMN workflow is both a visual model and executable code (there’s a 1:1 relationship between the two), a Zeebe workflow will always have a visual representation.
We believe BPMN is especially valuable because it’s understandable by both technical and non-technical users, making it possible for business, engineering, and operations teams to use the model as a basis for collaboration.
This is referred to as BizDevOps, and we think it’s really important.
Storage of active workflow instance state
In Zeebe, the state of active workflow instances is stored on the same machines that are running Zeebe–no additional storage (e.g. relational database, NoSQL data store) is required. Under the hood, Zeebe uses RocksDB, an embeddable key-value store, for storing workflow state.
Zeebe’s no-database-required architecture is key to its scalability. It’s possible to scale Zeebe horizontally (via Partitions) to handle very high throughput without a data store acting as a bottleneck. Zeebe was designed from the start with high-throughput in mind, for use cases requiring thousands or tens of thousands (or more!) workflow instances started per second.
Zeebe is fault tolerant and can recover from failure without data loss even though it doesn’t require a separate data store for state of active workflow instances.
It’s possible to configure a replication factor to define how many times a partition is replicated across different machines in a cluster, and if a so-called “leader partition” fails, a replicated “follower” will be elected the new leader and processing will continue (see: Raft Consensus and Replication Protocol).
We also describe partitioning and replications in Zeebe in this blog post.
Supported programming languages
We’re referring here to clients the user will embed in their microservices to communicate with the engine. Zeebe ships with Java and Go clients, and users have also contributed clients in NodeJS, C#, and Ruby (see community-contributed clients here).
Because Zeebe uses gRPC for client-broker communication, it’s possible to generate clients in any of gRPC’s 10 supported programming languages.
Historic workflow data
Zeebe makes historic workflow data available in the form of exporters. To paraphrase the docs: “As Zeebe processes jobs and workflows, or performs internal maintenance (e.g. raft failover), it will generate an ordered stream of records. While the clients don’t provide a way to inspect this stream directly, an exporter provides a single entry point to process every record that is written on a stream.”
In other words, Zeebe itself doesn’t store historic workflow data, but it gives you a way to stream that data into a storage system of your choice. This data could be used simply to maintain an audit trail, to build monitoring and alerting dashboards, or to analyze workflow performance.
Zeebe ships with an Elasticsearch exporter, which will also serve as the underlying data store for Zeebe’s soon-to-come workflow monitoring and troubleshooting tool Camunda Operate and lays the groundwork for integrating Zeebe with workflow analytics tools such as Camunda Optimize in the future. In addition, the community has contributed Apache Kafka and Hazelcast exporters, too.
Integration with Apache Kafka and other messaging platforms
Zeebe’s message correlation features make it possible to build workflows that “react” to messages consumed from e.g. a Kafka topic Zeebe can also publish messages to a Kafka topic.
Learn more in this post about a prototype Kafka connector that we built or this message correlation blog post.
You can run Zeebe on any hardware of your choice–in the cloud (vendor-agnostic), on-premises, etc. The first production-ready Zeebe release in mid-2019 will be Kubernetes-ready, making Zeebe relatively flexible with regard to deployment.
You can either download a distribution or use Docker to run Zeebe.
Most Zeebe source files are made available under the Apache License, Version 2.0 except for the broker-core component. The broker-core source files are made available under the terms of the GNU Affero General Public License (GNU AGPLv3). See individual source files for details.