Introducing Calinora Pilot: A Simpler Alternative to Cruise Control
Every Kafka team hits the same wall.
Broker load drifts. Figuring out which partitions to move, where, and whether it’s safe - that’s the work nobody enjoys. Existing tools like Cruise Control require JMX metrics, broker-side agents, and complex configuration before you can even start.
That’s why we built Calinora Pilot - a simpler approach to Kafka cluster operations.
No JMX. No Broker-Side Changes.
Pilot understands cluster state using Kafka’s own metadata APIs. There’s nothing to install on your brokers, no JMX exporters to configure, and no sidecar processes to manage. Point it at your cluster and it starts working.
At a glance, you get a full health overview of your cluster - healthy partitions, under-replicated state, offline replicas, and rack-awareness violations - all in one place.
Pilot also tracks topic-level activity in real time, showing producer and consumer throughput per topic so you can identify hot spots before they become problems.
Multi-Metric Rebalancing
Under the hood, a multi-phase rebalancing engine optimizes leaders, replicas, disk usage, and partition activity simultaneously - not one goal at a time, but all metrics scored together.
Every move is evaluated against an integrated cost model that prefers leader elections over expensive data transfers, penalizes large partitions, and avoids cross-rack transfers when possible.
The proposal detail view breaks down every aspect: total moves, variance reduction, estimated duration, rack constraint analysis, and a full broker-by-broker comparison of current vs. projected metrics.
Common Use Cases
- Scaling brokers up or down - Pilot evens out partitions across the new set
- Adding or removing disks - prepare for disk replacement or expansion
- Fixing under-replicated partitions after broker failures
- Fixing rack-awareness violations to restore fault tolerance
- Changing replication factors across topics
- Reassigning partitions to specific brokers or changing leaders
All with audit logging and one-click revert.
For disk operations, Pilot provides a guided workflow to move replica data between log directories on a broker - balance disks automatically, or pick specific source and target disks.
Live Reassignment Monitoring
Once a proposal is applied, the reassignment monitor tracks every partition move in real time. You see overall progress, transfer speed, ETA, per-broker data flow, and individual partition lanes split into queued, moving, and done.
Beyond Rebalancing
Pilot isn’t just a rebalancer. It’s a full operational toolkit:
- What-if simulations - model broker failures, rack outages, or traffic spikes before they happen
- Blast-radius analysis - see which topics and consumer groups would be impacted
- Self-healing - automatic detection and repair of under-replicated partitions
- Message browser - inspect topic contents without a separate consumer
- Consumer lag, quotas, ACLs - manage everything from one UI
There’s also an optional AI assistant that can interpret cluster state, run simulations, and guide operations workflows in natural language. Ask it to “rebalance the cluster” and it generates a proposal, explains the metrics, and asks for your approval before applying.
Single Binary, Modern UI
Everything ships as a single ~30MB container with a modern React UI. No external databases, no complex deployment - just a single binary that connects to your Kafka cluster.
Get Started
Pilot is available as a Docker image. Check out the documentation to get started, or visit calinora.io to learn more.
We’re looking for teams running Kafka in production who want to try Pilot and help shape what comes next. If that’s you, reach out - we’d love to hear from you.
