The real comparison isn't one technology — it's the whole pipeline

Every tool below is excellent at the job it was built for. The problem is that delivering "streaming and analytics and history" today means stitching several of them together — and every hop adds a copy of the data, a pipeline to move it, and a lag before it's queryable.

The current reality:

Producer → Kafka/Pulsar → Flink → Iceberg → S3 → Spark/Trino

The Streaming Lake vision collapses that into one system, where the same append-only log is the stream, the table, and the history:

Producer → Pulsar → Streaming Lake (Topic = Table · Stream = Historical Dataset · Ledger = Lakehouse Storage)

The global comparison

Across the five capabilities that matter for a streaming‑native lakehouse, only Streaming Lake (and, partially, Fluss) cover most of them in one system. Everything else covers a subset and relies on another system for the rest.

● available   ◐ partial / on roadmap   — not supported. Each engine is strong in its niche; the matrix shows what a single system delivers, not quality.

Technology by technology

Kafka + Iceberg

The industry‑standard stack: mature, huge ecosystem. But it keeps the data twice — a Kafka copy and an Iceberg copy — bridged by an ETL job (Kafka Connect / Flink / Spark), so the lake is always seconds‑to‑minutes behind the stream, and historical joins need yet another engine.

Solves well	Mature, industry‑standard streaming + a query‑able table format.
The gap	Double storage (Kafka + Iceberg), required ETL to move data, freshness lag, separate engine for historical joins.
Streaming Lake	One write, one storage copy, one metadata catalog — no stream‑to‑lake replication.

Pulsar + Iceberg

The same shape with Pulsar in front: a sink connector copies topics into Iceberg, so BookKeeper and S3 hold the same data.

Solves well	Pulsar's streaming semantics plus an Iceberg analytical table.
The gap	Two storage systems (BookKeeper + S3) holding the same bytes, plus the sink pipeline between them.
Streaming Lake	BookKeeper itself becomes both streaming storage and lake storage.

Apache Hudi

Hudi is excellent at CDC, upserts, and incremental processing on the lake — but it is not a messaging system. There are no consumer groups, subscriptions, replay APIs, or pub‑sub, so you still need Kafka or Pulsar in front of it.

Solves well	CDC, upserts, incremental queries on Parquet in object storage.
The gap	No pub‑sub — no consumer groups, subscriptions, or replay. You still run a separate streaming system.
Streaming Lake	No separate streaming system — the log is the messaging layer.

Apache Iceberg

Iceberg is a superb table format — snapshots, schema evolution, hidden partitioning. But a format is not storage and not execution: it still needs object storage (S3/HDFS/ADLS) underneath and an engine (Spark/Flink/Trino) on top, and a streaming system in front to feed it.

Solves well	Open table format: snapshots, schema evolution, time travel.
The gap	Not storage and not execution — needs object store + an external engine + a stream to populate it.
Streaming Lake	Storage and execution metadata are integrated into the same engine.

Delta Lake

Same answer as Iceberg. Delta solves lakehouse tables, not pub‑sub — you still need Kafka for streaming, Spark for compute, and object storage underneath.

Solves well	ACID lakehouse tables with time travel.
The gap	No pub‑sub; relies on Kafka + Spark + object storage around it.
Streaming Lake	Messaging, table, and storage in one log — no surrounding stack to assemble.

Apache Fluss — the closest in philosophy

Fluss is the closest match in philosophy: it explicitly unifies streaming and a lakehouse into one system, with streaming tables you can query as data arrives. Both pursue the same goal — one copy of data serving streaming and analytics.

The difference is the starting point. Fluss is table‑first and leans on Flink for SQL; Streaming Lake is message‑first on Pulsar, so it inherits a mature pub‑sub ecosystem rather than rebuilding it.

Solves well	Streaming tables, unified stream + batch, tight Flink integration.
The gap	No mature pub‑sub ecosystem — no Pulsar clients, multi‑tenancy, geo‑replication, millions of topics, functions, or connectors.
Streaming Lake	Builds on the proven Pulsar ecosystem and adds the lakehouse on the same bytes.

Databricks — the closest commercial vision

The closest commercial vision: Kafka → Delta → Photon → SQL solves nearly everything analytically. But it is still multiple layers — Kafka, Delta, and object storage — with the data duplicated across them.

Solves well	Nearly everything analytical — Delta, Photon, SQL, ML.
The gap	Still multiple layers (Kafka + Delta + object store) and duplicate data.
Streaming Lake	Removes the Kafka → Delta pipeline entirely.

Snowflake

Snowflake is an exceptional data warehouse — but not a streaming platform. Getting events in still requires Kafka/Pulsar + an ingestion pipeline.

Solves well	Cloud data warehouse: elastic SQL analytics.
The gap	Not streaming‑native; needs Kafka/Pulsar/Flink to ingest.
Streaming Lake	Streaming‑first architecture — ingestion and analytics on one log.

Druid / Pinot

Both excel at low‑latency analytics with columnar storage, min/max ranges, and bloom filters, ingesting from Kafka/Pulsar. But they are analytics sinks, not a source of truth — the data is duplicated out of the stream into the OLAP store.

Solves well	Low‑latency real‑time OLAP.
The gap	Not a source of truth — data is duplicated from Kafka; no pub‑sub, replay, or consumer groups.
Streaming Lake	Analytics run directly on the source‑of‑truth storage.

ClickHouse — OLAP‑first vs log‑first

ClickHouse is one of the most important comparisons here: it overlaps directly with the analytical half of Streaming Lake, and it is one of the fastest analytical engines ever built. The difference is less about individual features and more philosophical — it comes down to where each system starts:

ClickHouse: starts as an OLAP database → later adds streaming Streaming Lake: starts as a distributed event log → evolves into a lakehouse

Architecture. A typical ClickHouse deployment moves data into ClickHouse before it can be queried — notice the data movement at each hop:

Producer → Kafka → Kafka Engine → Materialized View → MergeTree Table → SQL (or) Producer → Pulsar → Connector → ClickHouse

Streaming Lake keeps one store and serves both access patterns from it, with no ingestion pipeline into a second database:

Producer → Pulsar → BookKeeper → Pub‑Sub Broker & DataLake Broker

Storage. The two storage designs are strikingly similar. ClickHouse's MergeTree stores data in parts with columnar layout, a primary index, bloom filters, and compression. Streaming Lake stores a BookKeeper ledger → columnar pages → page metadata → temporal index. The decisive difference is the source of truth: in Streaming Lake the log itself is the source of truth; in ClickHouse the source of truth lives upstream (usually Kafka), and ClickHouse holds a derived copy.

Streaming. ClickHouse supports streaming, but internally it is still INSERT → table. It is not fundamentally a messaging system — there are no subscriptions, consumer groups, replay cursors, acknowledgements, or message‑ordering APIs. Streaming Lake has all of these natively, because Pulsar already does.

Historical replay. In ClickHouse, SELECT * FROM orders returns rows. Streaming Lake can additionally replay — "deliver every message from 3 PM yesterday," or "resume from message id X." Replaying a stream is a different capability from querying a table, and it is the foundation for reprocessing, recovery, and rebuilding downstream state.

Point‑in‑time joins. ClickHouse can certainly run Orders JOIN Customer — but the customer history must already exist as rows in a table that was maintained for it. Because BookKeeper is append‑only, every historical version is already present in the log, so an "as‑of" join reads history that is simply there rather than history that had to be pre‑materialized.

ETL & operational complexity. ClickHouse usually sits at the end of a pipeline (Kafka → ClickHouse or S3 → ClickHouse), and a full event‑to‑analytics stack often runs five systems: Kafka → Flink → Iceberg → Spark → ClickHouse. Streaming Lake collapses that to Pulsar → Streaming Lake — producer to log to SQL, with no intermediate database to operate.

Performance. This is where ClickHouse is very hard to beat — nearly a decade of vectorized execution, SIMD, compression, caches, a mature query optimizer, MergeTree, and distributed execution. The distinction here is architectural, not a performance claim: ClickHouse is likely to remain faster for heavy analytical SQL, while Streaming Lake's difference is unifying messaging, history, and analytics on a single source‑of‑truth log.

Where ClickHouse stays ahead	Dashboards, BI, complex aggregations, a mature optimizer, distributed joins — best‑in‑class raw analytical speed.
Where Streaming Lake differs	One append‑only log is the messaging system, the source of truth, the history, and the analytical store — no ingestion into a separate database.
In one line	Event log = table = historical dataset = replay log — an identity no mainstream OLAP database fully adopts.

RisingWave

Philosophically very close on the query side — streaming SQL, materialized views, joins. But it is state‑heavy and SQL‑focused rather than pub‑sub‑focused, and it consumes Kafka as its source rather than being the source.

Solves well	Streaming SQL, materialized views, incremental joins.
The gap	State‑heavy; SQL‑first, not pub‑sub; reads from Kafka rather than being the source of truth.
Streaming Lake	Log‑first architecture — the stream is the store.

Materialize

Materialize delivers excellent streaming SQL on top of Kafka — but it requires Kafka, stores additional state, and is not itself the source of truth.

Solves well	Low‑latency streaming SQL / incremental views.
The gap	Requires Kafka and maintains a separate copy of state.
Streaming Lake	No separate database — queries run on the log itself.

Event sourcing + Kafka

Many teams already run Kafka + an event store + Iceberg. That's three systems, three copies, and three operational teams to keep in sync.

Solves well	A familiar pattern that covers messaging, history, and analytics — separately.
The gap	Three systems, three copies, three teams.
Streaming Lake	One append‑only log serves messaging, historical queries, temporal joins, and analytics.

What a single log actually changes

It's tempting to reduce this to "one less copy of the data." But storage is cheap — the real difference is everything the second copy and its pipeline drag along with them:

• Eliminate ETL pipelines — no Connect / Flink / Spark job to move stream → lake.
• Reduce operational complexity — one system to run, secure, and scale instead of three.
• Eliminate freshness lag — query the live log, not a table that commits minutes behind.
• Native temporal joins on the source‑of‑truth event log — point‑in‑time‑correct, leakage‑free features and training sets, with no Iceberg snapshots or separate historical tables.
• One programming model for streaming and analytics on the same data.

An honest take

Each system above is the right tool for the job it was built for, and a few capabilities here — native joins, the full SQL engine, distributed execution — are on the roadmap rather than shipping today (the matrix marks them ◐). For a generic warehouse, Iceberg + Spark stays simpler. But for an organization already built on event streams, running messaging, analytics, historical queries, and temporal joins without copying data into a separate lake is a genuinely different architecture — not "another lakehouse," but every stream is also a historical table and every table is also a stream, on one copy in BookKeeper.