– Room 5
Mick is an Apache Software Foundation Member, and active Committer, PMC member and previous PMC Chair of multiple Apache projects. An avid open source contributor wherever the itch scratches.
A Principal Software Engineer at IBM working on database internals, he has spent over a decade helping and working with engineering teams across finance, insurance, IoT, and intelligence systems – on data platforms that don't forgive downtime.
Elsker Norge, særlig vintrene.
Your application database handles writes. Your data warehouse handles analytics. Your ETL pipelines connect them. Your data scientists complain about stale data. Your platform team owns three systems, four schemas, and a Friday incident rotation.
None of this was inevitable. It was only ever a workaround to technical limitations. We just forgot.
The transactional/analytical split made sense when RAM was expensive, consensus protocols were slow, and columnar scans blocked the write path. Those constraints have shifted quietly, over the last five years. The architecture most of us inherited now costs more than it earns. Agentic AI is making this urgent, because agents want fresh data and don't care which system of record it lives in.
Today you can build an entirely vendor-neutral open-source stack that runs transactional and analytical workloads on the same dataset, zero data duplication, no ETL, SQL and Strict-Serializable ACID across your entire data estate, with a performance profile of OLTP p99 write latency under 5ms and read p99 under 50ms, on a cluster simultaneously serving 1.7 Tb/s of analytical reads. At 5-20% of the cost of the Postgres + ETL + Databricks or Snowflake stack. These savings are documented in the repo. Ground-up architecture is what makes it possible.
Mick will live-demo it. The demo spins up in four minutes on a laptop. The components are all Apache-licensed, all JVM, and most of them you already know: Kafka, Spark, Presto. The storage and transaction layer is the interesting part, and how it works will be explained from first principles.
But the talk is not about this one stack or any individual piece of technology. It is about what happens when the semantic layer of data management — transactions, consistency, schemas — decouples from the physical layer — storage format, execution engine, hardware target — so that each can evolve independently into what we now see as the open table format data lakes and the deconstructed database turned platform. Accord, Iceberg v3, Velox, and cuDF all landed within the last two years, pursuing the same goal from different directions. This talk is about their compound effect.
Bring your skepticism. The code is public, the trade-offs are real, and I'd rather be argued with than believed.
Engineers and architects making data-platform decisions — anyone running a transactional database plus ETL plus a warehouse and quietly wondering if there's a better answer.
Programmet er oppdatert.
