The thesis
Analytics has spent fifteen years squeezing the CPU. The next order of magnitude is somewhere else.
This page is the long version of the argument behind RayoDB. It is about hardware, economics and timing — and about why the change has not happened yet.
01 · Where analytics came from
The CPU era and its summit.
Modern analytical databases are an engineering triumph. Columnar storage meant reading only the bytes a query touches. Vectorized execution meant processing them in tight, cache-friendly batches. SIMD meant several values per instruction. Compression meant more data per byte moved. Every one of those ideas is about the same thing: feeding a handful of CPU cores as efficiently as physics allows.
ClickHouse is the summit of that road. It is a superb engine and we are compatible with its syntax on purpose. But a summit is also the point where the road stops going up.
02 · Why the road ends
Per-core gains are flat and bandwidth is the wall.
An analytical query is mostly a bandwidth problem: scan a column, filter it, aggregate it, join it. How fast that goes is bounded by how many bytes per second the processor can pull from memory. A high-end server socket, with as many DDR5 channels as the platform allows, gives you a few hundred gigabytes per second — shared by every core on it.
The industry's answer has been to scale out: more nodes, more sockets, more coordination, more cost. It works, and it hits diminishing returns in operations and money long before it hits a physical limit.
Data table
| Device | Memory bandwidth |
|---|---|
| Server CPU socket — 8–12 channels DDR5 | ≈ 300–600 GB/s |
| One GPU — NVIDIA RTX 4090, 24 GB GDDR6X | 1,008 GB/s |
| One GPU node — 8 × RTX 4090, aggregate | ≈ 8,000 GB/s |
03 · What the GPU changes
Parallelism that matches relational operators.
A GPU is thousands of simple cores attached to memory that moves a terabyte per second. Scans, filters, hash aggregations, joins and sorts are exactly the kind of data-parallel work those cores were built for. Fusing a filter with an aggregation means the data is read once and never leaves the device until the answer is ready.
A single node with eight GPUs has more aggregate memory bandwidth than a rack of CPU servers, in a footprint one team can understand and operate. That is the order of magnitude we are talking about — not a benchmark, a different tier of hardware.
04 · Why it did not happen before
GPUs were scarce, tooling was young, products were closed.
GPU databases have existed for a decade and they proved the thesis. They also stayed proprietary, priced for a few large accounts, and built around their own SQL dialects and tooling. Most data teams never got to try one.
Three things changed. GPUs are now everywhere, from clouds to on-premise racks bought for inference. The CUDA software stack is mature, with libraries and compilers that make a modern engine feasible for a focused team. And the industry has standardized on a common analytical SQL vocabulary, so compatibility is a design goal you can actually meet.
05 · What RayoDB does about it
A database built for the device, with the SQL you already know.
Four design choices, each one a consequence of taking the GPU seriously.
Storage laid out for the device.
Columnar segments with zone maps and dictionary encoding are designed so that hot columns live in GPU memory and cold ones stream through it. The GPU is not a cache bolted onto a CPU engine; it is where the data is meant to be.
One GPU per worker.
Each worker owns exactly one GPU. Capacity is a count, not a tuning exercise: more workers, more GPUs, more shards. The coordinator fans out and merges.
Durability off the hot path.
A write-ahead log, atomic manifests, replication with read failover and cluster-wide backup keep the boring guarantees — without putting them between the query and the device.
Models served where the data is.
Forecasting, anomaly detection, embeddings and similarity run as SQL functions on the same GPUs, from an ONNX model catalog you manage with SQL.
06 · What it means for you
Interactive over billions. And new questions.
The obvious consequence is interactivity at scales where today you pre-aggregate, sample or wait. The less obvious one is that living on the GPU puts machine learning next to the data: a forecast or an anomaly score becomes a clause in the query, not a pipeline.
We do not put numbers on this page. When we publish a performance figure it will come with the artifact, the workload and the hardware that produced it — here is the policy. Until then, the argument stands on hardware you can look up and a product you can try.
Want to see what the GPU changes on your data?
Early access is open to a small group of design partners with real workloads.