“GPU-accelerated” is a phrase that promises little: a CPU engine that offloads some operators to a device when it can. RayoDB is not that, and the difference is not marketing — it is a set of design choices, each with consequences. Here are the ones that matter.
Storage laid out for the device
Our columnar segments exist to be uploaded. Columns are stored in fixed-width, dictionary- encoded, compressed blocks with per-block zone maps, so a query can skip what it does not need and move what it does in a shape the GPU consumes directly. Hot columns stay resident in device memory across queries; cold ones stream through it. The layout on disk and the layout in GPU memory are the same idea, not two formats with a conversion in between.
Residency, not caching
A cache is something you hope is warm. Residency is a decision: which columns of which tables live on the device, chosen from observed access patterns and protected from eviction by unrelated work. When a query touches resident data, nothing crosses the PCIe bus but the result. That is the single largest difference between a GPU-native engine and an accelerator bolted onto a CPU plan.
Fusion
A CPU engine executes a filter, then an aggregation, materializing between them. On the GPU the cost of materialization is bandwidth, so we fuse: a filter-plus-aggregate reads each block once and produces the aggregated state without the intermediate result ever existing. The same idea applies to top-k and to partitioned aggregation.
One GPU per worker
Each worker process owns exactly one GPU. It sounds like a constraint; it is a simplification. There is no scheduler deciding which query gets which device, no contention between queries for the same memory, and capacity planning is arithmetic: more workers, more GPUs, more shards. The coordinator fans a query out to the workers and merges what comes back. Replication, failover and rebalancing all happen at the level of shards, which is where a database wants them anyway.
The trade-offs, honestly
GPU memory is finite and expensive. A 24 GB device holds a great deal of compressed
columnar data, but not everything, which is why residency is a decision and not a default.
Mutations are batch-oriented: RayoDB deletes and updates by predicate, and it does it well,
but it is not an OLTP database and we do not pretend otherwise. And while our SQL surface
is wide, it is not complete — there is no UNION yet, no recursive CTE, and tables are
created through an API rather than DDL. The full list is on the product page.
Why this is the right set of bets
Every one of these choices trades generality for the thing analytical queries are actually bound by: how many bytes per second can reach the operators. A GPU-native engine is what you build when you take that bound seriously. We will show what it buys with evidence, one artifact at a time — the policy is public.