The boundary between relational database query engines and deep learning execution kernels is dissolving. A benchmark inside xarray-sql demonstrates how a standard FlashAttention execution plan can be synthesized automatically from roughly 25 lines of declarative SQL.
Standard self-attention requires materializing massive N-by-N intermediate attention matrices, which creates a critical memory bottleneck. FlashAttention overcomes this by tiling inputs and fusing softmax reduction with matrix multiplication into a single GPU pass. The authors show that relational query optimizers, when extended with array-aware relational algebra, recognize these exact loop-tiling and intermediate-fusion opportunities from pure SQL aggregates.
Instead of writing hundreds of lines of fragile, hardware-specific CUDA or Triton code, the execution engine views attention as a standard join-aggregate query. The compiler handles memory hierarchy placement, shared memory tiling, and online softmax scaling through standard query transformation passes.
Relational algebra continues to prove itself as one of the most durable abstractions in computing.













