Airbnb Engineer Shows Valkey Can Slash AI Latency

During a technical presentation, Airbnb lead engineer Dumanshu Goyal outlined how modern artificial intelligence workloads require a shift from millisecond to microsecond data retrieval. Goyal, who previously worked on Google Cloud Memorystore and AWS Timestream, explained that AI feature stores must serve data at ultra-low latencies to keep entire systems within strict operational budgets. For example, DoorDash's prediction service operates on a tight 100-millisecond budget to run fraud detection and restaurant recommendations. Because a single prediction requires fetching hundreds of features, tail latency determines the overall speed of the application.

Historically, developers scaled caching systems like Redis—and its fully open-source 2024 fork, Valkey—by implementing intermediate proxy layers like Envoy. While these proxies successfully managed connection multiplexing for up to 100,000 client pods, they introduced significant performance penalties. Goyal demonstrated this bottleneck using a memtier_benchmark run on AWS EC2 8-core instances. Simulating a read-heavy workload of 90 percent reads and 10 percent writes at 500,000 queries per second, the proxied architecture suffered a p99 tail latency of approximately 2.5 milliseconds.

To overcome these limitations, Goyal advocated for direct-access Valkey architectures. By removing the proxy layer, practitioners can eliminate hidden CPU costs, reduce system failure risks, and achieve true microsecond-level latency. This architectural shift is particularly vital as Valkey gains rapid adoption, trending close to Postgres in developer desire according to the Stack Overflow 2025 database survey. For practitioners, transitioning to direct-access Valkey means AI models spend less time waiting for data, allowing more of the 100-millisecond budget to be allocated to complex model inference while simultaneously slashing infrastructure costs.