OpenAI’s Habitat redesign pairs shared databases with workload controls

OpenAI’s [October 23 engineering account of Habitat](https://openai.com/index/scaling‑storage‑one‑billion‑users‑part‑two/) explains how its application‑storage platform returned to shared Azure Cosmos DB containers after trying dedicated containers for individual workloads. The new Part II focuses on the underlying storage and its operation; the authors describe Part I as covering Habitat’s evolution from a Python client into a service. The architectural problem is a tradeoff. Dedicated containers separate workloads, but each needs capacity planning. Shared containers put that planning in a central team while allowing a busy tenant or frequently accessed key to compete with unrelated work on the same physical partition. OpenAI describes returning to a shared default with controls aimed at those conflicts: throttling the hottest keys and assigning background operations, such as bulk inserts or data extracts, a lower request priority than interactive traffic. The important change is shared storage combined with workload‑aware limits, rather than treating either sharing or isolation alone as the solution.

The [account](https://openai.com/index/scaling‑storage‑one‑billion‑users‑part‑two/) says OpenAI uses regional Cosmos DB accounts for users with data‑residency requirements, but placing storage near every source of demand is constrained by available infrastructure. Storage footprint, datacentre capacity, power and cooling can affect where data is placed. OpenAI describes choosing between capacity for storage near compute and capacity for additional serving compute with storage requests travelling farther. It also selectively replicates small, latency‑sensitive datasets, such as authentication metadata, across regions. The authors say capacity limits keep that replication approach restricted to selected datasets. For product teams, the consequence is that global serving capacity and data placement are separate decisions. This engineering description does not establish a new residency option or contractual guarantee for every OpenAI customer.

Habitat’s [integrated cache](https://openai.com/index/scaling‑storage‑one‑billion‑users‑part‑two/) keeps copies of recently accessed objects and edges in Valkey servers near request processing. Instead of relying only on expiry timers, OpenAI uses the Cosmos DB change feed to invalidate cached entries when the underlying data changes. A later read misses the invalidated entry and refills it. The post also describes change‑data‑capture infrastructure feeding secondary indexes, data‑movement pipelines, analytics and other features. That adds operational work beyond storing the original document: capture workers, leases, configuration changes and delivery failures all need handling. OpenAI says it isolates failures between streams, uses dead‑letter queues for permanent delivery failures and can switch readers to alternate regions during regional outages. Those are the mechanisms described in the company’s account, rather than an OpenTools test of consistency or recovery.

OpenAI’s [description of co‑managed operations](https://openai.com/index/scaling‑storage‑one‑billion‑users‑part‑two/) includes shared dashboards, direct engineering channels with Azure and joint testing against a production‑sized, non‑production Cosmos DB account. The teams work together on SDK changes and account sizing, rather than treating the cloud provider as a support queue used only after an incident. That is the broader lesson for teams building on managed storage: the application boundary can stay simple while the operating responsibilities grow. Capacity ownership, workload priorities, data placement, cache freshness and failover testing still need named owners. Habitat is OpenAI’s internal platform, and the post is an account of its design choices, not a newly available database service or a guarantee that copying those choices will reproduce its results. *1515 Third Street in San Francisco, home to OpenAI’s headquarters when photographed on June 15, 2025. Archive company context photograph. “1515 Third Street.” Photo: [Coolcaesar](https://commons.wikimedia.org/wiki/User:Coolcaesar), via [Wikimedia Commons](https://commons.wikimedia.org/wiki/File:1515_Third_Street.jpg), [CC BY4.0](https://creativecommons.org/licenses/by/4.0/). Resized and cropped for display.*