On September 11th, OpenAI disclosed the expansion process of its internal online storage platform, Habitat. The official figures are quite impressive: this platform currently supports over 1 billion users per week, covers nearly 40 geographical regions, processes more than 70 million requests per second, and handles data volumes exceeding 500PB. It initially served as a client library, Python, prepared for GPT products around DevDay in 2023. Today, it has evolved into a unified service that connects Azure Cosmos DB, caching, object storage, and change data capture systems. Rather than focusing on another model capability upgrade, this engineering review is more about highlighting a fact often overlooked: whether a chat window can open quickly depends first and foremost on the stability of the underlying data systems.
A single login, viewing of Codex settings, or initiating a new conversation may trigger multiple data reads behind the scenes. If any one of these requests slows down, users will experience lag; if a critical read fails, the product may become completely unusable. In the early stages, Habitat encapsulated routing, authorization, encryption, serialization, connection pooling, and request formatting within the client library, allowing the product team to avoid having to directly understand the underlying database. However, as the number of products increased, the types of data varied, and regional requirements grew, this approach began to reveal issues such as scattered versions, difficulties in deployment, and a lack of unified control. As a result, the team transformed this library into independent services.
From the Python library to platform services, what is truly achieved is unified control, rather than just "a different way of writing."
After centralizing the storage logic, Habitat has become the common entry point for observing, deploying, and protecting downstream resources. The product team still uses relatively simple interfaces, but the platform can determine in the background whether the data comes from Cosmos DB, Valkey, Nanobase, or Blob for storage, and it can also perform access control, data residency, throttling, and isolation. The value of this architecture lies not in making each read operation seem more sophisticated, but in bringing together the basic functions that were previously implemented separately by dozens of teams, allowing errors to be detected and corrected in one place.
Rapid growth also forces the team to make a large number of not-so-obvious trade-offs. It is necessary to monitor the event loop latency of Python and asyncio, reduce the tail latency caused by feature toggling, balance different connection pools, and avoid traffic spikes that could overwhelm the downstream databases. An average latency that seems good is not enough; what truly affects the user experience is the latency of that small portion of requests with the highest latency. When a single conversation page relies on multiple reads at the same time, the latency of a particular service can be amplified, and users will only see the page not responding for a long time. Therefore, large-scale online systems often pay more attention to latency, timeout, and degradation strategies, rather than the best results from a single benchmark test.
According to OpenAI, the demand has grown by more than ten times each year over the past three years. Conventional projects can be designed in advance for the next tenfold scale and then gradually migrated over a few years; however, continuous rapid growth has forced the team to maintain the online systems while striving to gain time for fundamental restructuring. Therefore, the strategy of Habitat is not to build a perfect platform from day one, but rather to first optimize the existing components, address the most urgent capacity and reliability issues, and then gradually implement services, isolation, and a new storage layer. This sequence is closer to the real production environment than starting from scratch.
Complex queries are also deliberately placed outside of the main path. The online interface of Habitat emphasizes simplicity and predictability in reading and writing; teams that need to analyze and search data use data capture to direct the data nearly in real-time to an isolated instance of Rockset. While this does increase some operational friction, it avoids heavy queries from competing with online product traffic for the same database. For any platform that simultaneously handles transaction paths and analysis tasks, separating these two types of loads is often a more reliable approach than continuously adding machines to the main database.
Two engineers used Codex to rewrite the Rust service; the numbers are impressive, but the migration is still not complete.
The official also revealed that in the second quarter of 2026, two engineers used Codex and GPT-5.5 to rewrite the entire service from Python to Rust. The new service currently handles 95% of production requests, with CPU seeing a six-fold increase in efficiency and a fifteen-fold increase in memory efficiency. The average and tail latency have also significantly decreased. At its peak, the Python version was capable of processing over 20 million requests per second. This indicates that the migration was not due to the old system being completely unusable, but rather that as the scale continued to grow, the running costs, resource efficiency, and latency stability began to justify the cost of rewriting.
These data cannot be simply understood as “Rust is always fifteen times faster than Python.” The actual differences stem from the language runtime, memory model, network stack, concurrency methods, and architectural optimizations that are also accomplished during overwriting. The official statement also indicates that Python still handles about 5% of the production traffic and is planned to be phased out within the next few weeks, rather than being completely taken offline. The most important tasks during the migration process are typically double writing or shadow traffic, result consistency, gradual release, and the ability to roll back, rather than the code translation itself.
For other companies, the lesson from Habitat is not to simply copy their technology stack. Most teams do not have 70 million requests per second, nor do they need 500PB of online data. A more common approach is to narrow down high-frequency storage operations, provide isolated outlets for complex queries; make permissions, routing, throttling, and monitoring part of the platform's capabilities; quantify bottlenecks before rewriting; and retain old paths during migration while gradually verifying new ones. Only when resource costs and growth pressures are sufficiently high does language migration become more cost-effective than continuously optimizing the original system.
OpenAI This time, we are releasing the first part of a two-part series. The database layer and multi-tenant optimization will be discussed in subsequent articles. Therefore, for now, what we can confirm is the overall scale of Habitat, the service-oriented approach, and the migration results of Rust. However, this does not mean that all storage issues related to OpenAI have been resolved. AI The first impression a product gives is based on its model, but its long-term usability depends on these less visible engineering aspects. As we approach a user base of one billion, competition is no longer just about what the model can do; it also depends on whether each read operation can be completed with the correct permissions, in the right region, and at the right time.
