IT News on November 9th: According to Bloomberg, Google, a subsidiary of Alphabet, is expanding its lineup of artificial intelligence chips. It hopes to leverage its rapid rise in the AI chip market to further challenge NVIDIA. This market is already worth hundreds of billions of dollars.
Google will launch two new tensor processing units ( TPU ). TPU is a custom chip designed by Google, mainly used for developing and running AI software. The newly released versions are aimed at confidential computing, as well as helping AI models to access a wider range of information when processing requests. Overall, these chips are designed to meet the growing demands of the industry's transition towards AI intelligent entities, while also addressing the accompanying security challenges.
This release also signifies that Google's chip division is accelerating the pace of product iteration. In the past, Google typically launched a new AI processor every two years; this year, the division released two chips at once. The latest version, which will be released on Monday, will further expand the product lineup of these two chips.
Amin Vahdat, the head of Google's chip business ( Amin Vahdat ), said in an interview that the four TPU products will be optimized to meet different needs. He said, "This actually follows a concept that Google has always adhered to: the future will belong to specialization, and the advantages brought by specialization are still very significant."
Google chooses to further expand its presence at a time when the chip industry is experiencing turmoil. Concerns that investment in AI infrastructure may slow down have already caused fluctuations in tech stocks; moreover, the Democratic Party's victory in the U.S. mid-term elections has also introduced new uncertainties for data center projects.
However, the market demand for NVIDIA AI processors still exceeds their supply capacity, which leaves room for development for Google and other competitors, giving them the opportunity to build considerable businesses in this market.
Wahdat stated that the newly launched TPU 8tx, TPU 8i, and Max are not completely redesigned chips, but rather reconfigured based on existing chips to meet the specific requirements put forward by customers.
Among them, TPU 8tx is mainly targeted at customers who need to process secure and confidential data, such as government agencies and industries that are under strict regulation. This system will work in conjunction with the chips of AMD, which is part of Google's collaboration with AMD.
TPU 8i Max is aimed at scenarios that require running extremely long context windows AI. The context window determines how much data the AI system can take into account when processing a single request. Wachdat stated that such capabilities are very important for tasks such as drug development and mathematical calculations, and also have significant implications for programming applications.
The market in which TPU operates is currently dominated by NVIDIA's graphics processors ( GPU ). Over a decade ago, Google began developing this technology and collaborated with Broadcom ( Broadcom ), which assisted with chip design and coordinated production. Google initially developed TPU to more efficiently meet the growing computational needs of this search giant. Subsequently, Google discovered that TPU also had great potential in the field of AI computing, and has now begun selling these chips to customers outside of its own cloud computing business.
Google's transition into a chip supplier business is generating considerable revenue. Bloomberg Industry Research ( Bloomberg Intelligence ) predicts that by 2028, Google's ( TPU ) sales will exceed $135 billion, including revenue from related software and hardware. This figure is significantly higher than the $12 billion estimated for this year, and it does not yet take into account the revenue generated by customers using ( TPU ) through Google Cloud.
Currently, Google TPU has already attracted well-known customers such as Apple, Meta, and Oracle. Wachdat stated that launching specialized chip versions for different purposes will help Google further expand its market. He mentioned that these new versions are expected to be available next year.
He said, "We can create a general-purpose chip that tries to meet the needs of the entire market, or we can launch different specialized versions. With the latter, we can actually create more value for our customers."
Even NVIDIA, which has long positioned GPU as a general-purpose product suitable for all AI tasks, has now begun to offer more specialized chip systems. This includes the launch of its own central processing units ( CPU ). The category of CPU products was initially pioneered by manufacturers such as Intel ( Intel ). These processors are not responsible for the large-scale data computations required to train AI models, but they are very suitable for managing AI intelligents.
NVIDIA is also strengthening its chip architecture for the inference phase targeting AI. Inference refers to the process where a trained AI model performs specific tasks. NVIDIA is leveraging the technology acquired from the startup Groq to launch chip products that are more focused on this stage.
Google’s TPU 8tx will be used in conjunction with AMD’s CPU. AMD is also a competitor of NVIDIA and Intel in the chip market. This technology is capable of encrypting data at the chip level, thereby enhancing security. Wachdat stated that traditional computers usually need to decrypt information in memory before processing it, but TPU 8tx can avoid this issue.
TPU 8i Max integrates two TPU chips within the same package to provide stronger computing power and larger memory capacity.
Mandip Singh, an industry research analyst at Bloomberg, stated that one of the major attractions of TPU is its cost-effectiveness. Compared to NVIDIA GPU, TPU has a cost about 30% lower, yet it is competitive in terms of throughput. Throughput refers to the amount of data that a device can process within a certain period of time.
This solution has attracted customers from various industries, including finance. Google's parent company, Alphabet, revealed during its third-quarter financial report last month that the quantitative trading firm Jump Trading is also a customer of TPU.
Jump Trading Chief Technology Officer Alex Davies ( Alex Davies ) stated in an interview that the company has spent several years testing TPU, and about a year ago, they signed a larger-scale cooperation agreement with Google.
This trading company currently possesses thousands of TPU, which are mainly used for training AI models, such as to assist in trading research. Davis stated that Jump still primarily uses NVIDIA chips for model training, but there is a small group of employees within the company who have a particular fondness for TPU and they express this preference quite positively.
Davis stated that TPU performs exceptionally well in certain specific tasks. He believes that using different types of chips to solve different problems has inherent value.
He said, "It's actually quite appealing to let certain problems be solved in different ways. You can learn something from it."
He also stated that in application areas where TPU can achieve or exceed the performance of NVIDIA chips, TPU has the advantage of higher energy efficiency, which can further reduce usage costs.
Last month, Oracle announced the signing of an agreement that will, starting next year, allow its cloud computing customers to use Google TPU for the first time. Prior to this, Oracle's cloud customers could only choose AI chips provided by NVIDIA and AMD.
Karan Bhatta, Senior Vice President of Cloud Infrastructure at Oracle, stated in a declaration: "As the workload of AI becomes increasingly complex, customers need more options. Therefore, we have partnered with Google. This collaboration will expand our service capabilities and help customers choose the most suitable infrastructure to build and run AI according to their own needs."
Claude Developers of chatbots and other services are also Anthropic customers. According to a recent regulatory document, the company has an option to purchase TPU worth up to $150 billion between 2028 and 2031. Bloomberg previously reported that OpenAI also plans to acquire Google's TPU.
Google itself is also a AI software developer, and this identity has also provided support for its chip business. The TPU development team is able to use Google's latest Gemini models in advance to improve chip efficiency and identify software vulnerabilities, among other issues.
This is crucial for semiconductor design, as defects in chip design can result in billions of dollars in losses and render months of research and development work in vain. Wachdat stated that AI technology is reducing the likelihood of such issues occurring.
However, Google still faces other challenges, including a constant influx of new competitors. A startup company named Tenspresso was co-founded by two former TPU engineers and is currently focused on developing the first chip for long-context window reasoning, targeting the same market as TPU's 8i Max. Nevertheless, according to a report by Bloomberg last week, it will take at least another year for the company's product to be completed.
OpenAI and Anthropic are also advancing their own chip development projects, and their teams include senior engineers who have previously participated in the development of TPU. This will become another competitive pressure that Google will face in the future.
In addition, the risk of a slowdown in infrastructure investment, such as AI, cannot be ignored. Bloomberg Industry Research analyst Singer expects that opposition to data center construction could lead to a reduction of about 5% to 7% in the $1.4 trillion in related expenditures that tech companies were originally expected to make in 2027. These expenditures cover chips, network equipment, land, and other materials required for data center construction.
Singh also pointed out that, compared to NVIDIA, Google's TPU business does not have such high profit margins.
Like the entire chip industry and the AI infrastructure sector, Google is also facing shortages of key components such as memory, as well as insufficient chip manufacturing capacity, which makes it difficult to meet market demand. In its financial report, Google stated that all of the TPU that can be supplied at present have already been reserved by customers.
Wahdat's team is working to expand the sources of their supply chain as much as possible in order to reduce their dependence on a single supplier. This even means that Google will need to find partners other than Broadcom, which is currently responsible for coordinating the manufacturing of TPU.
In recent months, Google has signed a cooperation agreement with Marvell Technology ( Marvell Technology ), a competitor of Broadcom, which involves some work related to TPU. However, not many specific details have been disclosed by either party. Additionally, there have been reports previously that MediaTek ( MediaTek ) is providing support for Google's TPU business.
Wahdat refused to comment on the specific roles played by these two companies in the TPU business.
William Tong, an industry research analyst at Bloomberg, stated that Google is likely in negotiations with suppliers for longer-term cooperation agreements to stabilize the supply chain, "to ensure a stable source of supply and prevent further exacerbation of supply bottlenecks."
