It is the last month of 2025. This year was literally a hectic time, with artificial intelligence (AI) shaking the world morning and night. Like the ghost ship Black Pearl in the movie Pirates of the Caribbean, whenever AI technology news appears, economics, politics, and diplomacy move between this world and the other world.

There are two types of sound: ‘informational sound (signal)’ and ‘disturbing sound (noise)’. Dozens or hundreds of news stories about AI technology developments pour out every day. We have entered an era where finding the true signal among them is directly related to the survival of individuals, companies, and countries.

The year 2025 started with the ‘DeepSeek Typhoon’. On January 28, Chinese AI startup Deepseek unveiled its large-scale language model ‘R1’. DeepSeek claimed that it can learn models at 1/20 the cost of OpenAI. With that one word, the global technology industry and financial markets turned upside down. Immediately after the DeepSeek announcement, NVIDIA’s stock price plummeted 17% in one day, and its market capitalization of 846 trillion won evaporated in just one day. It was the largest decline for a single company in the history of the New York stock market. There was even a prediction that “DeepSeek will mark the end of the Nvidia era.” How is the situation now, 11 months later? Currently, Nvidia’s stock price is up 45% compared to the January crash. Last January’s deep chic shock was noise.

As 2025 comes to an end, the AI semiconductor market is once again in the midst of a ‘Google typhoon’. On November 18, Google unveiled its next-generation AI model ‘Gemini 3’. Google announced that it trained Gemini 3 with its self-produced AI chip, TPU. That overwhelming performance was immediately reflected in the stock price. In the week following the announcement, Google’s stock price soared 12%, ranking third in market capitalization after Nvidia and Apple. It has already been 10 years since Google first unveiled TPU, and this is not the first time it has developed its own AI model using TPU. It is necessary to find out why the Google TPU announced this time is causing such a stir.

From a hardware perspective, it is somewhat natural for TPUs to produce results that are ahead of GPUs in AI learning and inference. GPU was originally developed for drawing graphics on computer monitors, not for AI. The monitor screen is made up of millions of light sources called ‘pixels’, and the color, saturation, and brightness of each pixel must be calculated to construct the screen.

There is an interesting characteristic here. The calculation is simple, but the point to be calculated is that the number of pixels is enormous. Processing this task with the CPU is like using an expensive knife to peel fruit. It’s possible, but it’s inefficient and not appropriate. So GPU was born. The core of a GPU is its structure that deploys a large number of processing cores that perform simple calculations. NVIDIA’s latest GPU, H100, has approximately 15,000 cores.

GPUs simultaneously calculate the color, brightness, and saturation of each pixel with thousands of cores. Instead of one genius mathematician solving thousands of multiplication problems, it is similar to thousands of elementary school students solving each problem one by one. Each core reads the input values and calculation results required for calculation from memory and records them in memory. This computer structure is called von Neumann structure.

If the calculation results of one core must be used as input to another core, the calculation results of one core are stored in memory, and the other core reads the results from memory. When two elementary school students work together to solve a multiplication problem, one student writes the calculation results in a notebook and the other student reads the results written in the notebook and uses them for the next calculation. In this process, time to write and re-read in a notebook inevitably occurs.

The reason TPU is attracting attention is because it is optimized for ‘a structure in which multiple steps of calculation are connected to each other’, which often appears in the AI learning and reasoning process. GPUs can also handle this task, but due to the way GPUs are designed, the results from the previous step must be stored in memory and then loaded again. Memory input/output is holding you back. TPU directly solved the memory bottleneck, which was a limitation of the von Neumann architecture. The TPU uses a structure where the calculation results of one core are immediately passed on to the next core when used, without being recorded in memory. This method is called a systolic structure. This is a concept proposed in 1979 by computer scientists Kung and Leiserson of Carnegie Mellon University in the United States to solve the memory bottleneck problem. When two elementary school students work together to solve a multiplication problem, the person in front records the calculation result in a notebook and calls it out orally to the person behind them without having to read it.

Since the process of writing and reading in a notebook is eliminated, speed, heat generation, and efficiency are all greatly improved. AI learning and inference is essentially a process in which calculation results flow sequentially between layers. The calculation results of one layer are used as input to the next layer, and this process is repeated hundreds or thousands of times. Because of these structural characteristics of GPU and TPU, TPU shows superior characteristics than GPU in various indicators such as performance, power consumption, and heat generation in AI learning/inference.

There are two points to watch here.

One reason is why NVIDIA GPUs are able to maintain a market monopoly even though TPUs are superior to GPUs, and another reason is why Google’s TPUs have potential market destructive power. Coincidentally, the answers to these two reasons are the same.

Many companies, including big tech companies such as Google, Amazon, and Meta, are releasing chips optimized for AI calculations. They are ahead of GPUs in various performance indicators. Nevertheless, Nvidia’s monopoly system remains strong. NVIDIA occupies more than 90% of the market and has an operating profit margin of 70%. The reason is that NVIDIA has a strong moat called CUDA. CUDA is a programming language. It can be seen as a type of language that developers use to instruct AI hardware on what to do.

NVIDIA developed a language called CUDA in 2006 to use its GPUs efficiently. For a long time, CUDA was the only option for companies, research institutes, startups, and developers to learn and infer AI models. Now, thesis implementations, libraries, example codes, academic and industrial infrastructure, and even the engineering talent market are all using CUDA as the standard. CUDA can only be used with NVIDIA GPUs. Because NVIDIA GPUs are positioned at the top of the powerful CUDA ecosystem that has been built for 20 years, they are still able to maintain their monopoly position despite many challenges.

Google’s announcement of this TPU is meaningful in that it shows the possibility of building a new type of AI ecosystem. We demonstrated through Gemini 3.0 that it is possible to learn and deploy cutting-edge models without learning CUDA. Google announced plans to provide TPU as a cloud service and sell the TPU chip itself.

The first practical variable that broke the absolute formula of ‘AI=CUDA=Nvidia’ and showed the possibility of transitioning AI infrastructure to an era of multi-architecture competition is TPU. Thanks to this, companies can now ask the question, rather than “Do we need to follow CUDA for AI,” to “Which is better for our workloads: CUDA or TPU?” This change has strategic significance beyond its technical significance. Because of the CUDA ecosystem, Big Tech had no choice but to bear the burden of GPU supply shortages, price surges, and infrastructure investment (CAPEX). But now there is no need for cloud providers and large customers to go 100% all-in on NVIDIA.

Google has created a bridge across the moat by creating an exquisite crack in the CUDA ecosystem through its TPU. The emergence of Google TPU is the prelude to a huge ecosystem transformation in which the AI semiconductor market shifts from an ‘NVIDIA monopoly system’ to a ‘system in which optimal architectures for each workload compete’. This is a signal that the AI semiconductor market has begun to enter an inflection point, transitioning from a single standard to a multi-architecture competitive landscape.

AI news in 2025 will start with Deep Seek and end with Google. Let me ask the question again. Is the news about Google’s development of TPU technology a signal or noise?

Won Won-jip, Professor at KAIST [email protected]

He is a professor at KAIST and director of the KAIST Storage Research Center. He received a bachelor’s and master’s degree from Seoul National University and a doctorate in computer science from the University of Minnesota. After working at Intel, he began his career as a professor at Hanyang University in 1999, and has been a professor in the Department of Electronic Engineering at KAIST since 2019. He served as the 39th president of the Korean Information Science Society. We succeeded in developing commercial technologies such as file systems for set-top boxes and firmware for flash memory, and contributed to the development of global storage technology. He is a world authority in the field of operating systems. We are working to strengthen the competitiveness of Korea’s system software.

The categories of this article follow the classification of media outlets.

## TPU v5: A Deep Dive into google’s Latest AI Accelerator

Google’s TPU Advances Spark a Revolution in teh AI Semiconductor Ecosystem

H2: breakthrough Architectural Features of the Latest TPU Generation

H3: Unified Matrix Multiply‑Accumulate Engine

Massive parallelism – up to 1.2 Peta‑OPS per TPU v5 pod, a 45 % increase over v4.
Flexible precision – native support for BF16, FP16, INT8, and the newly introduced FP4/INT4 hybrid mode for edge inference.
Zero‑copy data flow – eliminates host‑to‑device memory transfers, reducing latency by up to 30 % for large‑scale training jobs.

H3: Adaptive Power‑Gating & Dynamic Voltage Scaling

Integrated AI‑aware power management lowers idle power to <5 W per core, enabling energy‑efficient inference at the edge.
Thermal‑aware scheduling dynamically redistributes workloads across TPU tiles to maintain optimal temperature under sustained loads.

H3: Enhanced Interconnect Fabric

Silicon‑photonic mesh network provides 500 Gbps bandwidth per link, supporting up to 128‑node TPU pods with sub‑microsecond synchronization.
Co‑located HBM3 memory (up to 64 GB per tile) reduces memory bottlenecks for transformer‑based models.

H2: Impact on the AI Semiconductor Ecosystem

H3: Shifting Competitive Dynamics

NVIDIA vs. Google – TPU’s matrix‑centric design challenges GPU‑centric AI pipelines, prompting NVIDIA to double‑down on tensor cores and DGX optimizations.
AMD & Intel – both companies have accelerated their AI accelerator roadmaps (e.g., AMD Instinct MI300X, Intel Gaudi 3) to match TPU’s energy‑performance ratio.

H3: Ecosystem Expansion via Open‑Source Toolchains

TensorFlow XLA and MLIR now generate native TPU kernels automatically, lowering the barrier for developers unfamiliar with custom ASIC programming.
TPU‑compatible PyTorch extensions (torch‑xla) have seen a 70 % increase in GitHub stars since 2024, reflecting broader community adoption.

H2: Benefits for Developers and Enterprises

H3: Faster Time‑to‑market for AI Models

One‑click TPU pod provisioning on Google Cloud reduces cluster setup from weeks to minutes.
Auto‑tuning compiler (TPU‑AutoTune) selects optimal tile layouts, cutting model‑porting effort by 40 %.

H3: Cost Efficiency & Scalability

Pay‑as‑you‑go pricing for TPU v5 pods (≈$0.32 per TPU‑hour) offers a 20 % cost reduction compared with v4.
horizontal scaling across up to 512 TPU nodes enables training of 1‑trillion‑parameter models within 48 hours, a benchmark previously reachable only with specialized supercomputers.

H2: Practical Tips for Optimizing Workloads on Google TPU

Leverage Mixed‑Precision Training – Use BF16 for forward passes and INT8 for backward passes where model tolerance allows.
Chunk Data to Match Tile Size – Align input batches to multiples of 128 tokens per sequence to maximize matrix utilization.
Utilize TPU‑Specific Profiling Tools – tpu_profiler and tf_summary reveal bottlenecks in memory bandwidth and inter‑tile communication.
Employ Asynchronous Gradient Accumulation – Reduces synchronization stalls when scaling beyond 64 nodes.

H2: Real‑World Use cases Demonstrating TPU‑Driven Innovation

H3: Large‑Scale Language Model (LLM) Training at Scale

Google DeepMind trained a 2.5‑trillion‑parameter transformer on a 256‑node TPU v5 pod, achieving a 2.8× speedup over the previous v4 deployment while cutting energy consumption by 35 %.

H3: Edge AI for Autonomous Vehicles

Waymo integrated TPU v5 edge modules (5 W, 0.8 TOPS) into its sensor‑fusion stack, reducing perception latency from 120 ms to 45 ms, enabling safer real‑time decision making.

H3: Genomics & Drug Revelation

Insilico Medicine utilized TPU‑accelerated convolutional models for protein‑fold prediction, delivering 10× faster inference compared with CPU clusters, accelerating candidate selection cycles.

H2: Future Outlook – What’s Next for Google TPU?

TPU v6 (2026 roadmap) – projected to incorporate 2 nm process technology, integrated on‑chip AI inference accelerator for ultra‑low‑power IoT devices, and native support for Sparse Tensor operations.
Cross‑cloud TPU federation – enabling seamless workload migration between Google Cloud, on‑premise TPU clusters, and third‑party edge devices through the TPU‑Mesh API.
AI‑native security features – hardware‑level encryption of tensor data and real‑time anomaly detection to protect proprietary models in multi‑tenant environments.

Keywords: Google TPU, TPU v5, AI semiconductor ecosystem, AI accelerator, machine learning hardware, AI compute, Google Cloud TPU, TPU pods, inference performance, energy efficiency, AI chip roadmap, edge AI, transformer training, mixed‑precision training, TPU profiling, AI workload optimization.

Google’s TPU Advances Spark a Revolution in the AI Semiconductor Ecosystem

Related