Study: Nvidia & Developers
Preface: We approach this study not from a technical lens, but a generalist viewpoint.
AI Fundamental Problem
AI fundamentally is a compute problem.
To solve this compute problem, you first have to win over developers. And that’s exactly what Nvidia has strategically been doing.
As much as Nvidia is very good at designing GPUs, it is also one of the most formidable “software” companies. It is increasingly far from being just a chip company, with the software being at least one of its most important economic advantages.
Tech Value Stacks
Let’s start by going back to pre-1990s and examine how the technology value stack looked like. In this early PC era, as we move up the stack # we can rank them from least to most profitable:
Stack #1: Hardware (Intel, Micron, Seagate)
Stack #2: Devices (Apple, Dell, Compaq)
Stack #3: Network (Cisco, 3Com)
Stack #4: Operating System (Microsoft, Apple, Linux)
Stack #5: Applications (Microsoft Office)
As time progressed and the tech world moved forward, the highest rank stacks belonged to companies that extracted the highest margin, while hardware and devices margins were competed away.
Then came the Internet-era during the 2000s. It was a period of major changes, where widespread of content and services would become possible at low incremental costs. Thus, the tech value stack changed too:
Stack #1: Internet Service Provider (Verizon)
Stack #2: Hardware (Cisco)
Stack #3: Software Infrastructure (Netscape, RedHat)
Stack #4: Content (Yahoo, MSN, AOL)
Stack #5: Services (Amazon, Google)
Following the same logic, as time went on the higher stacks were the ones getting most of the economic surplus that the Internet brought along. The bottom stacks were prone to commodification as technology became better over time.
Today, we have an AI-era compute tech wave. Nvidia, besides also starting in the lower stack, has managed to produce margins above 80%, which traditionally great software companies in the upper stacks enjoy.
The current value stack looks something like this, with Nvidia positioned at the bottom and top of the value chain:
Stack #1: Energy Providers & Chips (Nvidia)
Stack #2: Cloud Compute
Stack #3: Data
Stack #4: Software Infrastructure (Nvidia)
Stack #5: Applications
Jensen Huang has strategically constructed a computing strategy that is both hardware and software dominant.
It goes back to our statement at the start: to win the compute battle, you first win over developers.
CUDA
Over the past 2 decades, Nvidia’s focus has long been the developers who build AI systems and other software with its chips.
In the longer term, this is like a walled garden, within which houses an army of developers stuck to its software ecosystem.
For competitors, the battle is likely to focus on the company’s coding prowess, not just its circuitry design.
What’s this software we are talking about that has existed for almost 20 years?
No surprises: it’s CUDA (Compute Unified Device Architecture).
When it launched in 2007, this platform was a solution to a problem no one had yet: how to run non-graphics software, such as encryption algorithms and cryptocurrency mining, using Nvidia’s specialized chips, which were designed for compute-intensive applications like graphics and videogames.
CUDA enabled all kinds of other computing on those GPUs, among the applications was AI software.
But there is actually many things beyond CUDA that further protects Nvidia. The hundreds of open-source software kernels across dozens of vertical applications, from healthcare to manufacturing, to robotics and more.
Year after year, Nvidia responded to the needs of software developers by releasing specialized libraries of code, allowing a huge array of tasks to be performed on its GPUs at speeds that were impossible with general-purpose processors like those made by Intel and AMD.
Here are some useful libraries that are very popular among developers:
2013: cuBLAS – Linear algebra operators.
2013: cuFFT – Fourier Transform for signal processing.
2014: cuDNN – Deep Neutral Network libraries.
2016: NCCL (NVIDIA Collective Communications Library) – Optimized routines for high-bandwidth, low-latency communication.
2017: TensorRT – Deep learning inference libraries.
Nvidia is fast becoming the operating system (OS) of the AI industry, just like how Microsoft and Apple have their high margins OS installed on every device.
This approach also explains why more than half of Nvidia engineers are software people instead of hardware.
Every time a rival announces competing AI chips, it is up against Nvidia’s systems that developers have been using for more than 15 years to write mountains of code. That software can be difficult to shift to a competitor’s system.
Compatible Systems
Nvidia is a major contributor to the open-source community in many other areas, such as specific AI models and frameworks (cuOpt, AgentIQ, PyTorch) and even open-sourced its GPU kernel modules for drivers starting in 2024. However, this does not extend to the core components of the CUDA compiler that are essential for their hardware lock-in.
This combination of optimized code libraries, open-source compilers and closed-source CUDA produce a unified backward compatible architecture. An bad example would be the x86 architecture (Intel) which makes maintaining backward compatibility one of the most challenging aspects of computer architecture.
Developers love compatible systems and that’s what Nvidia achieved with CUDA. Put together, Nvidia can maintain backward compatibility at the developer level, but break compatibility at the microarchitectural level bridged using their driver and compiler. This differentiates them from Intel, who for the most part were dependent on Windows developers doing a good job at using their architecture.
As proof of success, CUDA now includes more than 300 code libraries and 600 AI models, and supports 3,700 GPU-accelerated applications used by more than 5 million developers at roughly 40,000 companies!
Full Stack Solutions
Jensen has been playing the long run game with AI for quite some time now. Way back in 2017, he already declared that AI is going to eat up software, this was even before Google researchers published the famous Attention is All You Need paper.
Today, Nvidia sells entire systems to fill AI datacenters and increasingly Nvidia rents its systems both directly and indirectly via its AI cloud datacenter partners like AWS, Azure, Google Cloud, Oracle Cloud and CoreWeave.
This gives them control of a bigger portion of the computing stack, and allows for more optimization; in the end what developers care about is how fast are their workloads running.
Vertical integration as an architecture company also buys time, even if Nvidia GPU architecture is not the best for a generation, the full solution they provide might still be better than anything on offer; a full solution is much easier for enterprises to deploy.