NVIDIA and AMD GPUs are very similar in both architecture and capability. The reason NVIDIA won the AI market is because NVIDIA GPUs are much easier to program. They have been working on CUDA for over 20 years and have made it a very reliable and solid development platform. AMD’s strategy — as articulated by none other than AMD’s CEO Lisa Su — is to let the open source software community take care of the software problem. It worked in their CPU battle against Intel so it should work in the GPU battle against NVIDIA. Or so she reasoned.

This was a huge strategic miscalculation. The problem is that it’s much more difficult to get high performance out of GPUs than CPUs. In addition, subtle architectural differences make it much more difficult for generic solutions to squeeze every bit of performance out of the devices. To achieve the best results the programmer needs to add special keywords and pragmas into their code, and these are not only language-specific (i.e. CUDA), but also often architecture-specific. Eventually the open-source community might catch up (I was working at a company doing this exact thing), but with the amount of money NVIDIA has, they will continue to advance CUDA to keep the playing field tilted in their favor.

AMD has recently been making more investments in their software. AMD’s answer to CUDA is HIP. I have used both and while HIP has gotten better, it’s still inferior and when AMD’s entire company revenue is less than NVIDIA’s revenue growth over a single quarter, they cannot possibly keep up. Even so, sooner or later HIP will be good enough that they will be competitive. People I know at AMD think that is already the case, but I’m skeptical.

The other problem that exists is that ALL of the new AI development is being done in CUDA for the obvious reason that almost all GPUs out there are NVIDIA. If you are developing a new AI model or a new AI algorithm and you need to create kernels to run on the GPU in order to be competitive, you are going to write that in CUDA. There are probably thousands of kernels out there that are written in CUDA. To address this, AMD has a tool called Hipify that will convert CUDA code into HIP code. This works up to a point, but as I mentioned above, there are not only syntactical differences, but there are often very architecture-specific algorithms that won’t directly translate from one GPU to another. AMD has a team of people working on implementing the most important of those kernels, but realistically when there are thousands or millions of people writing CUDA kernels, they will never be able to keep up.

So the summary is that it’s not a hardware problem. It’s a software problem. NVIDIA has a much larger software team than AMD has in their entire engineering group. They can’t keep up. AMD will be able to eat into NVIDIAs market, but barring NVIDIA making the kind of mistakes that Intel made to allow AMD to catch up in the CPU market, NVIDIA will always be the 900 lb gorilla in the GPU market and AMD will be getting table scraps.

paris.us