Why GPUs are essential for AI and high-performance computing

Graphics processing units (GPU) have become the foundation of artificial intelligence. Machine learning was slow and inadequate for many of today's applications. The inclusion and utilization of GPUs made a remarkable difference to large neural networks. Deep learning discovered solutions for image and video processing, putting things like autonomous driving or facial recognition into mainstream technology.

The connection between GPUs and Red Hat OpenShift does not stop at data science. High-performance computing is one of the hottest trends in enterprise tech. Cloud computing creates a seamless process enabling various tasks designated for supercomputers, better than any other computing power you use, saving you time and money.

How GPUs work

Let’s back up and make sure we understand how GPUs do what they do.

The term, graphics processing unit, was popularized in 1999 when Nvidia marketed its GeForce 256 with the capabilities of graphics transformation, lighting, and triangle clipping. These are math-heavy computations, which ultimately help render three-dimensional spaces. The engineering is tailored towards these actions, which allows processes to be increasingly optimized and accelerated. Performing millions of computations or using floating point values creates repetition. This is the perfect scenario for tasks to be run in parallel.

GPUs can dominate dozens of CPUs in performance with the help of caching and additional cores. Imagine we are attempting to process high-resolution images. For example, if one CPU takes one minute to process a single image, we would be stuck if we needed to go through nearly a million images for a video. It would take several years to run on a single CPU.

Scaling CPUs will linearly speed up the process. However, even at 100 CPUs, the process would take over a week, not to mention adding quite an expensive bill. A few GPUs, with parallel processing, can solve the problem within a day. We made impossible tasks possible with this hardware.

The evolution of GPUs

Eventually, the capabilities of GPUs expanded to include numerous processes, such as artificial intelligence, which often requires running computations on gigabytes of data. Users can easily integrate high-speed computing with simple queries to APIs and coding libraries with the help of complementary software packages for these beasts.

In November 2006, NVIDIA introduced CUDA, a parallel computing platform and programming model. This enables developers to use GPUs efficiently by leveraging the parallel compute engine in NVIDIA’s GPUs and guiding them to partition their complex problems into smaller, more manageable problems where each sub-problem is independent of the other's result.

NVIDIA further spread its roots by partnering with Red Hat OpenShift to adapt CUDA to Kubernetes, allowing customers to develop and deploy applications more efficiently. Prior to this partnership, customers interested in leveraging Kubernetes on top of GPUs had to manually write containers for CUDA and software to integrate Kubernetes with GPUs. This process was time-consuming and prone to errors. Red Hat OpenShift AI simplified this process by enabling the GPU operator to automatically containerize CUDA and other required software when a customer deploys OpenShift with a GPU server. 

What companies create GPUs?  NVIDIA is the most widely recognized name and also holds the majority of the market share.  But AMD and Intel (Habanna) have also entered the GPU card arena.  

GPUs in Red Hat OpenShift AI

Red Hat OpenShift AI (RHOAI) expanded the mission of leveraging and simplifying GPU usage for data science workflows. Now when customers create a project workbench on RHOAI, they have the option to customize the number of GPUs required for their workflow along with Pytorch, TensorFlow and/or CUDA GPU-enabled images.  

Note:  currently we only support the addition of NVIDIA GPUs, but we are working with AMD to add AMD GPUs.  

You may be able to select 1 or more GPUs, depending on the GPU machine pool added to your cluster. Customers have the power to use GPUs in their data mining and model processing tasks. 

Need to train a large dataset or model that may tax a GPU?  Then consider using Distributed Workloads with your GPUs.  ‘Workload distribution refers to the process of evenly allocating tasks or requests among multiple servers or components to optimize performance and efficiency in a computer system.’    

Why use distributed workloads?

1. You can iterate faster and experiment more frequently because of the reduced processing time.
2. You can use larger data sets, which can lead to more accurate models.
3. You can use complex models that could not be trained on a single node.
4. You can submit distributed workloads at any time, and the system then schedules the distributed workload when the required resources are available.²

1 AI generated definition based on: Architecting High Performing, Scalable and Available Enterprise Web Applications, 2015

For more information on how to configure distributed workloads with Red Hat OpenShift AI check out our Configure distributed workloads instructions.