AI Infrastructure Starts with One Question: How Does the GPU Reach Your Application?

Conversations about AI almost always stop at the most visible layer: which model, how many parameters, what the benchmarks say. But the moment a company decides to run AI on its own infrastructure, the first question that determines whether the project moves or stalls is far more mundane: how does the GPU sitting in your server rack actually reach the application that needs it?

The question sounds trivial. In practice, the answer decides how many teams can share a single card, how quickly your model responds, and how much of the most expensive asset in your data center sits idle.

The Stack Nobody Draws in Full

There is a chain worth straightening out first, because it is often simplified in a way that is simply wrong.

An AI model is not, in itself, a program. It is a set of weights: a large file full of numbers. It does nothing until some piece of software loads it into memory and runs the computation. That software is what we call the runtime or inference server, such as Ollama, vLLM, or TensorRT.

That runtime is an ordinary application. Applications need an operating system, the operating system needs a machine, and for AI, that machine needs an accelerator: a GPU. The chain runs from model weights, to runtime, to operating system, to machine, to silicon.

Every joint in that chain is an architectural decision point. The final joint, between the machine and the silicon, is the one most often ignored until the project is already underway.

Why One Application per Physical Server Makes No Sense

Putting one application on one physical server is the most expensive way to run anything. Most of the time, that server sits idle. You pay for power, rack space, cooling, and licensing on capacity nobody uses.

This is where two technologies were born, technologies that are usually mentioned in the same breath even though they work in fundamentally different ways.

Virtualization genuinely splits one physical machine into several virtual machines. Each gets its own kernel, its own operating system, and a hard boundary. What governs that division is the hypervisor, such as KVM, which is part of the Linux kernel itself.

Containerization does not split the machine. This distinction matters and is frequently misunderstood. Every container on a host shares one and the same kernel. What gets separated is only each process's view of the system: which processes are visible, which files are reachable, how much CPU and memory may be used. A container is process isolation, not a new machine. To manage them at scale, you use an orchestrator such as Kubernetes.

This is not a matter of terminology. It directly determines how a GPU can be attached to your workload, because virtual machines and containers see hardware in completely different ways.

For AI, the Economics Invert

Classical virtualization was born from a problem of CPU servers sitting mostly idle. Consolidation was the answer: many small loads stacked onto one large machine.

With AI workloads, the pressure moves. CPU and memory are rarely the bottleneck. What is scarce, expensive, and hard to obtain is the GPU. A single data center class card can be worth as much as several ordinary servers, and lead times are often measured in months rather than days.

The consequence is that the right question is no longer "how do we save on servers" but "how do we make sure the card we already bought is genuinely being used". A GPU idling overnight because one team holds it around the clock is a far more expensive form of waste than an idle CPU server.

That is where virtualization changes role. It stops being a cost-saving tool and becomes an allocation tool: a way to divide your most expensive resource fairly, measurably, and in a way you can bill for.

Two Ways a GPU Reaches a Virtual Machine

Two mechanisms are mature at industry scale, and they answer different needs.

PCI passthrough hands an entire physical card to a single virtual machine. The hypervisor detaches the card from the host and gives it whole to the guest, so the operating system inside the VM talks directly to the silicon. Performance is close to bare metal. The requirement is that your processor and motherboard support IOMMU, meaning Intel VT-d or AMD-Vi, and that it is enabled in both the BIOS and the kernel parameters.

vGPU, or mediated devices, slices one enterprise card into several virtual GPUs. A 16 GB Tesla T4, for example, can be divided into eight 2 GB slices serving eight virtual machines at once. This requires a specific host driver and a license from the GPU vendor.

	PCI Passthrough	vGPU (Mediated Device)
Sharing	One card, one VM	One card, many VMs
Performance	Close to bare metal	Proportional to the slice
Best suited to	Training, fine-tuning, sustained heavy loads	Inference, development, demos, many users
Licensing	None	Vendor vGPU license
Hardware	Any GPU, provided IOMMU is enabled	Generally data center class GPUs

Three technical constraints are worth knowing up front, because they tend to surface late during implementation.

First, one card cannot serve both modes at once. Once a GPU is bound to the passthrough driver, it cannot provide vGPU slices, and the reverse holds too. Second, vGPU does not support hot-plug: adding or removing a slice requires powering the virtual machine down first. Third, on Ampere generation GPUs and newer, SR-IOV virtual functions must be enabled on the host before vGPU profiles appear at all.

So Where Do Containers Fit?

Because containers share the kernel with the host, they do not receive a GPU through passthrough. Containers use the driver already installed on the host, and the container runtime injects access to that device into them.

On the Awanio platform today, the available GPU path sits at the virtual machine layer, in both CEP and Vapor. This is not a limitation that rules out containers, because the pattern is already common in production environments: give the virtual machine a GPU, then run containers inside that virtual machine. You keep container portability while gaining a firmer isolation boundary between teams or tenants, which matters precisely when one card is shared.

How Awanio Handles It

Two Awanio product lines answer this need from two different ends of the scale.

CEP (Cloud Enabler Platform) runs virtual machines on top of Kubernetes. The GPU is attached to the VM at creation time, and the attachment mode follows the device you select: cards marked as shareable are presented as vGPU, while cards that are not shareable are handed over whole as passthrough. Because CEP also carries IAM and billing management, GPU allocation can be treated as a service with an owner and a cost, rather than a device that happens to be bolted into some server.

The video below walks through the whole flow, from provisioning a vGPU-enabled compute instance to Ollama actually serving a model inside it.

Vapor takes the other end: a single host, managed directly. Vapor works on top of KVM, QEMU, and libvirt, and provides both mechanisms. Passthrough is attached as a PCI device, while vGPU is attached as a mediated device. What sets it apart in practice is that the setup work that usually causes the most pain has been turned into a guided flow: upload the vendor's vGPU Manager driver package, and Vapor runs nine readiness checks before installing, covering GPU presence, IOMMU status, kernel headers and build toolchain, Secure Boot, and whether the default driver is still holding the card. This vGPU support landed in Vapor version 3.0.0 in July 2026.

Worth underlining: neither replaces the GPU, and neither removes your need for vendor drivers or licenses. What both do is close the distance between "the card is installed in the server" and "the workload can use it", a distance normally filled with manual, error-prone work.

What Investors See in This Layer

For investors, three things make this layer worth attention, and all three rest on the same reality: the hardware is expensive and scarce.

First, utilization is the most direct lever on returns. GPU capital expenditure happens up front, while the value only materializes through hours of use. A platform that lets one card safely serve many workloads turns an asset previously locked to a single team into capacity that can be sold repeatedly.

Second, this is where the opportunity sits for local infrastructure providers. Domestic demand for AI compute is growing faster than most companies' ability to build and operate GPU clusters themselves. Local cloud providers that can offer GPU as a service, under contracts governed by Indonesian law and with data remaining in Indonesian jurisdiction, answer two needs at once: compute capacity and certainty of control.

Third, the durable advantage at this layer is not the model. Models change every few months. What persists is the ability to allocate, measure, bill, and secure accelerator usage, because that capability stays relevant no matter how many times the model is swapped out.

A Step You Can Schedule This Week

If your company is weighing running AI on its own infrastructure, four steps are enough to turn the discussion into a grounded decision.

Start by checking your hardware readiness: whether IOMMU is enabled, and whether the GPUs you own belong to a class that supports vGPU slicing. Then group your workloads, because training and inference demand different treatment. From there, the choice between handing over a whole card or dividing it into slices answers itself. Finally, run one real workload rather than a synthetic benchmark, and measure what percentage of GPU capacity is genuinely consumed.

That last number is usually the eye-opener, and it almost always becomes the basis for a far healthier budget conversation.

Running AI on your own infrastructure is ultimately a decision about control, much like the other infrastructure decisions we have written about before. The difference is that with AI the stakes are more visible: it is not only where your data is stored, but who decides when you are allowed to compute on it.

---

A visual, interactive version of this piece, a guided flight across seven regions, is available as an interactive AI infrastructure deck.