Create Workload

To run a workload on a TensorFusion vGPU, add the label tensor-fusion.ai/enabled: "true" and a few resource annotations to the Pod template. The TensorFusion webhook then injects what the Pod needs and the TensorFusion scheduler places it on a GPU. You don't change the image or the application code.

This guide takes a Deployment from sizing to a verified vGPU in four steps.

Prerequisites

• TensorFusion is installed and at least one GPUPool is running. See Kubernetes Install.
• kubectl access to the namespace where the workload runs.
• The pool has discovered GPUs. Check with:

kubectl get gpupool
kubectl get gpu -o custom-columns='NAME:.metadata.name,MODEL:.status.gpuModel,CAPACITY:.status.capacity,AVAILABLE:.status.available'

CAPACITY shows the TFLOPS and VRAM of each card. You need these numbers in step 1.

Step 1. Size the request

A vGPU is described by two resources. Each has a request, which the scheduler reserves, and a limit, which the workload cannot exceed.

If the pool contains a single GPU model, you can state compute as a percentage of one card with compute-percent-request and compute-percent-limit instead of TFLOPS. Percentages are not counted against namespace GPU quotas, and you cannot combine them with tflops-* on the same Pod.

If you cannot measure up front, start with a generous request and turn on autoscaling to let TensorFusion correct it from actual usage.

Step 2. Choose a QoS level

The QoS level has these effects:

• Scheduling priority. A high or critical Pod that has no priorityClassName of its own gets the tensor-fusion-high or tensor-fusion-critical PriorityClass, so it can preempt lower-priority Pods when GPUs are full.
• Autoscaling. With autoscaling on, the request follows the lower bound of observed usage for low and medium, the target for high, and the upper bound for critical.
• Billing. Pool administrators can set a price per QoS level. See Config QoS and Billing.

If you leave tensor-fusion.ai/qos out, the webhook picks high when both requests equal their limits, otherwise the pool's qosConfig.defaultQoS, otherwise medium.

Step 3. Add the label and annotations

Put the label and annotations on the Pod template, not on the Deployment itself.

# pytorch-vgpu.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pytorch-vgpu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pytorch-vgpu
  template:
    metadata:
      labels:
        app: pytorch-vgpu
        tensor-fusion.ai/enabled: "true"
      annotations:
        tensor-fusion.ai/tflops-request: "10"
        tensor-fusion.ai/tflops-limit: "20"
        tensor-fusion.ai/vram-request: "4Gi"
        tensor-fusion.ai/vram-limit: "4Gi"
        tensor-fusion.ai/qos: "medium"
    spec:
      containers:
        - name: pytorch
          image: pytorch/pytorch:2.4.1-cuda12.1-cudnn9-runtime
          command: ["sh", "-c", "sleep infinity"]

kubectl apply -f pytorch-vgpu.yaml

What the annotations you left out default to:

For every annotation, see the annotation reference. For manifests that cover other cases, such as remote vGPU, a specific GPU model or several GPUs, see Annotation usage scenarios.

Optional: share settings with a WorkloadProfile

When several workloads use the same settings, put them in a WorkloadProfile in the workload's namespace and reference it by name. Annotations on the Pod override the profile.

apiVersion: tensor-fusion.ai/v1
kind: WorkloadProfile
metadata:
  name: small-inference
spec:
  resources:
    requests:
      tflops: "10"
      vram: 4Gi
    limits:
      tflops: "20"
      vram: 4Gi
  qos: medium
  gpuCount: 1

  template:
    metadata:
      labels:
        tensor-fusion.ai/enabled: "true"
      annotations:
        tensor-fusion.ai/workload-profile: small-inference

The pool, local or remote mode, and isolation mode are always taken from annotations or pool defaults, not from the profile.

Step 4. Verify

1. Check that the Pod is running and was handled by TensorFusion:

   kubectl get pod -l app=pytorch-vgpu -o wide
   kubectl get tensorfusionworkload pytorch-vgpu

   The Pod is Running, and a TensorFusionWorkload named after the Deployment exists.

2. Check which GPU was assigned. In local mode the scheduler records it on the Pod:

   POD=$(kubectl get pod -l app=pytorch-vgpu -o jsonpath='{.items[0].metadata.name}')
   kubectl get pod "$POD" -o jsonpath='{.metadata.annotations.tensor-fusion\.ai/gpu-ids}{"\n"}'

3. Check the GPU from inside the container:

   kubectl exec "$POD" -c pytorch -- nvidia-smi
   kubectl exec "$POD" -c pytorch -- python3 -c 'import torch; print(torch.cuda.device_count()); print(torch.ones(1024, device="cuda").sum().item())'

   nvidia-smi runs and reports a total memory close to your vram-limit instead of the full card. PyTorch sees one device and completes the computation. A Pod that is merely Running has not proven the GPU path, so always run a real CUDA call.

4. Check that the pool accounted for the request:

   kubectl get gpu -o custom-columns='NAME:.metadata.name,CAPACITY:.status.capacity,AVAILABLE:.status.available'

   AVAILABLE on the assigned GPU dropped by the request.

If something goes wrong

The webhook rejects invalid Pods at creation. For a Deployment, the error shows up in the ReplicaSet events: kubectl describe rs -l app=pytorch-vgpu.

For more cases, see the troubleshooting handbook.

Next steps

• Annotation usage scenarios: manifests for common requirements.
• Configure AutoScaling: let TensorFusion adjust requests and limits from usage.
• Migrate Existing Workload: move workloads that request nvidia.com/gpu today.
• Annotation reference: every annotation, with defaults and allowed values.