Config QoS and Billing

TensorFusion has four QoS levels: low, medium, high and critical. The level of a workload decides its scheduling priority and the unit price used to compute its cost. This guide shows how to assign QoS levels, set a pool's default level and prices, set the hourly cost of each GPU model, and query the resulting cost data.

The cost figures are for internal chargeback and showback. TensorFusion records them as metrics. It does not issue invoices or enforce budgets.

Prerequisites

• TensorFusion is installed and at least one GPU pool is Running. See Kubernetes Install.
• You can edit the TensorFusionCluster resource.
• To read cost data, you can query GreptimeDB. See Query Metrics & Logs.

What a QoS level changes

Notes on the table:

• The webhook adds the PriorityClass only when the Pod has no priorityClassName of its own. With a PriorityClass, the scheduler can preempt lower-priority Pods when GPUs run out. low and medium Pods keep the cluster's default priority.
• Each level can have its own unit price. See Set the default QoS and unit prices of a pool.

Assign a QoS level to a workload

Set the annotation tensor-fusion.ai/qos on the Pod template:

apiVersion: apps/v1
kind: Deployment
# ...
spec:
  template:
    metadata:
      labels:
        tensor-fusion.ai/enabled: "true"
      annotations:
        tensor-fusion.ai/qos: "high"
        tensor-fusion.ai/tflops-request: "100"
        tensor-fusion.ai/tflops-limit: "200"
        tensor-fusion.ai/vram-request: "8Gi"
        tensor-fusion.ai/vram-limit: "8Gi"

If you use a WorkloadProfile, set spec.qos there instead. The annotation on the Pod wins over the profile.

Without an explicit level, TensorFusion picks one in this order:

1. high, if the TFLOPS and VRAM requests both equal their limits and are not zero.
2. The pool's qosConfig.defaultQoS, if set.
3. medium.

critical is never assigned automatically, because it can preempt other workloads.

Check the result on a running Pod:

kubectl get pod <pod-name> -o jsonpath='{.metadata.annotations.tensor-fusion\.ai/qos}{"\n"}{.spec.priorityClassName}{"\n"}'

Protect a new Pod from preemption

A Pod with the annotation tensor-fusion.ai/eviction-protection cannot be preempted for the given time after its creation. The value is a duration such as 30m or 2h:

metadata:
  annotations:
    tensor-fusion.ai/eviction-protection: "2h"

During the protection period, the Pod's cost is multiplied by the pool's evictionProtectionPriceRatio. critical Pods are not charged this surcharge.

Set the default QoS and unit prices of a pool

QoS settings and prices belong to a GPU pool. Edit qosConfig of the pool in the TensorFusionCluster:

kubectl edit tensorfusioncluster tensor-fusion

apiVersion: tensor-fusion.ai/v1
kind: TensorFusionCluster
metadata:
  name: tensor-fusion
spec:
  gpuPools:
    - name: shared
      isDefault: true
      specTemplate:
        # ...
        qosConfig:
          defaultQoS: medium
          evictionProtectionPriceRatio: "1.2"
          pricing:
            - qos: low
              requests:
                perFP16TFlopsPerHour: "0.0035"
                perGBOfVRAMPerHour: "0.008"
              limitsOverRequests: "0.5"
            - qos: medium
              requests:
                perFP16TFlopsPerHour: "0.0069228"
                perGBOfVRAMPerHour: "0.01548"
              limitsOverRequests: "0.5"
            - qos: high
              requests:
                perFP16TFlopsPerHour: "0.0069228"
                perGBOfVRAMPerHour: "0.01548"
              limitsOverRequests: "1"
            - qos: critical
              requests:
                perFP16TFlopsPerHour: "0.0104"
                perGBOfVRAMPerHour: "0.0232"
              limitsOverRequests: "1"

The prices in this example are placeholders. Replace them with your own rates.

Three things to know:

• A pool without qosConfig records a cost of 0 for all its workers. The pool created by the Helm chart has no qosConfig, so add one before you rely on cost data.
• A QoS level without an entry in pricing is also recorded as 0. Add an entry for every level you use.
• qosConfig.definitions accepts a priority per level, but the current scheduler does not read it. Scheduling priority comes from the PriorityClass described above.

How the cost of a worker is computed

For each worker, per hour and per GPU:

cost = tflopsRequest × tflopsPrice
     + vramRequestGiB × vramPrice
     + (tflopsLimit − tflopsRequest) × tflopsPrice × limitsOverRequests
     + (vramLimitGiB − vramRequestGiB) × vramPrice × limitsOverRequests

The result is multiplied by the number of GPUs the worker uses, and by evictionProtectionPriceRatio while the Pod is under eviction protection.

Example: a medium worker requests 100 TFLOPS and 8 GiB, with limits of 200 TFLOPS and 8 GiB. With the medium prices above, it costs 100 × 0.0069228 + 8 × 0.01548 + 100 × 0.0069228 × 0.5 ≈ 1.16 per hour.

Cost follows the requests and limits a worker holds, not its measured usage. An idle worker costs the same as a busy one.

Set the hourly cost of each GPU model

Node cost is computed from the hourly cost of each GPU model. These costs live in the ProviderConfig of the vendor, under spec.hardwareMetadata[].costPerHour. For NVIDIA, the Helm chart creates the ProviderConfig from providerConfigs.nvidia.hardwareMetadata, with reference prices for common models.

To use your own prices, copy the whole hardwareMetadata list from the chart's values.yaml into your values file and edit costPerHour. Helm replaces lists as a whole, so a partial list would drop the other models.

# my-values.yaml
providerConfigs:
  nvidia:
    hardwareMetadata:
      # ... all other models copied from values.yaml ...
      - model: T4
        fullModelName: "Tesla T4"
        costPerHour: "0.53"
        fp16TFlops: "65"

Apply it with helm upgrade, then check the resource:

kubectl get providerconfig nvidia-provider -o yaml | grep -B2 -A2 'Tesla T4'

The controller picks up the new price without a restart.

Older versions read GPU prices from the ConfigMap tensor-fusion-sys-public-gpu-info. The chart still installs that ConfigMap, but the current controller takes prices from ProviderConfig only. Editing the ConfigMap has no effect on cost.

Query cost data

Costs are written to GreptimeDB in the raw_cost column. Each row holds the cost of the interval since the previous row, so sum the column over the period you want.

Cost per namespace and QoS level over the last 24 hours:

SELECT namespace, qos, SUM(raw_cost) AS cost
FROM tf_worker_resources
WHERE greptime_timestamp >= now() - '24h'::INTERVAL
GROUP BY namespace, qos
ORDER BY cost DESC;

Cost of GPU nodes per pool over the same period:

SELECT pool, SUM(raw_cost) AS cost
FROM tf_node_metrics
WHERE greptime_timestamp >= now() - '24h'::INTERVAL
GROUP BY pool;

Compare the two to see how much of what the GPUs cost is charged back to workloads.

Next steps

• Workload annotations reference
• GPUPool schema, including every qosConfig field
• GPUResourceQuota schema, to cap GPU resources per namespace
• Metrics