TensorFusion DocsCompare with HAMi
TensorFusion and HAMi compared along eleven dimensions, from architecture and isolation to elasticity, cost attribution, measured overhead and licensing, and when to choose each
HAMi (Heterogeneous AI Computing Virtualization Middleware) is an open source Kubernetes middleware for sharing GPUs and other accelerators, and a CNCF Incubating project. Of all the solutions compared in this section, it overlaps the most with TensorFusion: both slice a GPU by compute and memory for containers, both enforce the limits with a library loaded into the container, and both ship a GPU-aware scheduler.
The short version:
- HAMi is stronger on accelerator coverage (more than ten vendors), on openness (every component Apache 2.0, under CNCF governance), on community size, and on fitting into existing Kubernetes conventions (standard extended resources,
ResourceQuota, Volcano, Kueue, Koordinator and KAI Scheduler, and a DRA driver). - TensorFusion is stronger on what happens around the slice: separate requests and limits with QoS levels, limits changed while the workload runs, process-level and remote isolation, GPU-over-IP, GPU node autoscaling with Karpenter, usage-based right-sizing, and per-workload cost data.
Statements about TensorFusion describe the current release. Statements about HAMi come from its public documentation and releases, checked in October 2026; see References.
At a glance
| Dimension | TensorFusion | HAMi |
|---|---|---|
| Architecture and footprint | One controller (webhook, scheduler, CRD controllers in one process), a hypervisor DaemonSet with its own device plugin, and GreptimeDB for metrics. | A mutating webhook, a scheduler extender next to kube-scheduler, and a device plugin per vendor. |
| Isolation | Four modes: shared, soft (in-process limiter), hard (separate worker process), partitioned (MIG, Ascend vNPU). Remote mode adds a node boundary. | HAMi-core: an in-process library that intercepts CUDA calls. Dynamic MIG on NVIDIA. |
| Granularity and resource model | Compute in TFLOPS or percent, VRAM in bytes; request and limit set separately. | Memory in MiB or percent, cores in percent, plus a device count; one value per resource. |
| Scheduling | Own scheduler built on kube-scheduler: bin-packing or spreading, NUMA and peer topology, gang scheduling, namespace quotas, four QoS levels. | Scheduler extender: binpack, spread and mutex policies at node and GPU level, topology and NUMA awareness, ResourceQuota on GPU memory, PodGroup gang scheduling (v2.10). Queues through Volcano, Kueue or KAI Scheduler. |
| Elasticity | GPU nodes added and removed through Karpenter or cloud APIs, node compaction, requests and limits right-sized from usage, compute and VRAM oversubscription per pool. | No node provisioning or right-sizing built in (Cluster Autoscaler support is at the design stage). Device memory scaling above 1 (experimental). |
| Remote GPU and VMs | GPU-over-IP for Pods on CPU nodes; host-to-VM sharing (HyperGPU), including Windows guests, for compute APIs. | Not supported. |
| Hardware coverage | NVIDIA in all modes; Huawei Ascend, Moore Threads, Alibaba PPU in a subset. | NVIDIA, AMD Instinct, Ascend, Cambricon, Hygon, Iluvatar, Kunlunxin, MetaX, Moore Threads, Enflame, AWS Neuron, Biren, Vastai; capabilities vary by vendor. |
| Observability and cost | Per-workload compute and VRAM usage, alert rules, unit prices per QoS level and GPU model, cost metrics for chargeback and showback, optional cloud console. | Prometheus metrics, Grafana dashboards, HAMi-WebUI. No cost attribution. |
| Performance overhead | soft without a limit: within 1% of native, measured on a T4. API remoting (hard, remote): depends on the workload, see below. | Upstream benchmark: vLLM on an A100, v2.9 within about 1–3% of native latency. |
| Compatibility and migration | Runs beside the NVIDIA GPU Operator; can take over Pods that request nvidia.com/gpu. | Uses NVIDIA's container toolkit; replaces the NVIDIA device plugin on the nodes it manages. |
| License and support | Controller, scheduler, hypervisor: Apache 2.0. Vendor limiters and the remote client/worker: closed source, free under 10 GPUs. Commercial support by NexusGPU. | Apache 2.0, all components. CNCF Incubating since July 2026. Enterprise edition and support from Dynamia. |
Feature checklist
This checklist updates the feature matrix from earlier versions of this page. "Partial" is explained in the section linked from each dimension.
| Feature | TensorFusion | HAMi |
|---|---|---|
| Sharing | ||
| Fractional GPU, VRAM limit | ✅ | ✅ |
| Fractional GPU, compute limit | ✅ | ✅ |
| Separate request and limit (burst above request) | ✅ | ❌ |
| Limits changed while the Pod runs | ✅ | ❌ |
| Process-level isolation (GPU calls outside the app process) | ✅ hard | ❌ |
| Hardware partitions (MIG and similar) | ✅ MIG, Ascend vNPU | ✅ Dynamic MIG, vendor templates |
| Multiple GPUs per Pod | ✅ | ✅ |
| Oversubscription | ✅ compute and VRAM | Partial, memory scaling (experimental) |
| Host memory or disk counted as schedulable VRAM | ✅ | ❌ |
| Remote GPU (GPU-over-IP) | ✅ mainly NVIDIA | ❌ |
| VM guests using host GPUs | ✅ compute APIs, Linux and Windows | ❌ |
| Scheduling | ||
| GPU-aware scheduler | ✅ built-in | ✅ extender |
| Bin-packing and spreading | ✅ | ✅ |
| GPU topology awareness | ✅ | ✅ |
| Gang scheduling | ✅ built-in | ✅ PodGroup (v2.10), or through Volcano |
| Namespace GPU quotas | ✅ GPUResourceQuota | ✅ ResourceQuota on GPU memory and cores |
| QoS levels and priority | ✅ four levels | Partial, two priority levels on NVIDIA |
| Dynamic Resource Allocation (DRA) | ❌ | ✅ separate HAMi-DRA driver |
| Operations | ||
| GPU node autoscaling (Karpenter, cloud APIs) | ✅ | ❌ |
| Node compaction and defragmentation | ✅ | ❌ |
| Usage-based right-sizing of GPU limits | ✅ | ❌ |
| Metrics and dashboards | ✅ | ✅ |
| Alert rules | ✅ built-in | Partial, bring your own Prometheus rules |
| Per-workload cost attribution | ✅ | ❌ |
| Web console | ✅ optional cloud console | ✅ HAMi-WebUI |
| GPU live migration | Commercial edition | ❌ |
| Platform | ||
| Vendors beyond NVIDIA | Partial, three | ✅ more than ten |
| Fully open source | Partial | ✅ |
| Foundation governance | ❌ | ✅ CNCF |
Architecture and installation footprint
HAMi follows the extension points Kubernetes already has. Its Helm chart installs a mutating webhook, a scheduler extender that runs beside a kube-scheduler, a device plugin for each vendor you enable, and a monitor that exports per-container metrics. GPU nodes are labeled gpu=on to opt in, and on NVIDIA nodes the NVIDIA container runtime must be the default runtime (or CDI is used). Nothing else is required, which keeps the install small and easy to reason about. For clusters moving to Dynamic Resource Allocation, HAMi also ships a separate DRA driver and webhook (HAMi-DRA, Kubernetes 1.34 or later).
TensorFusion runs more of the stack itself. One Helm chart deploys:
- the controller, a single process that holds the CRD controllers, the Pod webhook, and a scheduler built on kube-scheduler (GPU Pods use it; other Pods keep the default scheduler);
- a hypervisor DaemonSet on every GPU node, which discovers devices, runs TensorFusion's own device plugin, and writes each Pod's limits;
- GreptimeDB and Vector for metrics and logs, and optionally an agent that connects the cluster to the cloud console.
The extra components carry the features HAMi leaves to other projects: node provisioning, autoscaling, metrics, and cost data. If you only need local fractional GPUs, HAMi has less to operate.
Isolation
Both projects enforce software limits with a library inside the container, but what surrounds that library differs.
| TensorFusion | HAMi | |
|---|---|---|
| VRAM | Enforced per Pod in every mode. In soft, the limiter rejects allocations above the limit. | Enforced by HAMi-core, which intercepts CUDA memory allocation calls and returns out-of-memory above the limit. NVML is hooked too, so nvidia-smi in the container shows the limit. |
| Compute | soft: a token bucket in the limiter, refilled by a control loop in the node hypervisor that tracks device busy time. hard: the worker gives the workload a fixed share of SMs. | HAMi-core throttles cuLaunchKernel with a token bucket; a background thread samples GPU utilization and refills it, so usage converges to the core percentage. |
| Limits changed at runtime | Yes. The hypervisor rewrites the quota in shared memory; autoscaling, cron rules, and QoS use this. | No. Limits are fixed when the container starts. |
| Fault and security isolation | soft is not a security boundary. hard moves GPU calls into a separate worker process; remote mode puts that worker on another node; partitioned gives a hardware boundary. | HAMi-core is in-process; its docs describe enforcement as soft and noise isolation as best-effort, and it can be bypassed by direct driver calls. Dynamic MIG provides the hardware boundary on Ampere, Hopper and Blackwell. |
| Images | soft needs a glibc-based image (the limiter loads through /etc/ld.so.preload). | HAMi-core is also loaded through /etc/ld.so.preload; same constraint. |
Measured on a T4 (October 2026): a 50% soft compute limit held device busy time at 53%, and 25% at 26%; changing a running Pod from 100% to 30% took effect within the cron window and recovered after it. Throughput is not linear in busy time, so a 50% limit can yield more than 50% of full-load throughput; see the performance section.
Sharing granularity and resource model
HAMi keeps the standard Kubernetes resource syntax: nvidia.com/gpu for a device count, nvidia.com/gpumem or nvidia.com/gpumem-percentage for memory, and nvidia.com/gpucores for a percentage of compute. Tools that understand extended resources keep working, and ResourceQuota can count them.
TensorFusion describes resources in Pod annotations or a WorkloadProfile:
- Compute can be set in TFLOPS. The scheduler knows each GPU model's FP16 TFLOPS, so "10 TFLOPS" is the same amount of work on a T4 and an A100, while "30%" is not. Percent is also supported.
- Each resource has a request and a limit. The scheduler packs by request and the limiter enforces the limit, which is how Kubernetes treats CPU and memory.
- A workload can be retargeted to another GPU model without rewriting its numbers.
Scheduling
| TensorFusion | HAMi | |
|---|---|---|
| Integration | Own scheduler profile (tensor-fusion-scheduler) built from kube-scheduler with GPU plugins. | Scheduler extender called by kube-scheduler (hami-scheduler); also a vGPU device plugin for Volcano, a Kueue integration, and a resource isolator for KAI Scheduler. |
| Placement | Bin-packing or spreading per pool, by node and by GPU. | binpack or spread, separately for nodes and GPUs, settable per Pod; mutex for a card nobody else uses. |
| Topology | NUMA and GPU peer-link aware selection for multi-GPU Pods. | Topology-aware policy from nvidia-smi topo on NVIDIA, vendor topology for Kunlunxin, Enflame and MetaX, optional CPU/GPU NUMA alignment (v2.10). |
| Gang scheduling | Built in. | PodGroup support since v2.10; queues and fairness through Volcano or Kueue. |
| Quotas | GPUResourceQuota per namespace, in TFLOPS and VRAM. | ResourceQuota with limits.nvidia.com/gpumem and similar, checked by the webhook. |
| Priority | Four QoS levels, mapped to PriorityClasses and to the share of spare compute. | nvidia.com/priority: a high-priority task pauses a low-priority one on the same GPU. |
| Device selection | Vendor, GPU model, isolation mode, pool. | GPU type and UUID, by annotation. |
If your cluster already runs Volcano, Kueue or KAI Scheduler for queues and fairness, HAMi plugs into it. TensorFusion's scheduler replaces that layer for GPU Pods rather than plugging into it.
Elasticity
This is where the two differ most.
- GPU node autoscaling. TensorFusion creates GPU nodes when Pods are pending, through Karpenter
NodeClaims or cloud provider APIs, and releases nodes that compaction has emptied. On Alibaba Cloud ACK with Karpenter and T4 nodes, three 20% Pods triggered one node instead of three and the run cost 72% less than the whole-GPU baseline. HAMi has no node provisioning; it works with whatever nodes you or the cluster autoscaler provide. Its v2.10 release added a design for Cluster Autoscaler scale-up simulation, and autoscaling is on its roadmap. - Vertical autoscaling. TensorFusion recommends and applies GPU requests and limits from usage history, and supports cron rules that change limits for a time window, in place, without restarting the Pod. HAMi limits are fixed for the life of the container.
- Oversubscription. TensorFusion sets a compute oversell ratio per pool and can count host memory and disk as schedulable VRAM. HAMi offers device memory scaling above 1, marked experimental.
Remote GPU and virtual machines
TensorFusion can run a Pod on a CPU-only node and serve its GPU calls from a worker on a GPU node (GPU-over-IP). The same client and worker let a VM use the host's GPUs without passthrough, with Linux or Windows guests, for CUDA and NVML; graphics APIs are not covered. These paths use API remoting, which costs performance for call-heavy workloads; see performance.
HAMi shares GPUs only with containers on the node that holds them.
Hardware and vendor coverage
HAMi is clearly ahead. Its device table lists NVIDIA, AMD Instinct, Huawei Ascend, Cambricon, Hygon DCU, Iluvatar, Kunlunxin, MetaX, Moore Threads, Enflame, AWS Neuron, Biren and Vastai. Memory and core limits are available on most of them (Kunlunxin has memory only; AWS Neuron allocates cores; Biren and Vastai share whole devices), sometimes through the vendor's own partitioning such as Ascend vNPU templates. Only NVIDIA can give one task shares of more than one card.
TensorFusion supports NVIDIA in every mode, and Huawei Ascend, Moore Threads and Alibaba PPU in a subset. AMD has only a skeleton and is not supported; Cambricon, Hygon and MetaX are not supported. The support matrix lists each combination.
Observability, metering and cost attribution
HAMi exports Prometheus metrics for device and container memory and core usage, ships Grafana dashboards, and offers HAMi-WebUI for a cluster view.
TensorFusion writes GPU, node, pool and workload metrics to GreptimeDB in your cluster and evaluates built-in alert rules. It also prices usage: you set unit prices per QoS level and an hourly cost per GPU model, and TensorFusion records the cost of each workload and node for chargeback and showback. It does not issue invoices or enforce budgets. See Configure QoS and billing.
Performance overhead
Neither project has published a side-by-side run on the same hardware, and we have not benchmarked HAMi ourselves. The two sets of numbers below come from different GPUs and workloads, so compare them only as orders of magnitude.
HAMi, from its benchmark page: vLLM serving Qwen3-8B on an A100 40GB, batch 1, against the NVIDIA device plugin. HAMi v2.9.0 adds about 1–3% to time to first token and per-token latency (v2.8.0 added about 8–13%). The page does not report results under contention or how closely the core limit is met.
TensorFusion soft isolation, Alibaba Cloud ACK, NVIDIA T4, October 2026, limit set to 100%, compared with the same workload without TensorFusion:
| Workload | Throughput vs. native |
|---|---|
| vLLM, Qwen3-0.6B, 8 concurrent streams | −0.4%, same time per output token |
| ONNX Runtime, resnet50 / MiniLM | −0.2% / −0.3% |
| TorchBench, resnet50 / bert_base / tiny_mlp | −0.8% / −0.6% / +0.3% |
With a limit, throughput follows the share but not linearly: at a 50% limit, vLLM kept 74% of its throughput and two 50% tenants on one T4 together reached 115% of one tenant alone. Two open issues: tail latency of long convolution kernels grows more than proportionally at 50%, and a 25% limit gives about 15% less throughput than the share on a matmul loop.
API remoting (hard isolation and remote GPU) adds a fixed cost per GPU call. In the published benchmark (July 2025, TorchBench inference), a worker in a separate process on the same node cost about 1% to 38% depending on the model, 12% on average. Added network latency of 0.3 ms and 1 ms raised the average to 41% and 128%, while models that make few, long GPU calls (BERT_pytorch, hf_clip, hf_distil_whisper) stayed within 11% even at 1 ms. Measure your own workload before you rely on remote mode.
Compatibility and migration
HAMi replaces the NVIDIA device plugin on the nodes it manages and relies on the NVIDIA container toolkit. Moving to it means relabeling nodes and changing resource names in Pod specs.
TensorFusion runs beside the NVIDIA GPU Operator (set controller.compatibleWithNvidiaContainerToolkit: true when its validation files are present) and can take over Pods that request nvidia.com/gpu through Helm values, without editing their manifests. The recommended path is to move a workload as a whole GPU first, then shrink it to the fraction it uses. See Migrate existing workloads.
Moving between HAMi and TensorFusion means changing how Pods ask for GPUs: resources.limits extended resources in HAMi, a label and annotations in TensorFusion. Both can run in one cluster on separate node groups.
License, governance and support
| TensorFusion | HAMi | |
|---|---|---|
| Open source | Controller, scheduler, hypervisor and CRDs: Apache 2.0, in NexusGPU/tensor-fusion. | All components, including HAMi-core: Apache 2.0. |
| Closed source | Vendor limiter libraries used by soft mode, and the client and worker behind hard and remote mode. Distributed as public images; the client and worker are free for organizations with fewer than 10 GPUs and need a license file. | None. |
| Governance | Developed by NexusGPU. | CNCF Sandbox from August 2024, Incubating since July 2026; maintainers from Dynamia, NVIDIA and independent contributors. About 4,700 GitHub stars. |
| Commercial | NexusGPU: commercial license, live migration, support. | Dynamia sells an enterprise edition built on HAMi, with support. |
Total cost of ownership
- Small pool, local sharing only. If your team is comfortable operating Kubernetes, Prometheus and GPU nodes, HAMi's open source release is a good fit and costs nothing to license. TensorFusion's open source components cover the same case.
- Larger or cloud pools. Savings come less from slicing than from running fewer nodes: packing fractions onto fewer GPUs, releasing empty nodes, and right-sizing limits from usage. TensorFusion automates that loop; with HAMi you build it from the cluster autoscaler, your own monitoring and manual tuning.
- Remote or VM access, or
hardisolation. Only TensorFusion offers them, and they need its closed-source components, which are licensed above 10 GPUs.
Requesting a GPU
HAMi:
spec:
containers:
- name: app
image: pytorch/pytorch:2.6.0-cuda12.4-cudnn9-runtime
resources:
limits:
nvidia.com/gpu: 1 # number of GPUs
nvidia.com/gpumem: 4096 # MiB of GPU memory on each
nvidia.com/gpucores: 30 # percent of each GPU's computeTensorFusion:
metadata:
labels:
tensor-fusion.ai/enabled: "true"
annotations:
tensor-fusion.ai/tflops-request: "10"
tensor-fusion.ai/tflops-limit: "20"
tensor-fusion.ai/vram-request: "4Gi"
tensor-fusion.ai/vram-limit: "4Gi"
spec:
containers:
- name: app
image: pytorch/pytorch:2.6.0-cuda12.4-cudnn9-runtimeTo request a percentage of one GPU instead of TFLOPS, use tensor-fusion.ai/compute-percent-request and tensor-fusion.ai/compute-percent-limit. Create a workload explains every annotation.
When to choose which
Choose HAMi when:
- You run accelerators that TensorFusion does not support, or many vendors in one cluster.
- Every component must be open source, or you want a project under CNCF governance.
- You already use Volcano, Kueue, Koordinator or KAI Scheduler, or are moving to DRA, and want GPU sharing to plug into it.
- Local fractional GPUs with fixed limits are all you need, with as few components as possible.
Choose TensorFusion when:
- You run on a cloud and want fewer GPU nodes: packing, node autoscaling with Karpenter, and releasing empty nodes.
- You want GPU limits right-sized from real usage and changed without restarting Pods.
- You want requests and limits on GPU compute, with QoS levels deciding who gets spare capacity.
- Pods on CPU-only nodes, or VMs, need to use GPUs on other machines.
- You need per-workload cost data for chargeback.
References
TensorFusion measurements: the end-to-end suite in NexusGPU/tensor-fusion (e2e/ack/TEST-SUMMARY.md, October 2026) and the benchmark page.
HAMi, checked in October 2026:
- HAMi README: features, supported devices, prerequisites.
- HAMi releases: v2.10.0 (August 2026) is the latest checked.
- HAMi-core: how memory and core limits are enforced.
- HAMi configuration: resource names, scheduler policies, device memory scaling.
- Supported devices and benchmark.
- HAMi-DRA and HAMi-WebUI.
- HAMi becomes a CNCF incubating project, CNCF blog, July 2026.