Overview

TensorFusion is a GPU virtualization and pooling system for Kubernetes. It splits a physical GPU among several Pods, enforces a compute and VRAM limit on each of them, and schedules all GPUs in the cluster as one pool. A Pod asks for GPU resources with a label and a few annotations; the application image and code stay as they are.

This page explains what TensorFusion does, when it fits, and where to go next.

What TensorFusion does

Capability	What you get
Fractional GPU	Request and limit compute (in TFLOPS or as a percentage of one GPU) and VRAM per Pod, instead of whole cards.
Four isolation modes	`shared`, `soft`, `hard`, and `partitioned`. Pick per workload how strictly it is separated from its neighbors. See Architecture.
Local and remote GPU	A Pod can use a GPU on its own node, or run on a node without a GPU and use one over the network (GPU-over-IP).
GPU-aware scheduling	A scheduler that understands TFLOPS and VRAM, with bin-packing or load-spreading placement, four QoS levels, namespace quotas, gang scheduling, and GPU topology awareness.
Oversubscription	Each pool sets how far compute can be oversold and how much host memory and disk count toward schedulable VRAM.
Autoscaling	Adjusts a workload's GPU requests and limits from its usage history, and adds or removes GPU nodes through Karpenter or a cloud provider API.
Node compaction and defragmentation	Empties and releases underused GPU nodes.
Multiple vendors	NVIDIA, Huawei Ascend, Moore Threads, and Alibaba PPU, each described by a `ProviderConfig`. See the support matrix.
Monitoring and alerting	GPU, node, pool, and workload metrics are written to GreptimeDB inside your cluster. Alert rules are evaluated by the controller and sent to Alertmanager.
Cloud console (optional)	A web console to manage clusters. An agent in your cluster dials out to it, so the cluster needs no inbound access.

GPU live migration and some advanced autoscaling policies are part of the commercial edition. See Licensing.

Demo

End-to-end demo

Scale on demand

Enforce the limit

Remote GPU

GPU live migration

How it works

You install one Helm chart. It deploys the controller (CRD controllers, a Pod admission webhook, and a scheduler in one process) and a metrics database.
The controller starts a hypervisor Pod on every GPU node. The hypervisor discovers the GPUs and reports them as GPU resources.
You add the label tensor-fusion.ai/enabled: "true" and resource annotations to a Pod template.
The webhook rewrites the Pod, the scheduler picks GPUs for it, and the hypervisor enforces its limits.

Architecture covers the components and each step in detail.

When to use it

Many small or bursty models share a cluster. Inference services, Model-as-a-Service platforms, and AI SaaS backends rarely need a whole GPU each. Fractional GPUs and oversubscription pack more of them onto one card.
Development and teaching environments. Notebooks and labs sit idle most of the time. Give each user a slice, or let Pods on CPU nodes use GPUs remotely.
GPU rental and internal platforms. QoS levels, namespace quotas, and usage metrics per workload let you divide a pool between tenants and account for what they use.
Mixed hardware. One scheduler and one set of annotations cover NVIDIA and the other supported vendors.

When not to use it

Each job already saturates whole GPUs. Large training runs that occupy every card gain little from fractional sharing. You can still run them in shared mode to use the scheduler and pooling.
Untrusted tenants on soft isolation. The soft limiter runs inside the application process and is not a security boundary. Use hard or partitioned for workloads you do not trust.
Images that are not based on glibc. soft isolation loads its limiter through /etc/ld.so.preload, which needs a glibc-based image with /bin/sh. Alpine and other musl images are not supported in this mode.
Clusters without Linux GPU nodes. The controller and hypervisor run on Linux only.

Quick start

Common tasks

Reference

Compare with other solutions

TensorFusion combines fractional GPUs, remote GPU sharing, a GPU-aware scheduler, and node autoscaling in one system. It runs its own scheduler and device plugin, and can take over Pods that request nvidia.com/gpu, so an existing NVIDIA device plugin setup can be migrated step by step.

Each comparison below goes through features, isolation, and operations in detail:

Licensing

Open source (Apache 2.0). The controller, scheduler, and hypervisor.
Free to use, closed source, no license needed. The vendor limiter libraries that soft mode loads into the application, shipped in public images. Pooling, scheduling, and the shared, partitioned, and local soft modes need nothing beyond these and the open source components.
Free to use, closed source, license file needed. The client and worker that implement remote GPU sharing and hard isolation. They are free for organizations with fewer than 10 GPUs in total.
Commercial. Larger deployments of the closed-source components, and enterprise features such as GPU live migration. See pricing or contact us.

FAQ

Is TensorFusion open source?

Most of it is. The controller, scheduler, and hypervisor live in NexusGPU/tensor-fusion under Apache 2.0. The client and worker used for remote GPU sharing and hard isolation are distributed as container images only. Their design follows the API remoting approach described in rCUDA.

When is TensorFusion free?

The open source components are always free. The closed-source components are free for commercial and non-commercial use by organizations with fewer than 10 GPUs in total. Beyond that, contact us for a commercial or educational license.

Which GPUs does it support?

NVIDIA GPUs have the widest coverage: all four isolation modes, locally and remotely. Huawei Ascend, Moore Threads, and Alibaba PPU are supported in a subset of modes. AMD is not supported yet. The support matrix lists every combination.

Do I have to change my application?

No. You add a label and annotations to the Pod template. The webhook injects what the selected mode needs, and the application sees a regular GPU device. With more than one container in the Pod, name the containers that need a GPU in the tensor-fusion.ai/inject-container annotation.

Does it need the NVIDIA GPU Operator?

No. GPU nodes need a working vendor driver and a container runtime that can expose the GPU to containers. If the GPU Operator is already installed, TensorFusion can run next to it. See Migrate existing GPU workloads.

Are there production users?

Yes. See How TenClass saved 80% on GPU costs with TensorFusion.

Who develops TensorFusion?

NexusGPU PTE. LTD., a company headquartered in Singapore, develops and maintains TensorFusion and the related GitHub projects.

This page explains what TensorFusion does, when it fits, and where to go next.

What TensorFusion does

Capability	What you get
Fractional GPU	Request and limit compute (in TFLOPS or as a percentage of one GPU) and VRAM per Pod, instead of whole cards.
Four isolation modes	`shared`, `soft`, `hard`, and `partitioned`. Pick per workload how strictly it is separated from its neighbors. See Architecture.
Local and remote GPU	A Pod can use a GPU on its own node, or run on a node without a GPU and use one over the network (GPU-over-IP).
GPU-aware scheduling	A scheduler that understands TFLOPS and VRAM, with bin-packing or load-spreading placement, four QoS levels, namespace quotas, gang scheduling, and GPU topology awareness.
Oversubscription	Each pool sets how far compute can be oversold and how much host memory and disk count toward schedulable VRAM.
Autoscaling	Adjusts a workload's GPU requests and limits from its usage history, and adds or removes GPU nodes through Karpenter or a cloud provider API.
Node compaction and defragmentation	Empties and releases underused GPU nodes.
Multiple vendors	NVIDIA, Huawei Ascend, Moore Threads, and Alibaba PPU, each described by a `ProviderConfig`. See the support matrix.
Monitoring and alerting	GPU, node, pool, and workload metrics are written to GreptimeDB inside your cluster. Alert rules are evaluated by the controller and sent to Alertmanager.
Cloud console (optional)	A web console to manage clusters. An agent in your cluster dials out to it, so the cluster needs no inbound access.

GPU live migration and some advanced autoscaling policies are part of the commercial edition. See Licensing.

You install one Helm chart. It deploys the controller (CRD controllers, a Pod admission webhook, and a scheduler in one process) and a metrics database.
The controller starts a hypervisor Pod on every GPU node. The hypervisor discovers the GPUs and reports them as GPU resources.
You add the label tensor-fusion.ai/enabled: "true" and resource annotations to a Pod template.
The webhook rewrites the Pod, the scheduler picks GPUs for it, and the hypervisor enforces its limits.

Architecture covers the components and each step in detail.

When to use it

Many small or bursty models share a cluster. Inference services, Model-as-a-Service platforms, and AI SaaS backends rarely need a whole GPU each. Fractional GPUs and oversubscription pack more of them onto one card.
Development and teaching environments. Notebooks and labs sit idle most of the time. Give each user a slice, or let Pods on CPU nodes use GPUs remotely.
GPU rental and internal platforms. QoS levels, namespace quotas, and usage metrics per workload let you divide a pool between tenants and account for what they use.
Mixed hardware. One scheduler and one set of annotations cover NVIDIA and the other supported vendors.

When not to use it

Each job already saturates whole GPUs. Large training runs that occupy every card gain little from fractional sharing. You can still run them in shared mode to use the scheduler and pooling.
Untrusted tenants on soft isolation. The soft limiter runs inside the application process and is not a security boundary. Use hard or partitioned for workloads you do not trust.
Images that are not based on glibc. soft isolation loads its limiter through /etc/ld.so.preload, which needs a glibc-based image with /bin/sh. Alpine and other musl images are not supported in this mode.
Clusters without Linux GPU nodes. The controller and hypervisor run on Linux only.

Open source (Apache 2.0). The controller, scheduler, and hypervisor.
Free to use, closed source, no license needed. The vendor limiter libraries that soft mode loads into the application, shipped in public images. Pooling, scheduling, and the shared, partitioned, and local soft modes need nothing beyond these and the open source components.
Free to use, closed source, license file needed. The client and worker that implement remote GPU sharing and hard isolation. They are free for organizations with fewer than 10 GPUs in total.
Commercial. Larger deployments of the closed-source components, and enterprise features such as GPU live migration. See pricing or contact us.

FAQ

Is TensorFusion open source?

When is TensorFusion free?

Which GPUs does it support?

Do I have to change my application?

Does it need the NVIDIA GPU Operator?

Are there production users?

Yes. See How TenClass saved 80% on GPU costs with TensorFusion.

Who develops TensorFusion?

NexusGPU PTE. LTD., a company headquartered in Singapore, develops and maintains TensorFusion and the related GitHub projects.

What TensorFusion does

Demo

End-to-end demo

Scale on demand

Enforce the limit

Remote GPU

GPU live migration

How it works

When to use it

When not to use it

Quick start

Common tasks

Reference

Compare with other solutions

Licensing

FAQ

Table of Contents

Overview

What TensorFusion does

Demo

End-to-end demo

Scale on demand

Enforce the limit

Remote GPU

GPU live migration

How it works

When to use it

When not to use it

Quick start

Common tasks

Reference

Compare with other solutions

Licensing

FAQ

Table of Contents