Concepts

The four ideas that explain most of vkML's behaviour.

Tensors are lazy

An operation records a node; it does not run. Work happens when a result is needed — realize(), .numpy(), .item(), or a backward pass.

This is not only an optimisation. Batching lets many operations share a single GPU submission, and a submission costs roughly 105 µs against 9 µs for a dispatch on the development hardware. Reducing submissions is worth far more than making any one kernel faster.

set_eager(True), or VKML_EAGER=1, runs everything immediately. It is slower, and it is the right setting while debugging because a failure surfaces at the operation that caused it.

Two backends, one of which is a reference

Because the CPU backend is the reference, CPU support must be a superset of Vulkan support, and that is enforced by a test rather than by convention. It also means the CPU backend is slow by design: roughly 116× slower than PyTorch on the same machine. Use it to check answers, not to get them.

Devices are explicit

Tensors do not move on their own. An operation whose operands are on different devices raises rather than inserting a transfer, because an implicit copy across PCIe is the kind of cost that should appear in your code and not in a profile.

copy>>> a = vkml.tensor(np.zeros((2, 2), dtype=np.float32))
>>> b = vkml.tensor(np.zeros((2, 2), dtype=np.float32), device=vkml.device("vulkan:0"))
>>> vkml.matmul(a, b)
DeviceError: 'matmul' operands are on different devices: cpu and vulkan:0

Device indices are not stable across environments — the same machine can report a discrete GPU at index 0 natively and at index 1 inside a container. Prefer best_device(), or select on device_type from vulkan_device_reports().

Determinism is a hard invariant

Identical inputs give bit-identical outputs on the same device, and across drivers wherever the contract claims it. Two consequences show up in the API:

• Reductions use a fixed pairwise tree determined by shape, not by whichever workgroup finished first.
• float32→float16 narrowing is implemented in software, because SPIR-V leaves OpFConvert's rounding mode implementation-defined and two drivers disagreed.

Any change that would trade this for speed needs a re-derived error bound first. It is not a preference.

NaN follows PyTorch

vkML matches PyTorch's NaN semantics wherever a Vulkan primitive allows it, and documents the cases where it cannot. A finite input never produces NaN; NaN propagates through reductions; relu and amax do not silently swallow it. Where a deliberate divergence exists it is stated on the operator's own page.