What vkML does not do, and why. Each entry says whether the limit is a decision, a consequence of a guarantee the project keeps, or work that has not happened yet — those are different things, and a reader deciding whether to adopt this needs to tell them apart.
- By design
- The question does not apply, or supporting it would cost more than it gives.
- Would break a guarantee
- Possible, but not without giving up something the project promises.
- Not yet implemented
- Wanted; nothing blocking it.
- Waiting on a measurement
- Deferred until a number justifies the work.
- Waiting on architectural work
- Blocked on a change tracked elsewhere.
Operations without a gradient rule
48 of 67 graph operations carry one. Of the
19 that do not, 16 are cases where a gradient is
not a thing that exists — a leaf has no input, a comparison returns
Bool — and 3 are genuine gaps.
| Operation | Why | |
|---|---|---|
Input | By design | A leaf. Gradients accumulate here rather than flow through. |
Const | By design | A constant has no input to differentiate with respect to. |
Full | By design | Produces a tensor from a shape and a scalar; no tensor input. |
Arange | By design | Produces a tensor from start, stop and step; no tensor input. |
Rand | By design | Produces a tensor from a seed; no tensor input. |
Equal | By design | Returns Bool. Zero derivative off the boundary, undefined on it. |
Less | By design | Returns Bool. Zero derivative off the boundary, undefined on it. |
Greater | By design | Returns Bool. Zero derivative off the boundary, undefined on it. |
LessEqual | By design | Returns Bool. Zero derivative off the boundary, undefined on it. |
GreaterEqual | By design | Returns Bool. Zero derivative off the boundary, undefined on it. |
NotEqual | By design | Returns Bool. Zero derivative off the boundary, undefined on it. |
Sign | By design | Piecewise constant: derivative zero almost everywhere, undefined at zero. |
Erf | Not yet implemented | The forward op exists on both backends. The rule is d/dx erf(x) = 2/sqrt(pi) * exp(-x^2) and nothing blocks it. |
Erfc | Not yet implemented | As Erf, negated. gelu reaches its gradient through a different path, so nothing on a training path is waiting on this. |
Prod | Not yet implemented | Reachable as prod(x) * sum(grad / x), which needs care where x contains a zero. prod is CPU-only for a separate reason -- see the backend table. |
ArgMax | By design | Returns integer indices, which are not a differentiable function of the input. |
ArgMin | By design | Returns integer indices, which are not a differentiable function of the input. |
MaxPool2dBackward | By design | Already a backward op. A rule for it would be a second derivative. |
SliceBackward | By design | Already a backward op. A rule for it would be a second derivative. |
Operators that do not run on Vulkan
113 of 114 run on the GPU.
| Operator | Why | |
|---|---|---|
prod | Would break a guarantee | A parallel reduction reorders the fold, and for a product that is a different answer rather than a rounding difference: multiplying 1e20 and 1e-20 alternately gives 1.0 in the CPU's index order and inf once the large values are grouped, which lane-striding does immediately (measured). Matching the CPU would mean one lane multiplying in index order -- a kernel with no parallelism, slower than the CPU at the only thing it would be correct for. The CPU backend is the oracle the GPU is checked against, so a GPU prod that legitimately disagreed would break that chain rather than extend it. Nothing in nn, the losses or the optimisers calls it. |
Features
Mixed CPU and Vulkan execution in one graph By design
A graph runs entirely on one backend. There is no automatic per-node fallback, so an operator the GPU cannot run is an error rather than a silent transfer. Splitting a graph across devices means inserting transfers the author did not write, and a transfer is the most expensive thing in this system -- a submission costs about 105us against 9us for a dispatch. The decision and the alternatives are recorded in ADR 0008.
Tensors of rank above 4 By design
kMaxDims is 4. Every push-constant block carries extents and strides inline, and Vulkan guarantees only 128 bytes of push constants -- the budget that rank cap buys is what keeps every shader inside the guaranteed minimum on any conformant device.
float64 By design
The dtypes are f32, f16, i32, i64 and bool. Double-precision compute is optional in Vulkan and absent on most consumer GPUs, so supporting it would mean a CPU-only dtype -- which is the backend divergence the oracle design exists to avoid.
Distributed or multi-GPU training Not yet implemented
One device per graph today. The device is already a first-class part of every tensor and the allocator is per-device, so the groundwork is there, but nothing coordinates two of them.
Operator fusion Waiting on a measurement
layer_norm and rms_norm are composed from smaller operators rather than fused. Fusing them would save bandwidth, not accuracy, and the measurement that would justify it has not been taken -- on the profile so far the time goes to submission overhead rather than to kernel bandwidth, so fusing first would be optimising the part that is not the constraint.
A lazy detach() Waiting on architectural work
detach() shares its source's buffer, so an unrealized source has nothing to share and the call forces evaluation -- which cuts the graph. Every optimiser calls it on its intermediates, so this caps how much work any batching can combine. Measured: a prototype SGD that builds all updates lazily still submits 13 times rather than the predicted 9. Fixing it is a change to autograd, not to the optimiser.