Autograd & execution

5 functions — 5 documented.

backward¶

Compute gradients of a scalar with respect to every leaf that requires them.

Walks the recorded graph in reverse and accumulates into each leaf's .grad — it does not overwrite. A training loop must therefore call optimizer.zero_grad() between steps, exactly as in PyTorch.

Backward rules are built from forward operators wherever they can be, rather than each having a hand-written kernel. That is why the kernel count stays near 64 instead of roughly doubling, and it is a recorded design rule rather than an accident: a new operator earns a dedicated backward kernel only if its gradient genuinely cannot be composed.

Parameters

root (Tensor)

A 0-d tensor. Calling on a non-scalar raises.

Example

copy>>> x = vkml.tensor(np.array([3.0], dtype=np.float32), requires_grad=True)
>>> vkml.backward(vkml.sum(vkml.mul(x, x)))
>>> x.grad.numpy()
array([6.], dtype=float32)

From the header

Accumulates gradients into every leaf reachable from root that has requires_grad set.

root must be a scalar; the seed gradient is 1. This matches torch.Tensor.backward() with no arguments.

Gradients ACCUMULATE into .grad rather than replacing it, as in PyTorch, which is what makes gradient accumulation across micro-batches work. Callers are responsible for zeroing (optim.zero_grad()).

Implementation

See also realize, detach, set_eager

realize¶

Force a lazily built graph to execute.

Operations are recorded, not run, until a result is needed. realize is the explicit trigger; .numpy(), .item() and a backward pass trigger it implicitly.

Batching work this way is what lets many operations share one submission, and on this hardware a submission costs the host 40–105 µs — measured three ways: by regressing wall time on the number of trivial realizes, by batching eight parameter copies into one, and earlier against a ~9 µs dispatch. Reducing submissions is worth far more than making any single kernel faster, which is why the laziness is a performance mechanism and not just an API style.

Parameters

tensor (Tensor)

The graph root to evaluate.

From the header

Forces evaluation. Idempotent.

Implementation

See also set_eager, is_eager, backward, assign

assign¶

Assign sources[i] into destinations[i], as one unit of work.

The batched form of dst.assign_(src), and the same implementation underneath — but a loop pays one submission per assignment, and an optimiser step is one assignment per parameter.

Measured on eight parameters: 8 submissions and 0.499 ms one at a time, 1 submission and 0.096 ms batched. This is why an optimiser step now costs a constant three submissions regardless of the parameter count.

Batching is not expressible above the backend, because only the backend knows what a submission is. That is why this exists as an operation rather than as advice to call assign_ less often.

Parameters

destinations (Sequence[Tensor])

Tensors to overwrite, in place.

sources (Sequence[Tensor])

Values to write, one per destination.

Returns

None.

From the header

Assigns src[i] into dst[i] for every i, as ONE unit of device work.

Identical in effect to calling dst[i].assign_(src[i]) in a loop, and it is the same function underneath -- but a loop pays one submission per assignment, and an optimiser step is one assignment per parameter. Measured on the CIFAR-100 CNN's eight parameters, the loop was 8 submissions at ~80 µs of host time each.

This is the batched sibling of realize(std::span<const NodePtr>), for the same reason and with the same shape: the work is independent, so the only thing forcing it apart was the API.

Sources needing an overlapping or host-staged copy fall back to the per-tensor path individually, so mixing them is allowed and costs only what those particular ones cost.

Implementation

See also realize, backward

set_eager¶

Run every operation immediately instead of building a graph.

Off by default. Turning it on makes a failure surface at the operation that caused it rather than at the next realize, which is the point — it is a debugging aid, not a performance mode, and it is slower because every operation becomes its own submission.

Parameters

enabled (bool)

Whether to execute eagerly.

From the header

Eager mode: realise after every operation instead of at observation points.

Required, not a nicety. Under lazy evaluation a bad kernel surfaces at the realize() call, arbitrarily far from the op that caused it; the per-operator validation suite (docs/ARCHITECTURE.md §7.4) runs in eager mode so a failure names the operator directly. Also settable with VKML_EAGER=1.

Implementation

See also is_eager, realize

is_eager¶

Whether eager execution is currently on.

Returns

True if operations run immediately.

Implementation

See also set_eager, realize