Element-wise

22 functions — 22 documented.

abs¶

Absolute value, element-wise.

Dispatches GLSL's abs() on the GPU and std::fabs on the CPU. abs(-0.0) is +0.0 and abs(NaN) is NaN on both.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> x = vkml.tensor(np.array([-2.0, -0.0, 3.0], dtype=np.float32))
>>> vkml.abs(x).numpy()
array([2., 0., 3.], dtype=float32)

Implementation

See also sign, neg, square

neg¶

Arithmetic negation, element-wise.

Compiled as unary minus rather than a multiply by −1, so the sign bit is flipped and nothing else: neg(0.0) is -0.0, and neg(NaN) keeps the payload.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.neg(vkml.tensor(np.array([1.0, -2.5], dtype=np.float32))).numpy()
array([-1. ,  2.5], dtype=float32)

Implementation

See also abs, sub

scaled_add¶

input * alpha + other * beta, as one operation.

Both coefficients are plain numbers and travel in the kernel's push constants, so nothing is materialised for them.

The composed form costs FOUR nodes rather than one: a scalar operand becomes a rank-0 tensor, so a * 0.9 is a full and a mul. Measured on an SGD-with-momentum step over an MNIST MLP, the composed form issued 24 dispatches against 8 here.

Shapes are broadcast by NumPy rules. Broadcasting is implemented by giving the expanded axes stride 0 rather than by materialising a copy — Shape holds strides in bytes and permits 0, so a broadcast operand re-reads the same element instead of allocating an expanded one.

Parameters

input (Tensor)

The left operand.

other (Tensor)

The right operand, broadcastable against input.

alpha (float)

Coefficient applied to input.

beta (float)

Coefficient applied to other.

Returns

A tensor of the broadcast shape.

Example

copy>>> a = vkml.tensor(np.array([1.0, 2.0], dtype=np.float32))
>>> b = vkml.tensor(np.array([10.0, 20.0], dtype=np.float32))
>>> vkml.scaled_add(a, b, 0.5, 2.0).numpy()
array([20.5, 41. ], dtype=float32)

From the header

`a * alpha + b * beta`, as one operation.

The composed form costs four nodes, because each coefficient is materialised as a rank-0 tensor before the multiply. This costs one, and the coefficients travel in the push constants.

Bit-identical to the composed form by construction: both multiplies round to f32 before the add, exactly where mul would have stored its result, and the shader is written so the compiler cannot contract them into an FMA (ADR 0005 makes the same argument for GEMM).

Named for what it computes rather than for its caller. Every momentum optimiser is built from this shape -- SGD's velocity and parameter updates, RMSProp's and Adam's moving averages -- which is what earns it an operator.

In the CPU kernel

aalpha + bbeta.

Written as two multiplies and an add, each rounding to f32, so it matches the composed a*alpha then + b*beta exactly -- and matches the shader, which is written the same way for the same reason. float intermediates rather than the double a naive reading might use: widening here would make the oracle disagree with every backend it is meant to check.

src/backend/cpu/kernels_elementwise.cpp:256

Implementation

See also add, mul

exp¶

Natural exponential, element-wise.

exp(x) overflows to +inf above about 88.72 in float32 — that bound, ln(FLT_MAX), is the same one that makes a naive tanh return NaN. Operators built on exp here subtract a maximum first for exactly that reason; see softmax and sigmoid.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.exp(vkml.tensor(np.array([0.0, 1.0, 88.0], dtype=np.float32))).numpy()
array([1.0000000e+00, 2.7182817e+00, 1.6516363e+38], dtype=float32)

Implementation

See also log, sigmoid, softmax, tanh

log¶

Natural logarithm, element-wise.

log(0) is -inf and log(x) for x < 0 is NaN, matching IEEE and torch.

Parameters

input (Tensor)

Any shape. float32 or float16. Negative values give NaN.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.log(vkml.tensor(np.array([1.0, np.e], dtype=np.float32))).numpy()
array([0.        , 0.99999994], dtype=float32)

Implementation

See also exp, log_softmax, cross_entropy

sqrt¶

Square root, element-wise.

sqrt(-0.0) is -0.0 and sqrt(x) for x < 0 is NaN, both matching IEEE.

Parameters

input (Tensor)

Any shape. float32 or float16. Negative values give NaN.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.sqrt(vkml.tensor(np.array([4.0, 9.0], dtype=np.float32))).numpy()
array([2., 3.], dtype=float32)

Implementation

See also rsqrt, square

rsqrt¶

Reciprocal square root, 1/sqrt(x), element-wise.

Compiled to GLSL's inversesqrt() rather than a divide after a sqrt. It is one instruction on the GPU, and it is the form normalisation layers want — rms_norm and layer_norm both scale by an inverse root rather than dividing.

Parameters

input (Tensor)

Any shape. float32 or float16. Negative values give NaN.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.rsqrt(vkml.tensor(np.array([4.0, 16.0], dtype=np.float32))).numpy()
array([0.5 , 0.25], dtype=float32)

Implementation

See also sqrt, reciprocal, rms_norm

reciprocal¶

1/x, element-wise.

reciprocal(0.0) is +inf and reciprocal(-0.0) is -inf, matching IEEE division rather than raising.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.reciprocal(vkml.tensor(np.array([2.0, 4.0], dtype=np.float32))).numpy()
array([0.5 , 0.25], dtype=float32)

Implementation

See also div, rsqrt

square¶

x * x, element-wise.

A multiply, not pow(x, 2): exact for every representable input, one instruction, and its gradient is 2x rather than the general power rule.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.square(vkml.tensor(np.array([-3.0, 4.0], dtype=np.float32))).numpy()
array([ 9., 16.], dtype=float32)

Implementation

See also pow, sqrt, mse_loss

sign¶

The sign of each element: −1, 0 or +1.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> x = vkml.tensor(np.array([-2.0, -0.0, 0.0, 3.0], dtype=np.float32))
>>> vkml.sign(x).numpy()
array([-1.,  0.,  0.,  1.], dtype=float32)

In the Vulkan kernel

sign, matching torch on every input including the ones that surprise.

The fall-through returns +0.0, covering +0.0, -0.0 AND NaN, none of which satisfies either comparison. It used to return x, with a comment claiming that matched torch. Measured, torch agrees with neither half: torch.sign(-0.0) is +0.0 and torch.sign(nan) is +0.0. numpy differs from torch on NaN but agrees on -0.0, so the old behaviour matched neither reference (issue #27).

GLSL's built-in sign() would give the right answer for -0.0 and is undefined for NaN, so the comparisons are written out.

THE SIGN OF THE ZERO IS NORMALISED EXPLICITLY, and two drivers disagree about why that is needed. Written as the obvious chain --

if (x > 0.0) { return 1.0; } if (x < 0.0) { return -1.0; } return 0.0;

-- this returns -0.0 on lavapipe for every input that falls through: +0.0, -0.0, +NaN and -NaN alike, measured by reading the bits back. RADV gives +0.0 for the same SPIR-V, so it went unnoticed until a second driver ran it. The transform that explains it is merging the -1.0 and 0.0 returns into a negation, whose zero case is -0.0; many optimisers treat the sign of a zero as free to change.

Two rewrites were measured on both drivers before this one was chosen:

- Spelling the zero as uintBitsToFloat(0u) does nothing. It compiles to BYTE-IDENTICAL SPIR-V, because glslang folds it long before a driver sees it -- verified by hashing the generated module. - float(x > 0.0) - float(x < 0.0) is branchless and fixes lavapipe, and it is WRONG ON RADV: NaN comes back as -1.0, so the comparison it compiles to is not the ordered one the source asks for. It would have traded one driver's bug for another's.

So the branches stay -- they are correct on RADV, and ordered comparisons make NaN fall through with the zeros -- and the result's sign bit is cleared when its magnitude is zero. That is a no-op for ±1.0 and forces +0.0 for everything else, whatever the optimiser did on the way.

Implementation

See also abs, relu

sin¶

Sine, element-wise, in radians.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.sin(vkml.tensor(np.array([0.0, np.pi / 2], dtype=np.float32))).numpy()
array([0., 1.], dtype=float32)

Implementation

See also cos

cos¶

Cosine, element-wise, in radians.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.cos(vkml.tensor(np.array([0.0, np.pi], dtype=np.float32))).numpy()
array([ 1., -1.], dtype=float32)

Implementation

See also sin

tanh¶

Apply the hyperbolic tangent, element-wise.

Saturates to ±1 outside roughly |x| > 10, and the Vulkan kernel clamps there explicitly.

Parameters

input (Tensor)

Any shape, float dtype.

Returns

A tensor of the same shape and dtype.

Example

copy>>> x = vkml.tensor(np.array([-89.0, -1.0, 0.0, 89.0], dtype=np.float32))
>>> vkml.tanh(x).numpy()
array([-1.       , -0.7615942,  0.       ,  1.       ], dtype=float32)

In the Vulkan kernel

tanh, saturated before the built-in can overflow.

GLSL defines tanh as (e^x - e^-x) / (e^x + e^-x), and an implementation that evaluates that literally produces inf/inf = NaN once |x| passes ln(FLT_MAX) = 88.7228. A FINITE input then yields NaN, which breaks the oracle contract in ARCHITECTURE.md 7 and fails silently -- a NaN appearing mid-network reads as a diverged model, not a kernel defect. RADV computes it some other way and is unaffected; AMD's Windows driver is not (issue #26).

The same lesson as f16 narrowing in common.glsl: where a built-in's edge behaviour is left to the implementation, do not depend on it.

Clamping the INPUT rather than the output keeps the built-in on the path for every argument it handles. 10.0 is where the answer saturates in f32 -- tanh(10) is exactly 1.0f, while tanh(9.5) is still 0.99999994 -- so no value loses precision to this.

NaN passes through: abs(NaN) > 10.0 is false, as every NaN comparison is, so a NaN reaches the built-in and stays one, matching std::tanh on the CPU.

Implementation

See also sigmoid, gelu

sigmoid¶

The logistic function, 1/(1 + exp(−x)), element-wise.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.sigmoid(vkml.tensor(np.array([-1.0, 0.0, 1.0], dtype=np.float32))).numpy()
array([0.26894143, 0.5       , 0.7310586 ], dtype=float32)

In the Vulkan kernel

Logistic function, in the same two-branch form the CPU kernel uses.

The textbook 1/(1+exp(-x)) relies on exp() overflowing to inf and 1/inf giving 0 for large negative x. That happens to be right, but the branch keeps the argument to exp() bounded above by 0 in both tails and does not depend on the driver's overflow behaviour.

Implementation

See also tanh, silu, binary_cross_entropy_with_logits

erf¶

The error function, element-wise.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.erf(vkml.tensor(np.array([0.0, 1.0], dtype=np.float32))).numpy()
array([0.       , 0.8427008], dtype=float32)

In the Vulkan kernel

erf, split at |x| = 0.5.

Above the split, 1 - erfc is accurate because both terms are O(1). Below it that subtraction cancels -- erf(0.01) is 1.1e-2 against erfc = 0.989 -- so the Maclaurin series is used instead, where every term carries the factor x and no cancellation occurs. Seven terms leave a truncation error of ~3e-8 relative at the split point, below the 1 ULP the policy allows.

Implementation

See also erfc, gelu

erfc¶

The complementary error function, 1 − erf(x), element-wise.

Computed directly rather than as 1 − erf(x), which cancels catastrophically in the positive tail — the tail this function exists to serve. gelu is built on it for the same reason.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.erfc(vkml.tensor(np.array([0.0, 1.0], dtype=np.float32))).numpy()
array([1.       , 0.1572992], dtype=float32)

From the header

Complementary error function, 1 - erf(x), computed without forming that difference. Exists because the subtraction is what destroys the result: erf approaches +/-1, so 1 - erf(x) and 1 + erf(x) cancel and lose the significand exactly where erfc is still perfectly representable. gelu's value and its gradient are both built on it for that reason.

In the Vulkan kernel

erfc over the whole line, reflecting the x >= 0 approximation.

erfc(-x) = 2 - erfc(x) is used for the negative half, and unlike the erf case that subtraction is safe: erfc_pos(x) is at most 1 there, so the result is O(1) and nothing cancels. The tail this exists to protect is the POSITIVE one, where erfc decays toward zero and is returned by erfc_pos directly.

NaN falls to the second branch, since x >= 0.0 is false for it, and propagates through unchanged. +inf gives 0 and -inf gives 2, both exact.

Implementation

See also erf, gelu

relu¶

Apply the rectified linear unit, element-wise.

Computes x <= 0 ? 0 : x.

The comparison is written against zero rather than as max(x, 0) so that NaN propagates: max would return the non-NaN operand on some drivers and silently swallow it.

Parameters

input (Tensor)

Any shape, float dtype.

Returns

A tensor of the same shape and dtype.

Example

copy>>> import numpy as np, vkml
>>> x = vkml.tensor(np.array([-2.0, -0.0, 0.0, 3.0], dtype=np.float32))
>>> vkml.relu(x).numpy()
array([0., 0., 0., 3.], dtype=float32)

In the Vulkan kernel

relu, written so that NaN survives it.

x <= 0.0 ? 0.0 : x, NOT x > 0.0 ? x : 0.0. The two agree on every number and differ on NaN, which fails BOTH comparisons: the first form falls through to x and propagates, the second falls through to 0 and destroys it. torch propagates, as do maximum(x, 0) and clamp_min(x, 0) -- the same function spelled differently, which vkml already got right. relu was the odd one out (issue #27), and a NaN silently becoming 0 mid-network hides a diverged model rather than announcing it.

Implementation

See also gelu, silu, sigmoid, clamp_min

gelu¶

Apply the Gaussian error linear unit, element-wise.

Computes the exact form x · Φ(x), where Φ is the standard normal CDF — not the tanh approximation.

Both backends evaluate Φ through erfc rather than as 0.5(1 + erf(x/√2)). The second form cancels catastrophically in the negative tail: measured over 512 points on [-6, -3] it reached a relative error of 1.0, returning exactly zero on a domain where the true value never is.

Parameters

input (Tensor)

Any shape, float dtype.

Returns

A tensor of the same shape and dtype.

Example

copy>>> x = vkml.tensor(np.array([-6.0, -1.0, 0.0, 1.0], dtype=np.float32))
>>> vkml.gelu(x).numpy()
array([-5.9195355e-09, -1.5865526e-01,  0.0000000e+00,  8.4134471e-01],
      dtype=float32)

In the Vulkan kernel

Exact GELU: 0.5x(1 + erf(x/sqrt2)), matching torch's default approximate='none'. The tanh approximation differs by up to ~1e-3, far outside the 1e-5 gate, and would look like a bug rather than a choice.

Written through erfc rather than erf because 1 + erf(u) cancels for u < 0: at x = -3 the true value is 2.7e-3 formed from 1 and -0.997, losing most of the significand. 1 + erf(u) == erfc(-u) computes it directly.

Implementation

See also erf, erfc, silu, relu

silu¶

SiLU / Swish: x · sigmoid(x), element-wise.

Composed in the kernel as a multiply against sigmoid_op, so it inherits sigmoid's two-branch form and its overflow safety rather than repeating them.

Parameters

input (Tensor)

Any shape. float32 or float16.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.silu(vkml.tensor(np.array([-1.0, 0.0, 1.0], dtype=np.float32))).numpy()
array([-0.26894143,  0.        ,  0.7310586 ], dtype=float32)

Implementation

See also sigmoid, gelu, relu

clamp¶

Limit every element to [min, max].

Implemented as two nested comparisons, x < lo ? lo : (x > hi ? hi : x), rather than min(max(x, lo), hi). The bounds arrive as push constants, so no extra tensor is allocated for them.

Parameters

input (Tensor)

Any shape, float dtype.

min (float)

Lower bound, inclusive.

max (float)

Upper bound, inclusive.

Returns

A tensor of the same shape and dtype.

Example

copy>>> x = vkml.tensor(np.array([-3.0, 0.5, 7.0], dtype=np.float32))
>>> vkml.clamp(x, -1.0, 1.0).numpy()
array([-1. ,  0.5,  1. ], dtype=float32)

Implementation

See also clamp_min, clamp_max, maximum, minimum, relu

clamp_min¶

Raise every element to at least min.

clamp_min(x, 0) is relu computed a different way, and the two agree on every input including NaN — which is the point of relu's comparison form.

Parameters

input (Tensor)

Any shape, float dtype.

min (float)

Lower bound, inclusive.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.clamp_min(vkml.tensor(np.array([-2.0, 3.0], dtype=np.float32)), 0.0).numpy()
array([0., 3.], dtype=float32)

Implementation

See also clamp, clamp_max, relu, maximum

clamp_max¶

Lower every element to at most max.

Parameters

input (Tensor)

Any shape, float dtype.

max (float)

Upper bound, inclusive.

Returns

A tensor of the same shape and dtype.

Example

copy>>> vkml.clamp_max(vkml.tensor(np.array([-2.0, 3.0], dtype=np.float32)), 0.0).numpy()
array([-2.,  0.], dtype=float32)

Implementation

See also clamp, clamp_min, minimum