Devices & introspection
22 functions — 22 documented.
init_vulkan¶
Initialise a Vulkan device and make it available for allocation.
Must be called before any tensor is placed on vulkan:N. Creates the device, the allocator and a staging buffer, so it is a once-per-process cost rather than a per-tensor one.
Parameters
- index (int = 0) optional
- Which physical device, in enumeration order.
Returns
The device string that was initialised.
Enumeration order is not stable across environments. The same machine can report its discrete GPU at index 0 natively and at index 1 inside a container, with a software rasteriser appearing as a third device. Use best_device when the intent is "the fastest one", and select on device_type from vulkan_device_reports when the intent is a specific class of device.
Example
>>> vkml.init_vulkan(0)
'vulkan:0'Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Decisions | ADR 0008 |
| Tests (≥14) | test_device_limits.py test_device_report.py test_invariants.py test_vulkan_kernels.py |
See also best_device, vulkan_device_reports, available_devices
available_devices¶
Every device that can currently hold a tensor.
Returns
A list of device objects, always including cpu.
From the header
Devices with a registered backend.
Implementation
| Declared in | include/vkml/backend/api/backend.h:140 |
|---|---|
| CPU kernel | composed from other operators |
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥1) | test_device_report.py |
See also best_device, init_vulkan
best_device¶
Pick the most capable available device, preferring discrete Vulkan hardware.
Returns
A (device, reason) pair. The reason names the device and why it was chosen, so it can be logged rather than guessed at.
Example
>>> dev, why = vkml.best_device()
>>> print(why) # doctest: +SKIP
using Vulkan device 0: AMD Radeon RX 5600M (RADV NAVI10) (discrete, Vulkan 1.4.354, driver radv)Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥1) | test_device_report.py |
See also init_vulkan, available_devices, vulkan_device_reports
vulkan_available¶
Whether a usable Vulkan device was found.
Returns
True if at least one device passed the required-feature check.
From the header
True when at least one Vulkan device is present.
Implementation
| Declared in | include/vkml/backend/vulkan/vulkan_backend.h:224 |
|---|---|
| CPU kernel | composed from other operators |
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥2) | test_data_and_serialize.py test_device_report.py |
See also vulkan_unavailable_reason, vulkan_capabilities
vulkan_device_count¶
How many Vulkan devices were enumerated.
Returns
An integer, including devices that failed the feature check.
Implementation
| Declared in | include/vkml/backend/vulkan/vulkan_backend.h:226 |
|---|---|
| CPU kernel | composed from other operators |
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥4) | test_device_report.py |
See also vulkan_device_names, vulkan_device_reports
vulkan_device_names¶
The name of each enumerated Vulkan device.
Returns
A list of strings, in enumeration order.
Implementation
| Declared in | include/vkml/backend/vulkan/vulkan_backend.h:228 |
|---|---|
| CPU kernel | composed from other operators |
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥1) | test_device_report.py |
See also vulkan_device_reports, init_vulkan
vulkan_device_reports¶
A full capability report per device.
Each entry carries the device name, device_type (discrete, integrated or cpu), driver_name, the required and optional features, and the limits that decide which kernels can run — subgroup size and its controllable range, workgroup invocations, shared memory, push-constant bytes, memory heaps.
Select on device_type rather than on index when a specific class of device is wanted; the index is not stable across environments.
Returns
A list of dicts, one per device.
From the header
Describes every visible device WITHOUT creating a logical device.
Empty when no Vulkan loader or no device is present, rather than throwing: "this machine has no Vulkan" is an answer a report needs to be able to give.
Implementation
| Declared in | include/vkml/backend/vulkan/vulkan_backend.h:234 |
|---|---|
| CPU kernel | composed from other operators |
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥8) | test_device_report.py |
See also vulkan_capabilities, best_device, init_vulkan
vulkan_capabilities¶
The capability report for one initialised device.
Parameters
- index (int = 0) optional
- Which device.
Returns
A dict of features and limits.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥4) | test_device_limits.py test_invariants.py test_vulkan_kernels.py |
See also vulkan_device_reports
vulkan_stats¶
Allocation and dispatch counters for one device.
Parameters
- index (int = 0) optional
- Which device.
Returns
A dict of counters.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥8) | test_invariants.py test_vulkan_kernels.py |
See also vulkan_pipeline_stats, vulkan_last_profile
vulkan_pipeline_stats¶
Compiler statistics for every pipeline created so far.
Reports what the shader compiler produced per pipeline — register counts, spilled scratch, occupancy — which is how the register model in the GEMM work was fitted and tested. Because each geometry compiles to an independent pipeline through specialisation constants, they benchmark and report separately.
Parameters
- index (int = 0) optional
- Which device.
Returns
A list of dicts, one per pipeline.
Only populated when the driver supports pipeline executable properties. VKML_VULKAN_NO_PIPELINE_STATS=1 disables collection.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Decisions | ADR 0011 |
| Tests (≥3) | test_device_limits.py test_invariants.py test_vulkan_kernels.py |
See also vulkan_stats, vulkan_last_profile
vulkan_timestamps_supported¶
Whether the device can timestamp GPU work.
Parameters
- index (int = 0) optional
- Which device.
Returns
True if timestamp queries are usable.
A device may report valid timestamp bits and then never advance the counter — one virtualised device does exactly that — so a True here is necessary but not sufficient for a meaningful profile.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥2) | test_invariants.py |
See also vulkan_last_profile, vulkan_set_profiling
vulkan_unavailable_reason¶
Why Vulkan is unavailable, when it is.
Names the first required feature that was missing, rather than reporting a generic failure — the required set is bufferDeviceAddress, scalarBlockLayout and timelineSemaphore, and a device lacking any of them cannot run vkML's kernels at all.
Returns
A string, empty when Vulkan is available.
From the header
Empty when a device is visible; otherwise why none is, in a form a person reading a bug report can act on.
"No devices" has two quite different causes -- the loader cannot create an instance at all, or it creates one that enumerates nothing -- and telling them apart is most of the diagnosis on hardware nobody here can inspect.
Implementation
| Declared in | include/vkml/backend/vulkan/vulkan_backend.h:242 |
|---|---|
| CPU kernel | composed from other operators |
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥2) | test_device_report.py |
See also vulkan_available, vulkan_capabilities
vulkan_last_profile¶
Timings from the most recent profiled run.
Parameters
- index (int = 0) optional
- Which device.
Returns
A list of (label, milliseconds) pairs.
Labelled by operator, not by kernel. The entries are real per-operation GPU time, not submission totals — timestamps are written and resolved per node. What they cannot tell you is which KERNEL ran: a matmul entry does not distinguish the naive kernel from the register-blocked one, nor show split-K's partitions. Joining cost to kernel choice needs a shared dispatch identity, which is designed but not yet implemented. Until then, attribution comes from indirect evidence such as batch scaling or device substitution.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥3) | test_invariants.py |
See also vulkan_submit_ms, vulkan_set_profiling, vulkan_pipeline_stats
vulkan_profile_records¶
Measured GPU intervals carrying the identity of what they measured.
The same intervals vulkan_last_profile returns, plus a dispatch field. Join it against decisions() on the same field to answer what did this kernel cost — a question neither side can answer alone.
The profiler deliberately does not know which kernel ran, and the planner deliberately does not know what anything cost. Identity is a third fact, owned by the recorder, describing nothing. That is what stops kernel selection acquiring a second owner.
Parameters
- index (int = 0) optional
- Which device.
Returns
A list of dicts with label, gpu_ms, start_ms, submission and dispatch.
dispatch is 0 when the interval is not a dispatch — the whole-submission entry is the case that exists today. Compare ids for equality only; they are opaque and will widen when multiple queues arrive.
An entry is an INTERVAL, not a duration. start_ms places it inside its own submission, and that is the only admissible way to combine several: a dispatch's bracket closes at a global drain point, so concurrent dispatches each report a window reaching the end of the group. Sum their durations and split-K's sixteen partitions total 2.32 ms inside a 0.20 ms submission; take the union of their intervals and they total 0.196 ms.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥1) | test_attribution.py |
See also vulkan_last_profile, vulkan_profile_history, decisions, record_decisions
vulkan_profile_history¶
Every retained submission's intervals, oldest first.
Group by submission before doing anything else. start_ms is measured from that submission's own window, so intervals from two submissions are not on a common line; and whole-submit windows may be summed across submissions — they are serial — where the intervals inside one may not.
vkml.attribution is the worked consumer: it joins these against decisions() and partitions a step's wall time into GPU busy, GPU idle inside submissions, and host and driver.
Parameters
- index (int = 0) optional
- Which device.
Returns
A list of dicts, the same shape vulkan_profile_records returns.
Empty unless vulkan_set_profile_history asked for retention. Costing nothing until it is asked for is the point.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥2) | test_attribution.py |
See also vulkan_set_profile_history, vulkan_profile_submissions_resolved, vulkan_profile_records
vulkan_set_profile_history¶
Retain the last N submissions' intervals instead of only the most recent.
vulkan_profile_records covers one submission. A CIFAR training step makes thirty-nine, so anything reasoning about a STEP had nothing to read.
Bounded and opt-in: the window drops its oldest submission rather than growing without limit, and 0 disables it and frees what was held. Retention only — nothing here interprets what it stores.
Parameters
- submissions (int)
- How many to keep. 0 disables.
- index (int = 0) optional
- Which device.
Returns
None.
Submissions with no dispatch — a download is a copy — have nothing to time and are never retained, so the window holds fewer submissions than the backend made.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥2) | test_attribution.py |
See also vulkan_profile_history, vulkan_profile_submissions_resolved, vulkan_set_profiling
vulkan_profile_submissions_resolved¶
Submissions offered to the retention window, including any it dropped.
The way to tell a truncated report from a short one. Compare against the distinct submission values in vulkan_profile_history(): if this is larger, the window dropped its oldest entries, every per-kernel row understates, and the difference silently lands in whatever bucket absorbs the remainder.
The parallel to decisions_published, and it exists for the same reason.
Parameters
- index (int = 0) optional
- Which device.
Returns
An integer.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥1) | test_attribution.py |
See also vulkan_profile_history, vulkan_set_profile_history, decisions_published
vulkan_submit_ms¶
Total GPU milliseconds across a profile's submissions.
Parameters
- profile (list)
- The result of
vulkan_last_profile.
Returns
A float.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥1) | test_invariants.py |
See also vulkan_last_profile
vulkan_set_profiling¶
Turn GPU timing on or off for a device.
Parameters
- enabled (bool)
- Whether to record timings.
- index (int = 0) optional
- Which device.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥6) | test_attribution.py test_invariants.py |
See also vulkan_last_profile, vulkan_timestamps_supported
vulkan_set_subgroup_override¶
Force a subgroup width, for testing.
A testing facility rather than a tuning knob. The device reports a controllable range — 32 to 64 on the development GPU — and pinning a width inside it is how a kernel's dependence on subgroup size is tested without different hardware.
Parameters
- index (int = 0) optional
- Which device.
- size (int)
- The width to force. 0 restores the driver's choice.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests | none found by name |
See also vulkan_capabilities
vulkan_synchronize¶
Block until every submission has completed.
Anything timing a region with a host clock must call this before stopping it. Work that has been SUBMITTED has not been measured, and a wall clock stopped early reports the submission cost and none of the execution.
Parameters
- index (int = 0) optional
- Which device.
Returns
None.
Reading a result back — .numpy(), .item() — already waits, so this is for the case where nothing did.
Implementation
| CPU kernel | composed from other operators |
|---|---|
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥4) | test_invariants.py |
See also vulkan_set_profiling, vulkan_profile_history
set_log_level¶
Set the minimum severity vkML logs.
Parameters
- level (LogLevel)
OFF,ERROR,WARN,INFO,DEBUGorTRACE.
VKML_VULKAN_DEBUG=1 raises Vulkan-specific tracing separately, and logs every dispatch with its shape and grid.
From the header
Messages below this level are dropped. Default: Info (Release) / Debug (Debug).
Implementation
| Declared in | include/vkml/util/log.h:45 |
|---|---|
| CPU kernel | composed from other operators |
| Vulkan shader | composed, or dispatched through a shared kernel |
| Gradient rule | none — backward through it raises |
| Tests (≥15) | test_device_limits.py test_device_report.py test_invariants.py test_vulkan_kernels.py |
See also vulkan_stats