owns the device
OPEN SOURCE · LINUX FOUNDATION · APACHE-2.0
Kompute
The general purpose GPU compute framework for cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). Blazing fast, mobile-enabled, asynchronous, and optimized for advanced GPU acceleration usecases. Donated by the Institute and backed by the Linux Foundation as a hosted project of the LF AI & Data Foundation.
01 — THE MENTAL MODEL
The mental model
Four concepts carry the whole framework: a Manager owns the device, Tensors own data, an Algorithm wraps the shader logic, and a Sequence records operations to submit as a batch. Everything else is Vulkan doing what Vulkan does, without the boilerplate.
THE KOMPUTE MENTAL MODEL
One tensor, from host data to GPU results
Manager creates the device context
owns the data
wraps shader logic
records operations
executes the batch
SEQUENCE
01recordOpSyncDevice
→02dispatchOpAlgoDispatch
→03sync backresults on host
HOST TENSORWaiting for sequence
02 — TWO LANGUAGES, ONE API
Two languages, one API
The Python module covers experimentation; the C++ SDK goes as deep as the optimisation requires. Asynchronous and parallel execution comes through GPU family queues, and the BYOV design ("bring-your-own-Vulkan") plugs into existing Vulkan applications.
# 1. Create the manager (device 0, first queue, no extensions)
mgr = kp.Manager()
# 2. Create tensors through the manager
# Default constructor creates float tensors
tensor_in_a = mgr.tensor([2, 2, 2])
tensor_in_b = mgr.tensor([1, 2, 3])
# Typed constructor: uint32, int32, double, float, bool
tensor_out_a = mgr.tensor_t(np.array([0, 0, 0], dtype=np.uint32))
tensor_out_b = mgr.tensor_t(np.array([0, 0, 0], dtype=np.uint32))
assert(t_data.data_type() == kp.DataTypes.uint)
params = [tensor_in_a, tensor_in_b, tensor_out_a, tensor_out_b]
# 3. Build the algorithm from the shader
workgroup = (3, 1, 1)
spec_consts = [2]
push_consts_a = [2]
push_consts_b = [3]
# compile_source is in the shader docs
spirv = compile_source(shader)
algo = mgr.algorithm(params, spirv, workgroup, spec_consts, push_consts_a)
# 4. Run the sequence synchronously
(mgr.sequence()
.record(kp.OpSyncDevice(params))
.record(kp.OpAlgoDispatch(algo)) # Default push consts
.eval() # Runs both recorded ops
.record(kp.OpAlgoDispatch(algo, push_consts_b)) # New push consts
.eval()) # Runs only the last op// 1. Create the manager (device 0, first queue, no extensions)
kp::Manager mgr;
// 2. Create tensors through the manager
// Default constructor creates float tensors
auto tensorInA = mgr.tensor({ 2., 2., 2. });
auto tensorInB = mgr.tensor({ 1., 2., 3. });
// Typed constructor: uint32, int32, double, float, bool
auto tensorOutA = mgr.tensorT<uint32_t>({ 0, 0, 0 });
auto tensorOutB = mgr.tensorT<uint32_t>({ 0, 0, 0 });
std::vector<std::shared_ptr<kp::Memory>> params = {tensorInA, tensorInB, tensorOutA, tensorOutB};
// 3. Build the algorithm from the shader
kp::Workgroup workgroup({3, 1, 1});
std::vector<float> specConsts({ 2 });
std::vector<float> pushConstsA({ 2.0 });
std::vector<float> pushConstsB({ 3.0 });
auto algorithm = mgr.algorithm(params,
// compileSource is in the shader docs
compileSource(shader),
workgroup,
specConsts,
pushConstsA);
// 4. Run the sequence synchronously
mgr.sequence()
->record<kp::OpSyncDevice>(params)
->record<kp::OpAlgoDispatch>(algorithm) // Default push consts
->eval() // Runs both recorded ops
->record<kp::OpAlgoDispatch>(algorithm, pushConstsB) // New push consts
->eval(); // Runs only the last op03 — PROVEN WHERE IT COUNTS
Proven where it counts
Kompute backs on-device LLM inference in GPT4ALL, appeared as a backend in llama.cpp, and powers vkJAX, the JAX interpreter for Vulkan. It runs on Android via the NDK, on Raspberry Pi through Mesa, and in the Godot engine for game development.