owns the device
OPEN SOURCE · LINUX FOUNDATION · APACHE-2.0
Kompute
The general purpose GPU compute framework for cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). Blazing fast, mobile-enabled, asynchronous, and optimized for advanced GPU acceleration usecases. Donated by the Institute and backed by the Linux Foundation as a hosted project of the LF AI & Data Foundation.
01 — THE MENTAL MODEL
The mental model
Four concepts carry the whole framework: a Manager owns the device, Tensors own data, an Algorithm wraps the shader logic, and a Sequence records operations to submit as a batch. Everything else is Vulkan doing what Vulkan does, without the boilerplate.
THE KOMPUTE MENTAL MODEL
One tensor, from host data to GPU results
Manager creates the device context
owns the data
wraps shader logic
records operations
executes the batch
SEQUENCE
01recordOpSyncDevice
→02dispatchOpAlgoDispatch
→03sync backresults on host
HOST TENSORWaiting for sequence
02 — TWO LANGUAGES, ONE API
Two languages, one API
The Python module covers experimentation; the C++ SDK goes as deep as the optimisation requires. Asynchronous and parallel execution comes through GPU family queues, and the BYOV design (“bring-your-own-Vulkan”) plugs into existing Vulkan applications.
# 1. Create Kompute Manager with default settings (device 0, first queue and no extensions)
mgr = kp.Manager()
# 2. Create and initialise Kompute Tensors through manager
# Default tensor constructor simplifies creation of float values
tensor_in_a = mgr.tensor([2, 2, 2])
tensor_in_b = mgr.tensor([1, 2, 3])
# Explicit type constructor supports uint32, int32, double, float and bool
tensor_out_a = mgr.tensor_t(np.array([0, 0, 0], dtype=np.uint32))
tensor_out_b = mgr.tensor_t(np.array([0, 0, 0], dtype=np.uint32))
assert(t_data.data_type() == kp.DataTypes.uint)
params = [tensor_in_a, tensor_in_b, tensor_out_a, tensor_out_b]
# 3. Create algorithm based on shader (supports buffers & push/spec constants)
workgroup = (3, 1, 1)
spec_consts = [2]
push_consts_a = [2]
push_consts_b = [3]
# See documentation shader section for compile_source
spirv = compile_source(shader)
algo = mgr.algorithm(params, spirv, workgroup, spec_consts, push_consts_a)
# 4. Run operation synchronously using sequence
(mgr.sequence()
.record(kp.OpSyncDevice(params))
.record(kp.OpAlgoDispatch(algo)) # Binds default push consts provided
.eval() # evaluates the two recorded ops
.record(kp.OpAlgoDispatch(algo, push_consts_b)) # Overrides push consts
.eval()) # evaluates only the last recorded op// 1. Create Kompute Manager with default settings (device 0, first queue and no extensions)
kp::Manager mgr;
// 2. Create and initialise Kompute Tensors through manager
// Default tensor constructor simplifies creation of float values
auto tensorInA = mgr.tensor({ 2., 2., 2. });
auto tensorInB = mgr.tensor({ 1., 2., 3. });
// Explicit type constructor supports uint32, int32, double, float and bool
auto tensorOutA = mgr.tensorT<uint32_t>({ 0, 0, 0 });
auto tensorOutB = mgr.tensorT<uint32_t>({ 0, 0, 0 });
std::vector<std::shared_ptr<kp::Memory>> params = {tensorInA, tensorInB, tensorOutA, tensorOutB};
// 3. Create algorithm based on shader (supports buffers & push/spec constants)
kp::Workgroup workgroup({3, 1, 1});
std::vector<float> specConsts({ 2 });
std::vector<float> pushConstsA({ 2.0 });
std::vector<float> pushConstsB({ 3.0 });
auto algorithm = mgr.algorithm(params,
// See documentation shader section for compileSource
compileSource(shader),
workgroup,
specConsts,
pushConstsA);
// 4. Run operation synchronously using sequence
mgr.sequence()
->record<kp::OpSyncDevice>(params)
->record<kp::OpAlgoDispatch>(algorithm) // Binds default push consts
->eval() // Evaluates the two recorded operations
->record<kp::OpAlgoDispatch>(algorithm, pushConstsB) // Overrides push consts
->eval(); // Evaluates only last recorded operation03 — PROVEN WHERE IT COUNTS
Proven where it counts
Kompute backs on-device LLM inference in GPT4ALL, appeared as a backend in llama.cpp, and powers vkJAX, the JAX interpreter for Vulkan. It runs on Android via the NDK, on Raspberry Pi through Mesa, and in the Godot engine for game development.