Kernels

pllm-core implements bounded integer matrices, scalar reference code, runtime AVX2 and NEON selection, codecs, quantization, masking, subtraction, and operating system randomness. It copies validated weights into Rust once and reuses them. Python-to-Rust buffer conversions still create copies.

Kernel records list required CPU features, threading, input and output formats, supported operations, memory use, and evidence. Moving arithmetic to Rust does not remove protocol messages, offline preparation, client state, or conversion costs.

Pipeline selection and direct execution

Cpu is the component placed in a typed pipeline. MaskedGEMM, CompiledMatrix, and capabilities() are lower-level native APIs for direct execution and deployment inspection. Runtime SIMD chooses scalar, AVX2, or NEON internally after checking the host; those paths are not separate component classes.

API: Kernel objects and signatures

Python SDK example

Select the pipeline backend with Cpu

Cpu commits a thread count into the pipeline and owns the persistent native worker pool used by matrix stages. It does not choose the protocol or remove transport and preparation costs.

from pllm import Deployment, ExecutionBudget, Experiment
from pllm.kernels import Cpu
from pllm.profiles import MaskedLinearCpu
from pllm.sources import TinyModel

experiment = Experiment(
    name="two-thread-cpu",
    pipeline=MaskedLinearCpu(
        TinyModel(model_id="cpu-tiny"),
        kernels=Cpu(threads=2),
    ),
    deployment=Deployment.local(root="local://two-thread-cpu"),
    budget=ExecutionBudget(requests=1, max_input_tokens=64, max_new_tokens=1),
)
resolved = experiment.resolve()
kernel = experiment.pipeline.to_spec()["components"]["kernels"]
assert kernel["component"] == "pllm/cpu"
assert kernel["params"] == {"threads": 2}

Save the module as experiment.py and run pllm gateway --local --experiment experiment.py:experiment --trust-python to use that exact kernel choice in private loopback inference. Use pllm benchmark run --experiment experiment.py:experiment --trust-python --max-output-tokens 1 --repetitions 1 to measure it rather than inferring performance from selection.

Create a direct matrix engine with MaskedGEMM

MaskedGEMM exposes clear, modular, coefficient, wrap32, and wrap64 integer matrix operations. Use it for kernel integration or measurement, not as a substitute for a private protocol.

import numpy as np

from pllm.native import MaskedGEMM

engine = MaskedGEMM(threads=2)
weights = np.array([[1, -2], [3, 4]], dtype=np.int8)
matrix = engine.compile(weights)
result = matrix.clear(np.array([[5, 6]], dtype=np.int8))
assert result.tolist() == [[-7, 39]]

Reuse an immutable CompiledMatrix

Obtain CompiledMatrix from MaskedGEMM.compile(). It copies validated weights once; later source-array mutation does not change the native snapshot.

import numpy as np

from pllm.native import CompiledMatrix, MaskedGEMM

weights = np.array([[1, -2], [3, 4]], dtype=np.int8)
matrix = MaskedGEMM(threads=1).compile(weights)
weights[:] = 0
result = matrix.clear(np.array([[5, 6]], dtype=np.int8))
assert isinstance(matrix, CompiledMatrix)
assert result.tolist() == [[-7, 39]]

Inspect the installed backend with capabilities()

capabilities() reports whether the native extension is compiled and which backend is active. Inspect it before making deployment claims; a research fallback does not establish native evidence.

from pllm.native import capabilities

installed = capabilities()
assert isinstance(installed["compiled"], bool)
assert installed["implementation"] in {"rust", "python-reference"}

For protocol-level composition, return to masked linear. For the exact lower-level signatures, see the native matrix interface.