pllmPRIVATE INFERENCE
RUNTIME

Build

Define a reproducible PLLM system from configuration, models, components, and research methods.

PLLM documentationView Markdown ↗

SDK configuration records what you intend to build.
Models translate model-family settings into shared operations.
Model plans separate model structure from compiler and runtime support.
Components select versioned capabilities.
Research methods record provenance and plan changes.

Python SDK example

from pllm import Cpu, Model, Pipeline

pipeline = Pipeline.from_profile(
    "public-weight-local",
    model=Model("Qwen/Qwen2.5-0.5B-Instruct"),
    components={"kernel": Cpu(threads=2)},
)
assert pipeline.to_spec()["components"]["kernel"]["params"]["threads"] == 2

API: Python SDK objects and signatures

Models

Turn model-family configuration into a shared decoder plan.

On this page

Python SDK example