pllmPython SDK
Plan models, compose research methods, and use supported PLLM runtimes from Python.
PLLM ships one Python package backed by Rust. The public API separates model planning from live runtime state so you can inspect and reproduce a plan without including credentials, masks, sessions, or model weights.
Choose an API
| Task | Start here | Result |
|---|---|---|
| Declare a reproducible system | Configuration | Immutable Experiment input |
| Inspect model semantics | Plans | Immutable ModelPlan and coverage |
| Select implementations | Components | Typed capability composition |
| Compile executable regions | Compiler | Verified native plan handles |
| Send native private requests | Python client | Responses and model resources |
| Measure and compare | Benchmark and search | Cohort-bound evidence and candidates |
| Operate provider roles | Deploy and operate | Explicit client, preparation, and inference boundaries |
Use the Python API reference for exact signatures; the authored pages above explain lifecycle, trust, and evidence constraints that a signature cannot express.
Install the package
Add the pllm.run distribution from PyPI to a uv project. It provides the
pllm import package:
uv add pllm.runPython SDK example
pllm.lower_model accepts model configuration and workload bounds. It returns
an immutable semantic ModelPlan; it does not load a checkpoint or start an
inference session.
from pllm import lower_model
config = {
"model_type": "qwen2",
"hidden_size": 64,
"intermediate_size": 192,
"num_hidden_layers": 2,
"num_attention_heads": 4,
"num_key_value_heads": 2,
"vocab_size": 256,
"max_position_embeddings": 128,
"hidden_act": "silu",
"rms_norm_eps": 1e-6,
"rope_theta": 10000.0,
"tie_word_embeddings": True,
}
plan = lower_model(config, batch=1, max_input_tokens=16, max_new_tokens=4)
assert plan.to_dict()["adapter"] == "pllm.qwen2.v1"
assert plan.prefill["output"] == "token_feedback"API: Python SDK objects and signatures
Compose research methods
Apply a compatible component to a model plan to create a new plan with a record of the transformation. Compatibility with a semantic plan does not by itself establish compiler, runtime, privacy, quality, or deployment support.
Run supported workflows
Runtime APIs cover specific public-weight and confidential-weight workflows. Support depends on the model, protocol, operator set, deployment, and threat model. Check current support before choosing a path.
Continue
- Python API reference lists the installed public objects.
- Get started covers installation and the first inspection workflow.
- OpenAI Python SDK connects the official client to the trusted local gateway.
- Runtime explains live state and execution boundaries.
- Models separates semantic planning from executable support.