pllmThe client owns prompts, decoding, private state, masks, and token boundaries.
Private inference, built to improve
Run an LLM without giving one provider the plaintext conversation.
PLLM is a high-performance private LLM multi-party inference runtime and autonomous research harness. It keeps prompts, activations, and model state inside the client boundary while separate services perform the expensive linear work.
One-time masked corrections are produced offline, then consumed during inference.
Research components are composed into typed plans and compared by the same benchmark contract.
TrustedClientplaintext + state
OfflinePreparationmasked corrections
OnlineInferencemasked tensors
Privacy requires preparation and inference to follow the protocol and not collude. PLLM does not claim protection against arbitrary malicious providers.
Runtime + research harness
One system for execution and improvement
Private inference
- Prepare. The client authorizes an inventory. Preparation expands fresh seeds, computes
W·r−s, and pushes the corrections to inference. - Reserve. Before a response, the client reserves one-use rows for every remote stage. Cancellation and failure burn unused rows.
- Execute. The client sends a ticket and
x−r. Inference consumes the matching correction and returnsW·x−s. The client addss. - Decode. Token lookup, output projection, sampling, and model state remain with the client.
Online path
Clientticket + masked activation
Inferenceone-time masked result
Clientreconstruct + decode
Preparation is idle while a response runs. The online path is client-to-inference only.
Autonomous research loop
Build a library of interchangeable research components.
Papers are reimplemented behind capability-based contracts: cache policies, numeric approximations, nonlinear protocols, matrix protocols, preparation schemes, kernels, and placement strategies. New capability families can be added as research evolves.
01Specify02Implement03Assure04Benchmark05Compose06Search
Like-for-like components live together under stable typed interfaces. Paper names remain attribution, not runtime architecture. Compatibility rules reject invalid combinations before execution.
Current position
Working runtime, incomplete full-model coverage
Available
Responses API and Chat Completions API through a trusted loopback gateway; prepared public-weight transport; Qwen runtime path; semantic model planning; benchmark dashboard and evidence records.
Experimental
Bounded Q7 SiLU arithmetic garbling and MPCache plan transformation. These components do not establish complete-model execution or production security.
Not complete
The compiler cannot yet execute a complete Qwen plan under the preferred protected profile. Multi-host, WAN, GPU, energy, price, quality, and adversarial studies remain unperformed.
Complete Qwen coverage
Close every semantic, numeric, protected, placement, state, and token-feedback gap. Partial plans continue to fail closed.
Grow the component library
Reimplement relevant papers as reusable components, validate each source-scoped claim, and preserve its trust model.
Search compatible plans
Start with constrained grid and randomized search, then add smarter search only when repeated benchmark data justifies it.
Evidence rule
Compare complete plans, not isolated claims.
Every result binds model, graph, component versions, numeric profile, placement, trust assumptions, machine, workload, and cold/warm state. Functional correctness, privacy review, model quality, latency, communication, memory, and cost remain separate measurements.