# Python with PEFT

> Load the pinned base checkpoint, attach an installed adapter with PEFT, and switch it off in the same process.

Source: https://retoque.xyz/docs/python
Markdown: https://retoque.xyz/docs/md/python

Published evaluations run in transformers with PEFT, so this is the tested way to use a patch in your own code. Install the patch with the CLI first, so the signature and every file hash are checked, then point PEFT at the install directory.

## Requirements

- The exact base revision named in the patch manifest. Other revisions, sizes and quantized variants are not supported.
- The library versions the patch was tested with, recorded in the manifest under `runtimeCompatibility.testedVersions` and shown on the patch page. `pip install 'retoque[runtime]'` installs torch, transformers, peft and safetensors.
- About 3.1 GB of disk for the Qwen2.5-1.5B weights and about 7 GB of free memory in float32. CPU works but is slow.

## Load the base and the adapter

Keep remote code off and load weights from safetensors only.

```python
import os

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "Qwen/Qwen2.5-1.5B-Instruct"
REVISION = "989aa7980e4cf806f80c7fef2b1adb7bc71aa306"  # the full commit from the manifest
PATCH_DIR = os.path.expanduser("~/.retoque/patches/<slug>/<version>")

tok = AutoTokenizer.from_pretrained(BASE, revision=REVISION, trust_remote_code=False)
tok.padding_side = "left"
base = AutoModelForCausalLM.from_pretrained(
    BASE,
    revision=REVISION,
    torch_dtype=torch.float32,
    trust_remote_code=False,
    use_safetensors=True,
)
model = PeftModel.from_pretrained(base, PATCH_DIR, is_trainable=False).eval()
```

The install directory only holds the eight package files, so PEFT can only find `adapter_model.safetensors`. There is no pickle file to fall back to.

## Generate with and without the patch

Use the same system prompt and decoding settings the patch was evaluated with. Published results use greedy decoding with a 128-token limit. Sampled decoding is untested.

```python
messages = [
    {"role": "system", "content": SYSTEM_PROMPT},  # the prompt the patch was evaluated with
    {"role": "user", "content": "Today is 2026-03-02 (Monday).\nPay Halcyon Realty $4,200.00 for the office rent, due next Friday."},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    patched = model.generate(**inputs, do_sample=False, max_new_tokens=128, pad_token_id=tok.pad_token_id)
    with model.disable_adapter():
        base_only = model.generate(**inputs, do_sample=False, max_new_tokens=128, pad_token_id=tok.pad_token_id)

new_tokens = inputs["input_ids"].shape[1]
print(tok.decode(patched[0, new_tokens:], skip_special_tokens=True))
```

> [!fix] **Roll back in process.** `with model.disable_adapter():` runs the untouched base inside the same process. The published evaluation compares outputs under `disable_adapter()` with the base run made before the adapter was attached, and the report records whether they matched exactly.

## Check what you loaded

`retoque status` prints the active adapter sha256, and `retoque verify <slug>@<version>` re-checks the installed files. To confirm in Python that the adapter on disk is the one the manifest names:

```python
import hashlib, json, os

manifest = json.load(open(os.path.join(PATCH_DIR, "retoque.manifest.json")))
digest = hashlib.sha256(open(os.path.join(PATCH_DIR, "adapter_model.safetensors"), "rb").read()).hexdigest()
assert digest == manifest["adapter"]["sha256"]
```

## What not to do

- Don't call `merge_and_unload()` and redistribute the result. Merged weights are not a supported or tested form, and a merge can't be undone with `retoque disable`.
- Don't load an adapter from a directory you haven't verified. Run `retoque verify <slug>@<version>` if the files may have changed.
- Don't assume a quantized base behaves the same. It hasn't been evaluated.
- Don't set `trust_remote_code=True` for a patch. Nothing in a Retoque package needs it.