LLMJuryDocs
Open app

Python SDK

Install from PyPI (Python 3.8+, zero runtime dependencies — the transport is stdlib urllib):

pip install llmjury

Authentication

The client reads your org's publishable key (llmj_pk_…) from the LLMJURY_API_KEY environment variable, so Client() with no arguments just works once that's set. You can also pass it explicitly — Client(api_key="llmj_pk_…"). The key is sent as X-API-Key on every request. The SDK defaults to the production API; point it at a local stack with LLMJURY_BASE_URL (or base_url=). See where keys come from.

Quickstart

from llmjury import Client

# ---- setup, once at startup -------------------------------------------------
# Reads LLMJURY_API_KEY from the environment; defaults to https://api.llmjury.com.
client = Client(experiments=["checkout-prompt"])          # prefetch by experiment NAME
llm = client.wrap(anthropic_client, "checkout-prompt")    # every model call is now traced

# ---- per request ------------------------------------------------------------
with client.as_user(user_id):
    # Variant prompt from client memory; your in-code default survives an outage.
    p = client.get_prompt("checkout-prompt", user_id, default="You are a helpful assistant.")
    # Call your provider client DIRECTLY — latency/tokens/errors are intercepted.
    response = llm.messages.create(model="claude-haiku-4-5", max_tokens=1024,
                                   system=p.prompt,
                                   messages=[{"role": "user", "content": user_input}])

# The ONLY explicit metric: your business outcome.
client.track("business_event", {"experiment_id": "checkout-prompt", "user_id": user_id,
                                "variant": p.variant, "business_metric": "conversion", "value": 1})

What the client does

  • get_prompt(experiment, user, default) — the recommended entrypoint: resolves the user straight to their variant's prompt text. default is your in-code fallback, returned whenever the config isn't cached, assignment fails, or the variant has no prompt — the app always has a working prompt, even during a full LLMJury outage. Returns (variant, prompt, fallback).
  • get_variables(experiment, user, defaults) — same idea for custom variables (model, temperature, …): the variant's configured values merged over your in-code defaults, from client memory — never an API call on the request path.
  • wrap(provider_client, experiment) + as_user(user) — setup-once interception: wrap your OpenAI/Anthropic-style client at startup, bind the user per request with with client.as_user(user_id):, then call the provider directly. Every model call records the exposure plus a model_call event with measured latency, tokens, model, and errors — no call-site code. (intercept_model_call is the explicit context-manager form.)
  • assign(experiment, user) — the low-level call: deterministic local bucketing against the polled config; returns the variant key, or None until the config loads — treat None as "use control". See bucketing.
  • track(event, payload) — enqueue-only; a background thread flushes batches with bounded retries, then spills to an optional offline file (replayed within 24h with original timestamps) or drops with a log line. It never blocks or raises into your call path. With wrap(), the only event you track by hand is your business_event.
  • Experiments are addressed by unique name or id — names resolve through the polled config and bucketing always runs on the canonical id, so both address forms assign identically.
  • Config is polled from GET /v1/config (ETag/304, 60s) and refreshed immediately when an ingest ack reports a newer config_version. Async variants: aassign / atrack.

Full details in the SDK README.