Responsible AI Checklist
A pragmatic pre-flight checklist for shipping LLM features through Meridian. Walk through these three sections before exposing any model-backed endpoint to real users. The goal is not perfect safety theater; it is catching the obvious failure modes that erode trust and burn budget.
1. Scope & intent
Write down what the feature does in one sentence and what it explicitly does not do. If you cannot describe the user benefit without hedging, the scope is too wide. Bound the model by a system prompt that names the domain and refuses out-of-domain requests politely.
- Single-sentence purpose statement committed to the repo.
- Out-of-scope topics enumerated in the system prompt.
- Audience identified: internal staff, paying customer, anonymous visitor.
2. Data handling
Decide what enters the prompt and what leaves the response. Strip PII before the request and log request IDs, not raw bodies. Meridian routes through Azure adaptive routing by default, so confirm the deployment region matches your data residency commitment.
- PII redaction step in the request pipeline.
- Request log retention bounded to 30 days unless contracted otherwise.
- Region pinning verified via the model alias.
3. Evaluation & guardrails
Ship with a 25-prompt eval set covering happy path, adversarial input, and edge cases. Wire a simple judge model to score new responses against the baseline before each deploy. Add a kill-switch env var so the feature can be disabled without a code push.
# meridian.yaml feature: support_chat model: azure/model-router kill_switch_env: MERIDIAN_SUPPORT_CHAT_OFF eval_suite: evals/support_chat_v1.jsonl guardrails: pii_redact: true max_tokens: 1024 refuse_out_of_domain: true