PromptDrift — prompt regression testing
Python · asyncio · pytest · GitHub ActionsYou tweak a prompt to fix one bad case, it works, you ship it — and three other cases broke silently. PromptDrift snapshots prompt behaviour into a committed baseline and fails CI when a change makes things worse.
It refuses to answer “is this prompt good?”, which is unanswerable, and answers “is it worse than yesterday?” — the question that actually blocks a merge. Cheap deterministic checks run first and short-circuit the paid LLM-as-judge calls.