Map endpoints, run prompt injections, generate SBOMs, and scan vulnerabilities.
(loading)
Secure vs. Insecure
- Secure chatbot UI: http://localhost:5010/chatbot/chatbot_ui (endpoint: /chatbot/ask)
- Insecure chatbot UI: http://localhost:5010/chatbot/chatbot_ui_insecure (endpoint: /chatbot/ask_insecure)
Promptfoo Suites
- Secure fast suite (UI default): suites/promptfoo-api-chat-fast.yaml → /chatbot/ask
- Secure full suite: suites/promptfoo-api-chat.yaml → /chatbot/ask
- Insecure suite (expects leaks): suites/promptfoo-api-chat-insecure.yaml → /chatbot/ask_insecure
- UI buttons let you switch between secure and insecure suites before running.
- Secure fast: PYTHONPATH=src python -m ai_agent_security_suite run-promptfoo suites/promptfoo-api-chat-fast.yaml --no-cache --max-concurrency 1
Run Commands (CLI)
What to Expect
- Secure: refusals / no leaks → promptfoo passes.
- Insecure: hardcoded leaks (passwords, refund, SSN, system prompt) → promptfoo fails.
- Insecure: PYTHONPATH=src python -m ai_agent_security_suite run-promptfoo suites/promptfoo-api-chat-insecure.yaml --no-cache --max-concurrency 1
Toggles in UI
- Promptfoo tab has "Secure" and "Insecure" buttons to set the suite path.
- Docs tab shows this cheat sheet for the demo.