Microsoft Azure AI Foundry: Repeated assistant refusal detection

Azure API Management GatewayLogs can be monitored to detect repeated assistant refusals, which may indicate iterative attempts to bypass LLM safety guardrails through techniques such as prompt injection or jailbreaking. This detection pattern tracks three or more completed refusals by the same client, API Management subscription, and model within a 15-minute timeframe.

Impact

Repeated refusals indicate active attempts to elicit disallowed content from AI models. While often benign if stemming from automated testing or shared subscription keys behind NAT, these patterns may represent adversarial activity aimed at identifying model vulnerabilities or bypassing safety policies.

Recommendation

• Configure Azure API Management to emit resource logs (GatewayLogs) and ensure backend response bodies are captured in diagnostics.
• Deploy the provided detection logic to identify clients generating high volumes of refusal responses.
• Investigate alerts by comparing prompt/reply samples against known application behavior; focus on non-standard user agents and unfamiliar geo/ASN sources.
• Block source IPs or rotate subscription keys for entities confirmed to be performing unauthorized prompt injection or jailbreak attempts.