Testing ADK Agent Instruction-Following Accuracy

Instruction-following accuracy testing measures how reliably an ADK agent obeys explicit constraints in its system prompt or user instructions. ShopMax India's customer service agent is instructed to always respond in English, never reveal internal pricing logic, and always include an order ID when discussing order status - and these constraints must be verified programmatically in CI rather than relying on manual review across hundreds of daily conversations in Mumbai and Delhi.

Each instruction is expressed as a verifiable constraint - a Python function that takes a response string and returns True if the constraint is satisfied. An InstructionChecker runs all constraints against the agent's response and collects failures. Tests supply a response that should pass all constraints, a response that violates one specific constraint, and assert the correct failure is reported. This makes each instruction independently testable and gives precise failure messages in CI.

The example below defines four ShopMax India agent constraints, builds an InstructionChecker, and tests both a fully compliant response and responses that violate individual constraints.

It gives the following output,

Constraint failures: []
Constraint contains_order_id correctly flagged
Constraint no_internal_pricing correctly flagged
Constraint ends_with_helpline correctly flagged
4 passed in 0.06s

Write one constraint function per instruction so that each can be toggled independently as business rules change. Store the constraint set in a central instructions.py module so that all agent test files import the same source of truth. Run instruction-following tests after every system prompt change since even minor prompt rewording can cause an agent to drop a required element like the order ID or helpline number from its responses.