|
|
Negative Testing for ADK Agent Tools - Invalid Input Rejection
Author: Venkata Sudhakar
Negative testing for ADK agent tools verifies that invalid, malformed, or out-of-contract inputs are rejected gracefully rather than causing crashes, silent data corruption, or unexpected behavior. ShopMax India runs a dedicated negative test suite against every public-facing tool before each release to ensure that bad inputs from customers in Mumbai and Chennai are met with clear error responses rather than 500 errors or incorrect order modifications.
Negative test cases cover four categories: wrong type (string where int expected), missing required fields, values outside the valid domain (negative IDs, empty strings), and logically inconsistent combinations (return quantity greater than order quantity). Each case must raise a specific exception or return an error dict with a meaningful message - a bare exception with no message is a failing negative test because it gives the caller no recovery path.
The example below applies negative testing to a ShopMax India order tool, covering all four invalid input categories and asserting that each produces a specific, actionable error response.
It gives the following output,
Rejected (Invalid order_id): order_id=12345 qty=1 reason='damaged'
Rejected (positive integer): order_id='ORD-001' qty=-1 reason='damaged'
Rejected (cannot be empty): order_id='ORD-001' qty=1 reason=''
Rejected (exceeds maximum): order_id='ORD-001' qty=15 reason='damaged'
Rejected (positive integer): order_id='ORD-001' qty=0 reason='damaged'
Rejected (Invalid order_id): order_id='' qty=1 reason='damaged'
Valid return: {'order_id': 'ORD-7801', 'qty': 2, 'reason': 'product damaged on arrival', 'status': 'return_initiated'}
7 passed in 0.07s
Always pair negative tests with a positive control test (as shown above) to confirm the tool still works after the negative cases pass - this prevents a tool that rejects everything from passing the negative suite. Use pytest.raises(ValueError, match=...) rather than bare raises so that refactored error messages that lose meaning are caught immediately. Track negative test coverage separately from positive test coverage in CI reports since teams often ignore missing negative cases when looking at overall coverage percentages.
|
|