	In Browser
	StumbleUpon
	del.icio.us
	Google
	Google Buzz
	reddit
	LinkedIn

	Facebook
	Twitter
	Linkedin
	E-Mail

Agentic AI > ADK Agent Testing > Automated Test Data Generation for ADK Agents Using LLM

Automated Test Data Generation for ADK Agents Using LLM

Author: Venkata Sudhakar

Writing test cases manually for every ShopMax India query type is time-consuming and biased toward scenarios the developer has already thought of. Using an LLM to generate test data produces a broader, more diverse set of inputs covering edge cases, regional variations, and unusual phrasings that manual test writing often misses. The generator can be run on demand to refresh the test suite whenever a new agent category is added.

The generator sends a structured prompt to the LLM asking it to return a JSON array of test case objects. Each object contains a query, a list of required keywords expected in a good response, and a difficulty level. The result is parsed and validated to ensure the structure matches the expected schema before being used in downstream tests.

The example below mocks the LLM generator and validates that three auto-generated order tracking test cases have the correct fields, valid difficulty levels, and at least one required keyword each.

import json
import pytest
from unittest.mock import MagicMock

GENERATOR_PROMPT_TEMPLATE = (
    "Generate {count} test cases for a ShopMax India {category} agent. "
    "Return a JSON array. Each item must have: "
    "query (string), required_keywords (list of strings), difficulty (easy/medium/hard). "
    "Use Indian city names and realistic order IDs."
)

def parse_test_cases(json_text):
    data = json.loads(json_text)
    if not isinstance(data, list):
        raise ValueError("Expected a list of test cases")
    for item in data:
        if "query" not in item or "required_keywords" not in item or "difficulty" not in item:
            raise ValueError("Missing required fields: " + str(item))
    return data

def generate_test_cases(category, count, model):
    prompt = GENERATOR_PROMPT_TEMPLATE.format(count=count, category=category)
    response = model.generate_content(prompt)
    return parse_test_cases(response.text)

MOCK_JSON = json.dumps([
    {"query": "Track order ORD-7821 from Mumbai", "required_keywords": ["ORD-7821", "shipped", "delivery"], "difficulty": "easy"},
    {"query": "Why is my Samsung TV order ORD-9034 delayed in Chennai?", "required_keywords": ["ORD-9034", "delay", "expected"], "difficulty": "medium"},
    {"query": "I ordered a phone 3 weeks ago from Hyderabad and never got a delivery update", "required_keywords": ["escalate", "tracking", "support"], "difficulty": "hard"},
])

def test_generate_order_tracking_test_cases():
    mock_model = MagicMock()
    mock_model.generate_content.return_value = MagicMock(text=MOCK_JSON)
    test_cases = generate_test_cases("order_tracking", 3, mock_model)
    assert len(test_cases) == 3
    for tc in test_cases:
        assert "query" in tc
        assert "required_keywords" in tc
        assert "difficulty" in tc
        assert tc["difficulty"] in ("easy", "medium", "hard"), (
            "Invalid difficulty: " + tc["difficulty"]
        )
        assert len(tc["required_keywords"]) > 0, "Empty keywords for: " + tc["query"]
    difficulties = [tc["difficulty"] for tc in test_cases]
    print("Generated " + str(len(test_cases)) + " test cases: " + str(difficulties))

It gives the following output,

Generated 3 test cases: ['easy', 'medium', 'hard']
. (1 passed in 0.01s)

In production, call the real model with GENERATOR_PROMPT_TEMPLATE to build a fresh test bank for every new agent feature. Store the generated JSON to a file and commit it to source control so the test suite is reproducible without re-calling the LLM on every CI run. Add a schema validation step using Pydantic to catch malformed outputs from the generator before they cause cryptic failures in downstream test cases.

Send your comments, suggestions or queries regarding this site to [email protected].