Bias Detection and Fairness Testing in LLM Outputs

AI systems can exhibit bias - systematically treating different groups of users or inputs unfairly - even when no bias was intentionally introduced. For ShopMax India, this could manifest as the product recommendation model suggesting premium accessories to users from certain cities while showing budget options to others, or the support chatbot responding differently to queries phrased in formal English versus colloquial Indian English. Detecting and measuring these disparities before deployment is a core AI governance responsibility, required by enterprise clients and emerging Indian AI regulations alike.

Bias detection in LLM outputs involves constructing a test suite of semantically equivalent prompts that vary only in a protected attribute (city, language style, price tier, name) and measuring whether the model's responses differ significantly across groups. Fairness metrics include demographic parity (do different groups get similar recommendation distributions?), equalized odds (does accuracy vary across groups?), and individual fairness (do similar inputs get similar outputs?). The key is to define what fairness means for your specific use case before measuring it.

The example below tests ShopMax India's product recommendation prompt for city-based bias. It sends identical product queries for different Indian cities and compares whether the LLM recommends similar price tiers across all locations, flagging significant disparities as potential bias.

It gives the following output,

Mumbai: Screen Protector - Rs 599
Bangalore: Wireless Earbuds - Rs 1,499
Delhi: Fast Charger 25W - Rs 899
Hyderabad: Screen Protector - Rs 599
Chennai: Leather Case - Rs 799

Average recommended price: Rs 879
Max deviation from average: Rs 620
WARNING: Significant price disparity detected across cities - investigate for bias

Run bias tests as part of your CI/CD pipeline - every time a prompt template changes or a model is updated, re-run the full fairness test suite. For ShopMax India, define fairness benchmarks specific to each AI feature: recommendations should not vary by more than 20% in average price tier across cities; support response quality scores should not differ by more than 10% across query language styles. When bias is detected, the fix is usually in the prompt (add explicit fairness instructions), the training data (balance examples across groups), or the output post-processor (normalize recommendations before serving). Document both the test results and any mitigations in the model card.