|
|
Constitutional AI Prompting - Self-Critique and Revision
Author: Venkata Sudhakar
Constitutional AI prompting asks the LLM to critique its own output against a set of principles and then revise it. At ShopMax India, auto-generated customer communication drafts can be checked for tone (not too formal, not too casual), accuracy (no fabricated delivery dates), and policy compliance (no unauthorized refunds) before sending. The LLM acts as its own reviewer, catching issues before they reach the customer.
The workflow has three steps: generate an initial response, critique it by checking each principle in the constitution, and revise based on the critique. The constitution is a short list of rules specific to your use case. This self-refinement loop can run once or multiple times. One revision pass typically catches the most obvious issues; diminishing returns set in after two passes for most customer communication tasks.
The example below shows ShopMax India using a two-step constitutional AI prompt for customer complaint responses. The initial response is critiqued against four principles, and a revised response is generated from the critique.
It gives the following output,
Initial response:
Dear customer, I sincerely apologize for the inconvenience. Your LG washing machine
noise issue will be resolved within 24-48 hours. Our technician will contact you
shortly. As compensation for the trouble, we are offering you a Rs 500 cashback.
Critique:
1. ACCURACY: VIOLATION - states "24-48 hours" which is not confirmed in context.
2. TONE: COMPLIANT - warm and empathetic tone used.
3. BREVITY: COMPLIANT - under 80 words.
4. POLICY: VIOLATION - offers Rs 500 cashback which is not in standard policy.
Revised response:
Dear customer, I completely understand your frustration with your LG washing
machine. I have escalated this as a priority case and our service team will
contact you soon to schedule a technician visit. We are committed to resolving
this quickly. Thank you for your patience.
At ShopMax India, build your constitution from real incidents - every time a support agent catches an LLM error in a review, add it as a constitutional principle. Start with 4-6 core principles and expand carefully - too many principles create conflicting constraints that confuse the revision. For high-volume channels like WhatsApp support, run one critique-and-revise cycle before sending. For email escalations to angry customers, run two cycles to ensure the response is polished before it reaches a human reviewer for final sign-off.
|
|