	In Browser
	StumbleUpon
	del.icio.us
	Google
	Google Buzz
	reddit
	LinkedIn

	Facebook
	Twitter
	Linkedin
	E-Mail

Generative AI > Anthropic Claude API > Claude Async API - Concurrent Requests with asyncio

Claude Async API - Concurrent Requests with asyncio

Author: Venkata Sudhakar

The AsyncAnthropic client enables concurrent Claude API calls using Python's asyncio, dramatically improving throughput for batch workloads. For ShopMax India, generating product descriptions for 500 new catalog items sequentially at 2 seconds per call takes 17 minutes - running 20 concurrent async calls reduces this to under 1 minute. Async is also essential for building responsive web applications where multiple users send queries simultaneously and each must be handled without blocking others.

The AsyncAnthropic client is a drop-in replacement for the synchronous client, using async/await syntax throughout. asyncio.gather runs multiple coroutines concurrently, while asyncio.Semaphore limits concurrency to stay within API rate limits. The key difference from threading is that async I/O is cooperative - while one request waits for the API response, the event loop runs other coroutines, giving high throughput without the overhead of threads or processes.

The following example shows ShopMax India generating product descriptions for a batch of catalog items concurrently, with semaphore-based rate limiting to stay within API quotas:

import anthropic
import asyncio
import os
import time

async_client = anthropic.AsyncAnthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

PRODUCTS = [
    {"id": "SM-TV-55Q", "name": "Samsung 55-inch 4K QLED", "price": 54990},
    {"id": "LG-TV-50N", "name": "LG 50-inch NanoCell", "price": 44990},
    {"id": "IFB-WM-65F", "name": "IFB 6.5kg Front Load", "price": 23490},
    {"id": "DKN-AC-15S", "name": "Daikin 1.5T Inverter Split AC", "price": 38990},
    {"id": "SNY-TV-43B", "name": "Sony 43-inch Bravia Google TV", "price": 38990},
]

async def generate_description(product: dict, semaphore: asyncio.Semaphore) -> dict:
    async with semaphore:
        prompt = (
            "Write one sentence product description for ShopMax India catalog. "
            "Product: " + product["name"] + " at Rs " + str(product["price"]) + ". "
            "Highlight key feature and value for Indian customers. Max 25 words."
        )
        response = await async_client.messages.create(
            model="claude-haiku-4-5",
            max_tokens=60,
            messages=[{"role": "user", "content": prompt}]
        )
        return {
            "id": product["id"],
            "name": product["name"],
            "description": response.content[0].text
        }

async def generate_all_descriptions(products: list, concurrency: int = 5) -> list:
    semaphore = asyncio.Semaphore(concurrency)
    tasks = [generate_description(p, semaphore) for p in products]
    return await asyncio.gather(*tasks)

async def main():
    start = time.time()
    results = await generate_all_descriptions(PRODUCTS, concurrency=5)
    elapsed = time.time() - start
    for r in results:
        print(r["id"] + ": " + r["description"])
    print()
    print("Generated", len(results), "descriptions in", round(elapsed, 2), "seconds")

asyncio.run(main())

It gives the following output,

SM-TV-55Q: Experience stunning 4K Quantum HDR visuals with Samsung's 55-inch QLED
at Rs 54,990 - a premium home theatre upgrade for Indian living rooms.
LG-TV-50N: LG's 50-inch NanoCell at Rs 44,990 delivers accurate colors and
Dolby Vision for an immersive viewing experience at great value.
IFB-WM-65F: The IFB 6.5kg Front Load at Rs 23,490 offers energy-efficient
washing with hard water compatibility perfect for Indian homes.
DKN-AC-15S: Daikin's 1.5T Inverter Split AC at Rs 38,990 provides powerful
cooling with 5-star energy rating for year-round comfort.
SNY-TV-43B: Sony Bravia 43-inch at Rs 38,990 features Google TV with built-in
OTT apps for seamless streaming in compact Indian apartments.

Generated 5 descriptions in 1.43 seconds

For ShopMax India batch jobs, set concurrency between 10 and 20 for Haiku and between 3 and 5 for Opus - higher concurrency on cheaper models is safe since they have higher rate limits. Add exponential backoff inside the semaphore block to handle 429 rate limit responses without failing the entire batch. For very large batches (1000+ items), use asyncio.Semaphore combined with the Batch API for 50% cost savings - async handles the submission and polling while Batch API processes items asynchronously on Anthropic's side.

Send your comments, suggestions or queries regarding this site to [email protected].