	In Browser
	StumbleUpon
	del.icio.us
	Google
	Google Buzz
	reddit
	LinkedIn

	Facebook
	Twitter
	Linkedin
	E-Mail

Generative AI > Anthropic Claude API > Claude Vision API - Analyzing Images with Claude

Claude Vision API - Analyzing Images with Claude

Author: Venkata Sudhakar

Claude can analyze images sent as part of the messages array, making it a powerful tool for visual commerce applications. For ShopMax India, vision capabilities power features like product damage detection in return requests, counterfeit identification by comparing product images against catalog photos, and auto-generating alt text and descriptions from product images uploaded by sellers. Claude supports JPEG, PNG, GIF, and WebP formats up to 20MB per image.

Images are passed in the content array alongside text using the image content block type. The source field specifies how the image is provided: type base64 with media_type and data for inline images, or type url for publicly accessible image URLs. Multiple images can be included in a single message, and Claude can reason across all of them together. The image tokens are counted separately and depend on image dimensions - a 1024x1024 image costs approximately 1600 tokens.

The following example shows ShopMax India using Claude vision to analyze product images - detecting damage in return requests and generating catalog descriptions from seller-uploaded photos:

import anthropic
import base64
import os
from pathlib import Path

client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

def encode_image(image_path: str) -> tuple:
    path = Path(image_path)
    suffix = path.suffix.lower()
    media_types = {".jpg": "image/jpeg", ".jpeg": "image/jpeg",
                   ".png": "image/png", ".webp": "image/webp"}
    media_type = media_types.get(suffix, "image/jpeg")
    with open(image_path, "rb") as f:
        data = base64.standard_b64encode(f.read()).decode("utf-8")
    return media_type, data

def analyze_return_image(image_path: str, order_id: str) -> str:
    media_type, data = encode_image(image_path)
    response = client.messages.create(
        model="claude-opus-4-5",
        max_tokens=300,
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "image",
                        "source": {"type": "base64",
                                   "media_type": media_type,
                                   "data": data}
                    },
                    {
                        "type": "text",
                        "text": (
                            "ShopMax India return request for order " + order_id + ". "
                            "Analyze this product image and determine: "
                            "1. Is damage visible? Describe it briefly. "
                            "2. Approve return (Yes/No) based on visible damage. "
                            "3. Suggested action for ShopMax India staff. "
                            "Be concise - 3 lines max."
                        )
                    }
                ]
            }
        ]
    )
    return response.content[0].text

def generate_product_description_from_image(image_url: str, product_name: str) -> str:
    response = client.messages.create(
        model="claude-opus-4-5",
        max_tokens=200,
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "image",
                        "source": {"type": "url", "url": image_url}
                    },
                    {
                        "type": "text",
                        "text": (
                            "Generate a 2-sentence product description for ShopMax India "
                            "catalog based on this image. Product: " + product_name + ". "
                            "Mention visible features and appeal to Indian home buyers."
                        )
                    }
                ]
            }
        ]
    )
    return response.content[0].text

result = analyze_return_image("tv_return_photo.jpg", "ORD-MUM-8821")
print("Return Analysis - ORD-MUM-8821:")
print(result)
print()

desc = generate_product_description_from_image(
    "https://example.com/samsung-tv-product.jpg",
    "Samsung 55-inch 4K QLED TV"
)
print("Generated Catalog Description:")
print(desc)

It gives the following output,

Return Analysis - ORD-MUM-8821:
1. Damage visible: Cracked bottom-left corner of screen with visible LCD bleed.
2. Approve return: Yes - clear manufacturing defect or transit damage.
3. Action: Approve full refund, flag courier for transit damage report, arrange pickup.

Generated Catalog Description:
Bring home a stunning cinematic experience with the Samsung 55-inch 4K QLED TV,
featuring vibrant quantum dot colors and HDR precision that makes every scene
come alive in your Mumbai or Bangalore living room.
Its sleek slim-bezel design and smart Tizen OS with built-in OTT apps make
it the perfect centrepiece for modern Indian households.

For ShopMax India production vision pipelines, resize images to 1024px on the longest side before sending to Claude - this cuts token costs by up to 75% with minimal quality loss for damage detection and description tasks. Use URL-based images instead of base64 when the images are already hosted on your CDN, as this avoids base64 encoding overhead in your API payload. For high-volume return processing (1000+ images per day), combine Claude vision with the Batch API to reduce costs by 50% and process images asynchronously. Always validate the media_type matches the actual file format before sending - a mislabeled image type causes a 400 error from the API.

Send your comments, suggestions or queries regarding this site to [email protected].