Cloudflare Developers•8mo ago

fetch for Anthropic Claude3 never returns

Hello I trying to send a large prompt to Claude3, 66000 (characters) with the below code. After starting it never returns and I get no error back. - If I run the same code with a small prompt (100 characters) it works. - If I run the same code in a node.js or python app with (66000 size prompt) it works fine. It seems to be something related to the workers fetch. I try to add "node_compatibility=true" in wrangler.tom but it had no effect.

    const prompt = "My large prompt ...." // small prompt like "Generate a python program to print numbers from 1 to 100" works
    const messages = [
        {
            role: "user",
            content: [{
                type: "text",
                text: prompt
            }]
        }
    ]
    const data = {
        "model": "claude-3-sonnet-20240229",
        "max_tokens": 4096,
        "messages": messages
    }
    const fetchData = await fetch("https://api.anthropic.com/v1/messages",
        {
            method: 'POST',
            headers: {
                'accept': 'application/json',
                'Content-Type': 'application/json',
                'anthropic-version': '2023-06-01',
                'x-api-key': env.ANTHROPIC_API_KEY   
            },  
            body: JSON.stringify(data)
        }
    )

    const prompt = "My large prompt ...." // small prompt like "Generate a python program to print numbers from 1 to 100" works
    const messages = [
        {
            role: "user",
            content: [{
                type: "text",
                text: prompt
            }]
        }
    ]
    const data = {
        "model": "claude-3-sonnet-20240229",
        "max_tokens": 4096,
        "messages": messages
    }
    const fetchData = await fetch("https://api.anthropic.com/v1/messages",
        {
            method: 'POST',
            headers: {
                'accept': 'application/json',
                'Content-Type': 'application/json',
                'anthropic-version': '2023-06-01',
                'x-api-key': env.ANTHROPIC_API_KEY   
            },  
            body: JSON.stringify(data)
        }
    )

Thanks for any help

1 Reply

BonadioOP•8mo ago

I did some more testing and it seems to be a Workers fetch -> Anthropic API problem. Running the same code with OpenAI gpt-4-turbo works well. Seems that I found the problem. The fetch from workers timeout after +- 80 seconds and it closes the connection. Even trying to do streaming it stops in the middle after the timeout

Gaming

Programming

fetch for Anthropic Claude3 never returns