Cloudflare Developers•5mo ago

Worker AI Usage

Hi, how can I get statistics of workers AI usage? I don't see any api documentation for it? Is it possible?

3 Replies

Craig•5mo ago

Hi there @Tojak have you seen AI Gateway? https://developers.cloudflare.com/ai-gateway/providers/workersai/

Cloudflare Docs

Workers AI · Cloudflare AI Gateway docs

Use AI Gateway for analytics, caching, and security on requests to Workers AI.

Craig•5mo ago

I think this might be exactly what you are looking for (and more!)

shincurry•5mo ago

OpenAI's chat-completion API returns token usage information the data returned by the non-stream API contains

{
  // ...
  usage?: {
    /**
     * Number of tokens in the generated completion.
     */
    completion_tokens: number;

    /**
     * Number of tokens in the prompt.
     */
    prompt_tokens: number;

    /**
     * Total number of tokens used in the request (prompt + completion).
     */
    total_tokens: number;
  }
}

{
  // ...
  usage?: {
    /**
     * Number of tokens in the generated completion.
     */
    completion_tokens: number;

    /**
     * Number of tokens in the prompt.
     */
    prompt_tokens: number;

    /**
     * Total number of tokens used in the request (prompt + completion).
     */
    total_tokens: number;
  }
}

The stream API also returns usage in the last chunk of the stream. Does cloudflare plan to return this information? This is very useful for statistics.

Gaming

Programming

Worker AI Usage