shincurry
shincurry
PostsComments
CDCloudflare Developers
Created by Tojak on 6/18/2024 in #general-help
Worker AI Usage
OpenAI's chat-completion API returns token usage information the data returned by the non-stream API contains 
{
  // ...
  usage?: {
    /**
     * Number of tokens in the generated completion.
     */
    completion_tokens: number;

    /**
     * Number of tokens in the prompt.
     */
    prompt_tokens: number;

    /**
     * Total number of tokens used in the request (prompt + completion).
     */
    total_tokens: number;
  }
}
{
  // ...
  usage?: {
    /**
     * Number of tokens in the generated completion.
     */
    completion_tokens: number;

    /**
     * Number of tokens in the prompt.
     */
    prompt_tokens: number;

    /**
     * Total number of tokens used in the request (prompt + completion).
     */
    total_tokens: number;
  }
}
The stream API also returns usage in the last chunk of the stream. Does cloudflare plan to return this information? This is very useful for statistics.
4 replies