ItsWendell
Explore posts from serversCDCloudflare Developers
•Created by Kathy on 4/24/2025 in #ai-gateway
AI Gateway + Vertex AI Context Caching
If it helps, here's an AI Gateway succesful attempt log id: 01JSPQTPQVA6JEARQSZSMWTR57
3 replies
CDCloudflare Developers
•Created by Kathy on 4/24/2025 in #ai-gateway
AI Gateway + Vertex AI Context Caching
Hi @Kathy, thanks for getting back to this. I've made the wrong assumption that I couldn't proxy that cachedContents requests through AI Gateway, it does actually work, which is fantastic.
Google does tell you what tokens were used from the cache, and how many tokens you've used in the cache.
Here's a couple of references.
Vertex AI response:
https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#response
Cached content usage metadata:
https://cloud.google.com/vertex-ai/generative-ai/docs/reference/rest/v1/projects.locations.cachedContents#UsageMetadata
3 replies
CDCloudflare Developers
•Created by Kathy on 4/24/2025 in #ai-gateway
AI Gateway + Vertex AI Context Caching
3 replies
CDCloudflare Developers
•Created by fforres on 4/18/2025 in #workflows
I haven't had any issues so far but also
If I undestand it correctly, when a Workflow re-initializes after sleep, retries or any other reason, the promises of your
await step.do calls that were already successfully executed would just instantly resolve.
That means that re-defining state should be fine and your workflow should work as expected!3 replies
CDCloudflare Developers
•Created by ItsWendell on 4/21/2025 in #workflows
Workflow run in "waiting" state
Hmh I've reworked how I handle retrying / fault tolerance in my workflow, unsure how this happened because neither is the case but let's see how it goes. Haven't run into it again today after this.
3 replies
CDCloudflare Developers
•Created by ItsWendell on 4/21/2025 in #workflows
Workflow run in "waiting" state
3 replies