dave
dave
PostsComments
CDCloudflare Developers
Created by dave on 6/28/2024 in #ai-gateway
When using AI Gateway with Vertex AI (
but it's still a streaming response, no? Did you see the difference between using AI Gateway vs not?
24 replies
CDCloudflare Developers
Created by dave on 6/28/2024 in #ai-gateway
When using AI Gateway with Vertex AI (
with alt=sse it streams
24 replies
CDCloudflare Developers
Created by dave on 6/28/2024 in #ai-gateway
When using AI Gateway with Vertex AI (
it "works" in both situations. But when you use AI Gateway, the first request buffers instead of streams. When you don't use AI Gateway, both requests streams.
24 replies
CDCloudflare Developers
Created by dave on 6/28/2024 in #ai-gateway
When using AI Gateway with Vertex AI (
and you'll spot the difference
24 replies
CDCloudflare Developers
Created by dave on 6/28/2024 in #ai-gateway
When using AI Gateway with Vertex AI (
watch them in your terminal
24 replies
CDCloudflare Developers
Created by dave on 6/28/2024 in #ai-gateway
When using AI Gateway with Vertex AI (
curl -v --request POST \
  --url "https://gateway.ai.cloudflare.com/v1/REMOVED/REMOVED/google-vertex-ai/v1/projects/REMOVED/locations/us-east4/publishers/google/models/gemini-1.5-pro-002:streamGenerateContent" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header 'content-type: application/json' \
  --data '{"contents": {"role": "user","parts": [{"text": "tell me a long story"}]}}'

curl -v --request POST \
  --url "https://gateway.ai.cloudflare.com/v1/REMOVED/REMOVED/google-vertex-ai/v1/projects/REMOVED/locations/us-east4/publishers/google/models/gemini-1.5-pro-002:streamGenerateContent?alt=sse" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header 'content-type: application/json' \
  --data '{"contents": {"role": "user","parts": [{"text": "tell me a long story"}]}}'
curl -v --request POST \
  --url "https://gateway.ai.cloudflare.com/v1/REMOVED/REMOVED/google-vertex-ai/v1/projects/REMOVED/locations/us-east4/publishers/google/models/gemini-1.5-pro-002:streamGenerateContent" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header 'content-type: application/json' \
  --data '{"contents": {"role": "user","parts": [{"text": "tell me a long story"}]}}'

curl -v --request POST \
  --url "https://gateway.ai.cloudflare.com/v1/REMOVED/REMOVED/google-vertex-ai/v1/projects/REMOVED/locations/us-east4/publishers/google/models/gemini-1.5-pro-002:streamGenerateContent?alt=sse" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header 'content-type: application/json' \
  --data '{"contents": {"role": "user","parts": [{"text": "tell me a long story"}]}}'
24 replies
CDCloudflare Developers
Created by dave on 6/28/2024 in #ai-gateway
When using AI Gateway with Vertex AI (
compare these two requests:
24 replies
CDCloudflare Developers
Created by dave on 6/28/2024 in #ai-gateway
When using AI Gateway with Vertex AI (
I can still reproduce.
24 replies
CDCloudflare Developers
Created by dave on 6/28/2024 in #ai-gateway
When using AI Gateway with Vertex AI (
lemme try, one sec
24 replies