STRRL Comments - Answer Overflow

I think in the future, there would be more OSS projects or AI API Gateway(like Kong AI Gateway, our team is alaway exploring on it https://docs.konghq.com/gateway/latest/get-started/ai-gateway/#ai-provider-proxy ) services make the role of "API Adaptor", the echosystem is growing.

22 replies

CDCloudflare Developers

•Created by ben_makes_stuff on 4/20/2024 in #ai-gateway

Groq support & cases for Custom base URL

BTW if cloudflare AI Gateway could also works as "litellm", always provide OpenAI Compatible API for different upstream, that would be extramlly helpful.

22 replies

CDCloudflare Developers

•Created by ben_makes_stuff on 4/20/2024 in #ai-gateway

Groq support & cases for Custom base URL

that already becomes defacto for nearly all the openai compatible APIs. also there is an selfhosted api gateway called litellm, which could work as an adaptor between lots of different LLM API providers and "OpenAI compatible apis". https://github.com/BerriAI/litellm

22 replies

CDCloudflare Developers

•Created by ben_makes_stuff on 4/20/2024 in #ai-gateway

Groq support & cases for Custom base URL

the base url is https://api.groq.com/openai/v1, for groq

22 replies

CDCloudflare Developers

•Created by ben_makes_stuff on 4/20/2024 in #ai-gateway

Groq support & cases for Custom base URL

in other words, we could call gpt-3.5-turbo or gpt-4 on openai by sending http request to https://api.openai.com/v1/chat/completions. And on groq, we could use the "openai comtaible API" with mixtral and llama 2/llama 3 by sending http reqeust to https://api.groq.com/openai/v1/chat/completions.

22 replies

CDCloudflare Developers

•Created by ben_makes_stuff on 4/20/2024 in #ai-gateway

Groq support & cases for Custom base URL

for sure. here is the baseurl setup for openai compatible API, explained from groq's documentation: https://console.groq.com/docs/openai#url

22 replies

Gaming

Programming