Simon  πŸ“πŸ› 
Simon πŸ“πŸ› 
Explore posts from servers
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
thanks for the support
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
I figured it out. When disconnecting from Prisma Query Engine it would just freeze the server. I switched from using Hypercorn to Uvicorn and now it works!
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
How do I do that?
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
Ok so I added
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
Which offers plenty of system info during deploy. Although the debugging log stops displaying once prisma is disconnected. After that nothing (There should be callbacks logged at this point). If I try to make any further requests no debugging is displayed at all.
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
I was just confused to why it 'errored' with printing to stderr. The main problem is I am just struggling to work out how to debug this issue because all I get is a FUNCTION_INVOCATION_TIMEOUT when I make calls in production. In Development I get no errors come up and it works fine in development. What would be the best way to debug this?
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
alot of it is print(). Should I use 'structlog' or is there a preference on Railway?
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
I get it that this means that its printing to stderr
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
How come debugging in Deploy Logs is highlighted red with a level: "error" with really no other information besides this?
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
okay i'll try it out. thanks!
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
What is the best way of logging on Railway during API calls?
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
Okay will do. I have tested several different ways to make API calls but it seems once it hits one error or warning it stalls and I cant call it again... I thought it was a hypercorn thing maybe
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
the whole route.ts is as follows:
import { NextResponse, NextRequest } from 'next/server'
import { OpenAIStream, StreamingTextResponse } from 'ai'
export const maxDuration = 300;
export const dynamic = 'force-dynamic'; // always run dynamically

// POST /api/
export async function POST(req: NextRequest) {

const { lotInterestAccess } = await req.json();

try {
// const fetchResponse = await fetch(`${process.env.NODE_ENV !== 'production'? 'http://127.0.0.1:5000' : 'https://ideally-api.up.railway.app'}/ideal/zoneinfo?lotInterestId=${lotInterestAccess.interest.id}&zoneType=${lotInterestAccess.interest.lot.zoneType}&zoneDescription=${lotInterestAccess.interest.lot.zoneDescription}`)
const fetchResponse = await fetch(`${process.env.NODE_ENV !== 'production'? 'http://127.0.0.1:8000' : 'https://ideally.up.railway.app'}/api/parcel/genquery`, {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify({ "messages": [{ role: "user", interest_id: lotInterestAccess.interest.id }] })
})

return new StreamingTextResponse(fetchResponse.body!);
import { NextResponse, NextRequest } from 'next/server'
import { OpenAIStream, StreamingTextResponse } from 'ai'
export const maxDuration = 300;
export const dynamic = 'force-dynamic'; // always run dynamically

// POST /api/
export async function POST(req: NextRequest) {

const { lotInterestAccess } = await req.json();

try {
// const fetchResponse = await fetch(`${process.env.NODE_ENV !== 'production'? 'http://127.0.0.1:5000' : 'https://ideally-api.up.railway.app'}/ideal/zoneinfo?lotInterestId=${lotInterestAccess.interest.id}&zoneType=${lotInterestAccess.interest.lot.zoneType}&zoneDescription=${lotInterestAccess.interest.lot.zoneDescription}`)
const fetchResponse = await fetch(`${process.env.NODE_ENV !== 'production'? 'http://127.0.0.1:8000' : 'https://ideally.up.railway.app'}/api/parcel/genquery`, {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify({ "messages": [{ role: "user", interest_id: lotInterestAccess.interest.id }] })
})

return new StreamingTextResponse(fetchResponse.body!);
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
const fetchResponse = await fetch(`${process.env.NODE_ENV !== 'production'? 'http://127.0.0.1:8000' : 'https://ideally.up.railway.app'}/api/parcel/genquery`, {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify({ "messages": [{ role: "user", interest_id: lotInterestAccess.interest.id }] })
})
const fetchResponse = await fetch(`${process.env.NODE_ENV !== 'production'? 'http://127.0.0.1:8000' : 'https://ideally.up.railway.app'}/api/parcel/genquery`, {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify({ "messages": [{ role: "user", interest_id: lotInterestAccess.interest.id }] })
})
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
A call from next/api is sent to fastAPI via:
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
Yes, sorry, I am sending it to a clients browser. They make an API call from the next.js backend to Railway for this 'gen_query'.
33 replies
RRailway
Created by Simon πŸ“πŸ›  on 8/13/2024 in #βœ‹ο½œhelp
Unable to call Streaming Response from FastAPI in production
Yes its via an API call from a next.js server
33 replies