erajasekar
erajasekar
Explore posts from servers
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
Added logging for exception handling gives me this stack trace:
asyncio.exceptions.CancelledError: Cancelled by cancel scope 7fa2af3bd900



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

File "/app/api/v1/chat.py", line 60, in streaming_response

async for chunk in llm_service.continue_chat(user_id, session_id, chat_request.message):

File "/app/services/llm_service.py", line 128, in continue_chat

async for chunk in self.systhesis_video_transcript_stream(user_id, session['transcript']):

File "/app/services/llm_service.py", line 88, in systhesis_video_transcript_stream

async for chunk in chain.astream(input="Synthesize the video transcript."):

File "/usr/local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3430, in astream

async for chunk in self.atransform(input_aiter(), config, **kwargs):

File "/usr/local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3413, in atransform

async for chunk in self._atransform_stream_with_config(

File "/usr/local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2332, in _atransform_stream_with_config

await run_manager.on_chain_error(e, inputs=final_input)

asyncio.exceptions.CancelledError: Cancelled by cancel scope 7fa2af3bd900
asyncio.exceptions.CancelledError: Cancelled by cancel scope 7fa2af3bd900



During handling of the above exception, another exception occurred:



Traceback (most recent call last):

File "/app/api/v1/chat.py", line 60, in streaming_response

async for chunk in llm_service.continue_chat(user_id, session_id, chat_request.message):

File "/app/services/llm_service.py", line 128, in continue_chat

async for chunk in self.systhesis_video_transcript_stream(user_id, session['transcript']):

File "/app/services/llm_service.py", line 88, in systhesis_video_transcript_stream

async for chunk in chain.astream(input="Synthesize the video transcript."):

File "/usr/local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3430, in astream

async for chunk in self.atransform(input_aiter(), config, **kwargs):

File "/usr/local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 3413, in atransform

async for chunk in self._atransform_stream_with_config(

File "/usr/local/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2332, in _atransform_stream_with_config

await run_manager.on_chain_error(e, inputs=final_input)

asyncio.exceptions.CancelledError: Cancelled by cancel scope 7fa2af3bd900
Not sure what is causing this
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
async CancelledError. Attaching the logs and source code of original application that is failing. ( minimal example version works fine after increasing token size, it's still not clear how it works on local with 512 token size and not from Railway ).
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
Thank you @Brody . The example application works after increasing token count. But my actual application still fails. will troubleshoot.
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
This is based on what I find from troubleshooting. My goal to not to pin point problem with the platform and but to figure out a solution for my issue. Let me know if this is something that could be resolved through support or I should move to another platform instead of spending time in trouble shooting this. Thank you for being with me in debugging this.
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
Ok, it appears that there is some issue related to outbound communication from Railway to OpenAI related to streaming.
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
Hi @Brody , Is there any proxy used for outbound communication from Railway servers to external apis ? I use a python library for extracting youtube transcripts which works fine in Railway, but it doesn't work in render. When I researched further I found that youtube blocks requests from cloud machine servers and possible solution to get around is to use a proxy. But youtube is not blocking requests from Railway
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
I have to use open api for my product. I doubt it's issue in openapi python library as it is used in chatgpt and many other applications.
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
This is a minimum MVP example to confirm that I could deploy my app in Railway.
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
Thanks, It's very tricky problem. I tried lot of things on application level, but nothing worked.
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
hmm. Thank you for example. But it's sending events after every second. what happens when delay between events is more than 1 second and random. The example I provided has minimal code of my application and it is using openai's streaming sdk. If you show if my minimal example working or help me troubleshoot, I would really appreciate. Thank you for your support on weekend.
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
FYI - I deployed same exact code on another clould provider render and it works fine https://streamingfastapiexample.onrender.com/. I am guess that there is some http connection or timeout parameter we need to tweak on Railway.
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
Thank you @Brody Here is minimal example to reproduce. App url : https://streamingfastapiexample-production.up.railway.app/ - when you click streaming response will stream, but it will stop before finishing. Code : https://github.com/erajasekar/StreamingFastAPIExample ( Use master branch)
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
Thank you @Brody . I tried lot of troubleshooting options today like changing from guvicorn to uvicorn , setting keep-alive timeout in uvicorn and refactoring code to handle timeout etc. But none of it worked. Then, I installed docker and ran my app inside docker and it works perfectly fine. I will attach debug logs from local and inside railway app to see if that helps. I have spent whole day today trying to debug this and no luck. Any help will be greatly appreciated.
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
@Brody I don't have this issue when running locally. I also tried increasing async time out using middle ware and no luck. The response streams only for few seconds
38 replies
RRailway
Created by erajasekar on 9/21/2024 in #✋|help
Getting CancelledError Python FastAPI application using openai chat completion
ab55d7b2-61b7-4e97-b893-4b10bd69492b
38 replies
RRailway
Created by erajasekar on 4/6/2023 in #✋|help
Using JSON File as Environment variable in Railway for Firebase
cool, thank you for quick response
37 replies
RRailway
Created by erajasekar on 4/6/2023 in #✋|help
Using JSON File as Environment variable in Railway for Firebase
ok, I already have docker file. Do you have example code snippet copy contents env variable to local file.json ?
37 replies
RRailway
Created by erajasekar on 4/6/2023 in #✋|help
Using JSON File as Environment variable in Railway for Firebase
This is for python FastAPI project and not nodejs project.
37 replies