Builderman
Builderman
PostsComments
RRunPod
Created by justin on 5/9/2024 in #⚡｜serverless
How to WebSocket to Serverless Pods
we just spam the request to get all chunks
125 replies
RRunPod
Created by justin on 5/9/2024 in #⚡｜serverless
How to WebSocket to Serverless Pods
considering my method of streaming yes
125 replies
RRunPod
Created by justin on 5/9/2024 in #⚡｜serverless
How to WebSocket to Serverless Pods
is it possible to stream with this? websockets are definitely more efficient
125 replies
RRunPod
Created by houmie on 5/1/2024 in #⚡｜serverless
Which version of vLLM is installed on Serverless?
base_events.py      :1771 2024-05-08 01:23:56,733 Task exception was never retrieved
2024-05-08T01:23:56.735080453Z future: <Task finished name='Task-8514' coro=<_process_job() done, defined at /usr/local/lib/python3.11/dist-packages/runpod/serverless/worker.py:41> exception=UnicodeDecodeError('utf-8', b'\xff\xff\xff\xff\xff\x00\x06', 0, 1, 'invalid start byte')>
2024-05-08T01:23:56.735088543Z Traceback (most recent call last):
2024-05-08T01:23:56.735095700Z   File "/usr/local/lib/python3.11/dist-packages/runpod/serverless/worker.py", line 55, in _process_job
2024-05-08T01:23:56.735105243Z     await stream_result(session, stream_output, job)
2024-05-08T01:23:56.735118162Z   File "/usr/local/lib/python3.11/dist-packages/runpod/serverless/modules/rp_http.py", line 76, in stream_result
2024-05-08T01:23:56.735128930Z     await _handle_result(session, job_data, job, JOB_STREAM_URL, "Intermediate results sent.")
2024-05-08T01:23:56.735138994Z   File "/usr/local/lib/python3.11/dist-packages/runpod/serverless/modules/rp_http.py", line 50, in _handle_result
2024-05-08T01:23:56.735158189Z     await _transmit(session, url, serialized_job_data)
2024-05-08T01:23:56.735169194Z   File "/usr/local/lib/python3.11/dist-packages/runpod/serverless/modules/rp_http.py", line 37, in _transmit
2024-05-08T01:23:56.735178882Z     await client_response.text()
2024-05-08T01:23:56.735187102Z   File "/usr/local/lib/python3.11/dist-packages/aiohttp/client_reqrep.py", line 1147, in text
2024-05-08T01:23:56.735196332Z     return self._body.decode(  # type: ignore[no-any-return,union-attr]
2024-05-08T01:23:56.735204926Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-08T01:23:56.735211906Z UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
base_events.py      :1771 2024-05-08 01:23:56,733 Task exception was never retrieved
2024-05-08T01:23:56.735080453Z future: <Task finished name='Task-8514' coro=<_process_job() done, defined at /usr/local/lib/python3.11/dist-packages/runpod/serverless/worker.py:41> exception=UnicodeDecodeError('utf-8', b'\xff\xff\xff\xff\xff\x00\x06', 0, 1, 'invalid start byte')>
2024-05-08T01:23:56.735088543Z Traceback (most recent call last):
2024-05-08T01:23:56.735095700Z   File "/usr/local/lib/python3.11/dist-packages/runpod/serverless/worker.py", line 55, in _process_job
2024-05-08T01:23:56.735105243Z     await stream_result(session, stream_output, job)
2024-05-08T01:23:56.735118162Z   File "/usr/local/lib/python3.11/dist-packages/runpod/serverless/modules/rp_http.py", line 76, in stream_result
2024-05-08T01:23:56.735128930Z     await _handle_result(session, job_data, job, JOB_STREAM_URL, "Intermediate results sent.")
2024-05-08T01:23:56.735138994Z   File "/usr/local/lib/python3.11/dist-packages/runpod/serverless/modules/rp_http.py", line 50, in _handle_result
2024-05-08T01:23:56.735158189Z     await _transmit(session, url, serialized_job_data)
2024-05-08T01:23:56.735169194Z   File "/usr/local/lib/python3.11/dist-packages/runpod/serverless/modules/rp_http.py", line 37, in _transmit
2024-05-08T01:23:56.735178882Z     await client_response.text()
2024-05-08T01:23:56.735187102Z   File "/usr/local/lib/python3.11/dist-packages/aiohttp/client_reqrep.py", line 1147, in text
2024-05-08T01:23:56.735196332Z     return self._body.decode(  # type: ignore[no-any-return,union-attr]
2024-05-08T01:23:56.735204926Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-08T01:23:56.735211906Z UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
63 replies
RRunPod
Created by houmie on 5/1/2024 in #⚡｜serverless
Which version of vLLM is installed on Serverless?
not sure if runpod is down
63 replies
RRunPod
Created by houmie on 5/1/2024 in #⚡｜serverless
Which version of vLLM is installed on Serverless?
no changes on my end
63 replies
RRunPod
Created by houmie on 5/1/2024 in #⚡｜serverless
Which version of vLLM is installed on Serverless?
@Alpay Ariyak hey, i've been using worker-vllm for months now and the streaming broke all of a sudden
63 replies
RRunPod
Created by Builderman on 2/19/2024 in #⚡｜serverless
Mixtral Possible?
kk thanks
21 replies
RRunPod
Created by Builderman on 2/19/2024 in #⚡｜serverless
Mixtral Possible?
what GPU do you use?
21 replies
RRunPod
Created by Builderman on 1/13/2024 in #⚡｜serverless
streaming
theres gotta be a way
5 replies