Dockerless dev and deploy, async handler need to use async ?
handler.py in HelloWorld project, there is not 'async' before def handler(job): . But in serverless endpoint, there are Run and RunSync . So if I want to use async handler, Is it necessary to go like this ? Or am I misunderstanding something?
async def async_generator_handler(job):
for i in range(5):
output = f"Generated async token output {i}" yield output for a large language model await asyncio.sleep(1) runpod.serverless.start( { "handler": async_generator_handler,
"return_aggregate_stream": True, } )
output = f"Generated async token output {i}" yield output for a large language model await asyncio.sleep(1) runpod.serverless.start( { "handler": async_generator_handler,
"return_aggregate_stream": True, } )
2 Replies
I don't understand the question, but here are docs for an async handler.
https://docs.runpod.io/serverless/workers/handlers/handler-async
By the way
run and runsync are independent of your handler type. You can still use run even if your handler is not async.Asynchronous Handler | RunPod Documentation
RunPod supports the use of asynchronous handlers, enabling efficient handling of tasks that benefit from non-blocking operations. This feature is particularly useful for tasks like processing large datasets, interacting with APIs, or handling I/O-bound operations.
Understood . For my ai generate image , use async handler is a better choice ?
Given that serverless has already implemented the creation of a new worker for each new request, would it be sufficient to use a synchronous handler, without the need for an asynchronous handler? Since I want customers to receive their portraits as quickly as possible, and each worker's GPU is dedicated solely to processing the task of generating that customer's portrait，not for other customer.