Manoj
when embedded with FastAPI is there a
yes, I do think the same as well, we do have load balancing, but I notice that when we pass small text as input to model it works perfect but it's only when processing large text (around 1000 words) is when it's breaking and disconnecting. The model is converted to ONNX and is placed within the container itself
19 replies
when embedded with FastAPI is there a
no it takes longer and doesn't reconnect and I manually have to refresh the page. Is there a predefined timeout set on Solara UI that checks for server response? I see the server processed the input and by the time it sends back the server disconnects
19 replies
when embedded with FastAPI is there a
sl.Style(css)
with sl.Card(margin=30): with sl.Column(margin=30, align="start"): with sl.Row(): sl.Markdown(ner_description)
with sl.Card(margin=30): sl.InputText("Enter Input Text", value=text, continuous_update=True) with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.Button("Execute", color="green", on_click = predictions) sl.Button("Clear", color="primary", on_click = clear, on_value = prev_text)
if loading: with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.SpinnerSolara(size="200px") elif content != "": with sl.Card(margin=30): sl.Markdown("NER Model Response: ") with sl.Row(): sl.HTML(style= "margin: auto; font-size: 16px; color: black;", classes=["body"], unsafe_innerHTML=content) with sl.Card(margin=20): with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.Button("Copy To Clipboard", color="#d14d4d", on_click=copy_to_clipboard)
with sl.Card(margin=30): with sl.Column(margin=30, align="start"): with sl.Row(): sl.Markdown(ner_description)
with sl.Card(margin=30): sl.InputText("Enter Input Text", value=text, continuous_update=True) with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.Button("Execute", color="green", on_click = predictions) sl.Button("Clear", color="primary", on_click = clear, on_value = prev_text)
if loading: with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.SpinnerSolara(size="200px") elif content != "": with sl.Card(margin=30): sl.Markdown("NER Model Response: ") with sl.Row(): sl.HTML(style= "margin: auto; font-size: 16px; color: black;", classes=["body"], unsafe_innerHTML=content) with sl.Card(margin=20): with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.Button("Copy To Clipboard", color="#d14d4d", on_click=copy_to_clipboard)
19 replies
when embedded with FastAPI is there a
attaching the code for your reference
@sl.component
def Page():
MyHome()
prev_text, set_prev_text = sl.use_state("") text = sl.use_reactive("") content, set_content = sl.use_state("") loading, set_loading = sl.use_state(False)
def predictions(): if text.value.strip() != "" and text.value != prev_text: set_prev_text(str(text.value)) try: set_loading(True) param = {"msg": str(text.value.strip())} res = requests.get(fhttp://0.0.0.0:8080/predict_response, params = param, verify=False) sl.Markdown(res.text) set_content(res.text) set_loading(False) except Exception as e: sl.Markdown(e)
def clear(): set_content("") text.set("") set_prev_text("")
def copy_to_clipboard(): pyperclip.copy(content)
MyHome()
prev_text, set_prev_text = sl.use_state("") text = sl.use_reactive("") content, set_content = sl.use_state("") loading, set_loading = sl.use_state(False)
def predictions(): if text.value.strip() != "" and text.value != prev_text: set_prev_text(str(text.value)) try: set_loading(True) param = {"msg": str(text.value.strip())} res = requests.get(fhttp://0.0.0.0:8080/predict_response, params = param, verify=False) sl.Markdown(res.text) set_content(res.text) set_loading(False) except Exception as e: sl.Markdown(e)
def clear(): set_content("") text.set("") set_prev_text("")
def copy_to_clipboard(): pyperclip.copy(content)
19 replies
when embedded with FastAPI is there a
@MaartenBreddels thanks for helping on this issue. In production the server disconnects every now and then during execution and below is the pod logs
INFO: 10.141.12.1:45012 - "HEAD / HTTP/1.1" 200 OK
INFO: ('10.141.12.1', 45002) - "WebSocket /jupyter/api/kernels/d89f2b18-9ecb-41b8-909c-d258f2f7e56b/channels?session_id=c4304626-18b5-4778-bff7-0a41498d23a1" [accepted]
INFO: connection open
INFO: connection closed
INFO: connection closed
INFO: 10.141.4.1:49444 - "HEAD / HTTP/1.1" 200 OK
19 replies