avif Posts - Answer Overflow

avif

Explore posts from servers

•Created by avif on 4/5/2024 in #⚡｜serverless

Having problems working with the `Llama-2-7b-chat-hf`

I have the following request going to the runsync endpoint.

{
    "input": {
        "prompt": "the context. Give me all the places and year numbers listed in the text above"
    }
}

{
    "input": {
        "prompt": "the context. Give me all the places and year numbers listed in the text above"
    }
}

(Full request here: https://pastebin.com/FLqjRzRG) this is the result:

{
    "delayTime": 915,
    "output": {
        "input_tokens": 794,
        "output_tokens": 16,
        "text": [
            " Sure! Here are all the places and year numbers listed in the text:\n"
        ]
    },
    "status": "COMPLETED"
}

{
    "delayTime": 915,
    "output": {
        "input_tokens": 794,
        "output_tokens": 16,
        "text": [
            " Sure! Here are all the places and year numbers listed in the text:\n"
        ]
    },
    "status": "COMPLETED"
}

This is a very bad answer: " Sure! Here are all the places and year numbers listed in the text:\n"` What am I missing? Thanks

7 replies

RRunPod

•Created by avif on 4/5/2024 in #⚡｜serverless

Question about billing

My app is most of the time idle, from time to time i will come with a context and asl my llm model questions about the context. While my app is idle, do I pay anything for this idle time except disk space? Thanks.

6 replies

Gaming

Programming