Just a heads up, but **@cf/google/gemma-
Just a heads up, but @cf/google/gemma-7b-it-lora is returning mostly blank responses for me regardless of prompt.
18 Replies
Unknown User•2d ago
Message Not Public
Sign In & Join Server To View
temperature: 2,
stream: false,
top_k: 20,
top_p: .1,
trying again, no response
i'll try that temp now
Unknown User•2d ago
Message Not Public
Sign In & Join Server To View
btw, did something change recently for local dev wrangler needing approval more often? it used be maybe a few hours / a day, now it seems like its minutes and i need to re-allow wrangler to use AI
Unknown User•2d ago
Message Not Public
Sign In & Join Server To View
this model was responding fine w these params for weeks til today,
tried removing those top params, 0.1 temp, still blank responses with several prompts
other models are working fine, swapped out that one for @cf/meta/llama-3-8b-instruct on the same prompts and they work w same original params/topk etc.
but i liked the responses out of gemma lore better when i tested these prompts initially
Unknown User•2d ago
Message Not Public
Sign In & Join Server To View
yeah i have 5 models in use for text gen across a couple dozen prompts with the same params, but if i can get gemma working with separate params thats fine, but it doesn't seem to
Unknown User•2d ago
Message Not Public
Sign In & Join Server To View
i'm looking for highly structured uncreative responses in some cases
trying again, only param is stream false now, let's see
still blank
Unknown User•2d ago
Message Not Public
Sign In & Join Server To View
i just tried streaming, and not even getting a response from the binding when i pick gemma
only params
stream: true,
temperature: .1
Unknown User•2d ago
Message Not Public
Sign In & Join Server To View
maybe it just doesnt like my prompt since the update, i'll try the same prompt in your CF playground
I hijacked my own prompt for "how are you today?" for multiple requests and it responded
I removed the hijack, no change in parameters, and i get no response from gemma
I think it's related to the prompt length, I just tried a repeat of 'how are you today' 200 times in a single prompt and nothing from gemma, but other models did respond fine
I just sent gemma a long prompt with more structure than that and it also responded with nothing,
the same prompt on the CF playground worked, so it's something to do with the API/binding
Yeah, any prompt I send it above some large number of characters, and I get nothing from gemma via the service binding. All with your recommended params 0.1 etc.
here is a long prompt that doesnt work via the service binding, but does work in CF playground on gemma
Short prompt, 'how are you?' etc. works in both cases.
Unknown User•2d ago
Message Not Public
Sign In & Join Server To View
👍