mfeldstein67
Problems with larger models
I'm having trouble consistently running models larger than 70b parameters in webui. They only work maybe one in ten times. When I do get them to work, even if I keep the pod, put it to sleep, and spin it up again later, I get error messages. Here's an example of error messages I'm getting from trying to load a model that I have successfully loaded before using the exact same configuration:
Traceback (most recent call last):
File "/workspace/text-generation-webui/modules/ui_model_menu.py", line 214, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
File "/workspace/text-generation-webui/modules/models.py", line 90, in load_model
output = load_func_maploader
File "/workspace/text-generation-webui/modules/models.py", line 399, in ExLlama_HF_loader
return ExllamaHF.from_pretrained(model_name)
File "/workspace/text-generation-webui/modules/exllama_hf.py", line 174, in from_pretrained
return ExllamaHF(config)
File "/workspace/text-generation-webui/modules/exllama_hf.py", line 31, in init
self.ex_model = ExLlama(self.ex_config)
File "/usr/local/lib/python3.10/dist-packages/exllama/model.py", line 852, in init
self.embed_tokens.weight = nn.Parameter(tensors["model.embed_tokens.weight"])
KeyError: 'model.embed_tokens.weight'
75 replies
Storage contents disappear
There isn't a channel for storage so I'm putting it here. At least half a dozen times in the past couple of weeks, the contents of my storage containers have disappeared. The Ooba template is there but the models and other data are gone. I always delete the storage container and start fresh in case it was a freak corruption, but it keeps happening. Your storage capability does not seem to be reliable. This is a problem for me since I'm using your service for research purposes and don't need or want to be paying for the computing power to be always-on.
10 replies