Head size 160 is not supported by PagedAttention
Hello, I hope everyone is doing great! I am stuck with this error: ValueError: Head size 160 is not supported by PagedAttention. Supported head sizes are: [64, 80, 96, 112, 128, 256]
Does it mean I have to RETRAIN MY MODEL?
Full logs are in attachment
1 Reply
What are you trying to load?
xformers, the library we use for computing attention, only supports certain shapes. I guess this is because their efficient attention implementation requires a memory alignment that is only satisfied with the supported shapes.
maybe try deactivating the pagedAttention somehow if you don't wanna use it