Head size 160 is not supported by PagedAttention

Hello, I hope everyone is doing great! I am stuck with this error: ValueError: Head size 160 is not supported by PagedAttention. Supported head sizes are: [64, 80, 96, 112, 128, 256] Does it mean I have to RETRAIN MY MODEL? Full logs are in attachment
1 Reply
nerdylive
nerdylive4mo ago
What are you trying to load? xformers, the library we use for computing attention, only supports certain shapes. I guess this is because their efficient attention implementation requires a memory alignment that is only satisfied with the supported shapes. maybe try deactivating the pagedAttention somehow if you don't wanna use it
Want results from more Discord servers?
Add your server