T
Termux2w ago
JJXB

Vulkan/CLBlast acceleration with koboldcpp

I have a Samsung S21 FE 5G running android 14 and termux from the f-droid repo. I've got hardware acceleration enabled in some fashion as i can run gears via turnip. and i can use termux-x11 just fine to display said gears (vulkaninfo --summary output attached). i use a old version of mesa-vulkan-icd-freedreno that i'm not sure if it supports the modification made to support virtio. if anyone can help me build a version of it that supports it on an unrooted phone, i'd be grateful. i have used multiple versions of koboldcpp including latest, compiled on device with make LLAMA_CLBLAST=1 and LLAMA_VULKAN=1. I also have clvk and clblast installed. when i use --usevulkan 0 to use the "native" passthrough drivers, koboldcpp stops compiling vulkan shaders and crashes. when i use --usevulkan 1 to use the turnip drivers, i do get it to load on smaller models but bigger (read 3-4gb) tend to crash on trying to load and i get a lot of gibberish output to my prompts. and when i use "LD_LIBRARY_PATH=/system/vendor/lib64:$LD_LIBRARY_PATH" to force the device native drivers without a wrapper of any sort, i get a "OSError: dlopen failed: cannot locate symbol "_ZTTNSt6__ndk114basic_ifstreamIcNS_11char_traitsIcEEEE" referenced by "/data/data/com.termux/files/home/koboldcpp/koboldcpp_vulkan.so"..." error. and the same error with CLBlast through the LD_LIBRARY_PATH load. and trying to use clblast on either the wrapper or the turnip driver gives me an error that i can't seem to capture and would be way too long to paste straight into the post. i'll try to capture it if anyone can give a better way of doing so than redirecting with > clblast_log.txt as i use a bash script to list my models and then let me input the gguf file as an option before launching koboldcpp. i've tried looking around for clblast, clvk, vullkan and koboldcpp issues and found nothing of note here on the discord that would help. (yes, running this stuff on phone is silly, i like to tinker)
No description
3 Replies
JJXB
JJXBOP2d ago
am i missing any info? or is this just a problem that has no solution right now?
Biswa96
Biswa962d ago
there is a PR for vulkan hardware rendering https://github.com/termux/termux-packages/pull/22500. It may or may not work. you can test the package from CI artifact.
JJXB
JJXBOP2d ago
not had any luck with it so far but i am testing out if i need to recompile kcpp again nope. recompiling didn't change a thing with vulkan or clblast on the wrapper. i uninstalled the freedreno driver too to make sure i was getting wrapper access for sure

Did you find this page helpful?