[SOLVED] AUR llama.cpp and paru question

Succulent of your garden · 2025-12-30 16:15:57

So a couple of days ago I installed https://aur.archlinux.org/packages/llama.cpp-cuda to use it in a project for fine tuning an LLM. But in the end I just ended with building the llama.cpp inside the uv venv to make my code works. But that gave me some questions. I needed to use this command to convert my model to gguf fomrmat:

convert_hf_to_gguf.py final_model     --outfile model-F16.gguf --outtype f16     --split-max-size 50G

But I failed to do so. But that makes me wonder: is there a way to use a command like this but into a venv ? the command has been installed as a host dependency, but in the end I needed llama.cpp to be installed in the venv. But after I had done with fine tuning the model. Is there a way to be able to launch system hosts binaries [commands] in my virtual environment but also be able to interact with all the dependencies that has been installed in the venv ?

The other question has to be with AUR usage. As I installed llama.cpp from AUR now I got his when I need to update the packages with paru -Sua:

:: Looking for devel upgrades...
:: Resolving dependencies...
:: Calculating conflicts...
:: Calculating inner conflicts...
:: marked out of date: gcc14  gcc14-libs  llama.cpp-cuda

Repo Make (11) fmt-12.1.0-1  spdlog-1.16.0-1  doxygen-1.15.0-1  expect-5.45.4-5
        dejagnu-1.6.3-15  inetutils-2.6-1  python-iniconfig-2.1.0-1  python-packaging-25.0-1
        python-pluggy-1.6.0-1  python-pygments-2.19.2-1  python-pytest-1:8.4.2-1
Aur (1) gcc14-14.3.1+r416+g44d5743651c4-2 (gcc14-libs gcc14)

:: Proceed to review? [Y/n]:

As you can see there is a lot of packages, but i'm not sure if those ones are just needed for the build and then dropped out or it also needed to install it in my host system. Not sure why is trying to update gcc14 for example. It seems it's gonna update in the host system dependency, and not as a build thing necesary just to install llama.cpp. I'm not sure if this can make me start a dependency hell, so that's why I'm asking. Not sure to be honest. All other packages of AUR in my system are pretty simple and most of the time doesn't show this kind of needs.

Last edited by Succulent of your garden (2026-01-06 12:56:38)

seth · 2025-12-30 20:09:59

https://aur.archlinux.org/packages/llama.cpp-cuda lists only cmake as make dependency, you can check

pacman -Qi gcc14

to see why it's installed, possibly cuda-pascal?

is there a way to use a command like this but into a venv ?

https://wiki.archlinux.org/title/Python … ment#Usage
You just create/activate the venv and run it there?

loqs · 2025-12-30 20:51:23

When you ran `convert_hf_to_gguf.py` had you installed python-gguf? Or was this from the llama.cpp-cuda package in which case you would have need at least python-sentencepiece from the optional dependencies the package lists.

Succulent of your garden · 2026-01-01 12:26:38

seth wrote:

to see why it's installed, possibly cuda-pascal?

Depends On : gcc14-libs binutils>=2.28 libmpc zstd libisl.so=23-64

What do you think ? doesn't seem like cuda-pascal for me

seth wrote:

You just create/activate the venv and run it there?

I created the venv with uv, and tried to launch the commmand with that in mind, with the shell with the venv activated, like source /bin/activate also, apart from the uv way of doing that. But it doesn't worked. It seem that framework in which I'm making the fine tuning does need to have the complete llama.cpp compiled in the venv folder. With that I was able to fix the issue.

Currently now I'm talking with the devs of Unsloth [the framework for fine tuning], and they said to me that I should not compile the thing, that it's an issue from the framework. But they are going to check it, but with that I was able to run the code and start fine tuning some models . So I created a temporal fix on that ^^

loqs wrote:

When you ran `convert_hf_to_gguf.py` had you installed python-gguf? Or was this from the llama.cpp-cuda package in which case you would have need at least python-sentencepiece from the optional dependencies the package lists.

No I don't have python-gguf installed as a dependency system and also I don't have sentencepiece. Do you think with that could work ? I mean llama-cli can create instances of models in my machine using the AUR pacakge, but unable to export to gguf. Didn't hear about sentencepiece before.

loqs · 2026-01-01 14:50:35

What is the output from `convert_hf_to_gguf.py final_model --outfile model-F16.gguf --outtype f16 --split-max-size 50G` when the command fails?

seth · 2026-01-01 16:23:04

What do you think ?

I think you're looking for the "Required By" line

Succulent of your garden · 2026-01-06 12:56:23

loqs wrote:

What is the output from `convert_hf_to_gguf.py final_model --outfile model-F16.gguf --outtype f16 --split-max-size 50G` when the command fails?

This:

Traceback (most recent call last):
  File "/usr/bin/convert_hf_to_gguf.py", line 19, in <module>
    from transformers import AutoConfig
ModuleNotFoundError: No module named 'transformers'

I'm assuming that the host installation binary of the llama.cpp is checking for a host binary of transformers library, which it does exists, but in the virtual environment, and not as a host depency. I'm assuming that llama.cpp is failing because it need to have all other dependencies installed as host ones and not in the virtual environment, long story short: Asking for global installation, instead of local in the venv. I guess what I'm doing is like impossible without installing transformer with AUR right ? which defeats in some kind the efforts of using a venv ;C

seth wrote:

I think you're looking for the "Required By" line tongue tongue

I think so , gonna mark solved cuz llama.cpp I think just really needs the dependencies in host. Manual building inside the venv works fine.

Arch Linux

#1 2025-12-30 16:15:57

[SOLVED] AUR llama.cpp and paru question

#2 2025-12-30 20:09:59

Re: [SOLVED] AUR llama.cpp and paru question

#3 2025-12-30 20:51:23

Re: [SOLVED] AUR llama.cpp and paru question

#4 2026-01-01 12:26:38

Re: [SOLVED] AUR llama.cpp and paru question

#5 2026-01-01 14:50:35

Re: [SOLVED] AUR llama.cpp and paru question

#6 2026-01-01 16:23:04

Re: [SOLVED] AUR llama.cpp and paru question

#7 2026-01-06 12:56:23

Re: [SOLVED] AUR llama.cpp and paru question

Board footer