Score:3

Ubuntu

Install llama.cpp locally?

Pablo

3/30/24, 7:15 AM

Hey I'm looking for install llama.cpp from this github link... May someone help me please ? There is no ubuntu tutorial on youtube and I don't want to follow chatGPT for something so important. Thanks

5576

1 + 5

github

22.04

pLumo

3/30/24, 7:49 AM

I use Alpaca, a fork of Llama. I think it's easier to install and use, installation is straightforward. -> https://github.com/antimatter15/alpaca.cpp

Pablo

3/30/24, 9:22 AM

ok thanks i'll try it

Pablo

3/30/24, 10:07 AM

@pLumo can you send me the link for ggml-alpaca-7b-q4.bin please, i can't find it

pLumo

3/30/24, 11:38 AM

check https://github.com/antimatter15/alpaca.cpp/tree/test

chovy

4/23/24, 7:01 AM

it looks like changes were rolled back upstream to llama.cpp from alpaca

Score:1

Ubuntu

Sergiusz Golec

4/30/24, 2:11 PM

Save LLama.cpp files locally. Open terminal in a folder where you want the app.
```
git clone https://github.com/ggerganov/llama.cpp
```
```
cd llama.cpp
```
```
make
```
Download model, and place it into the 'models' (sub)folder. For example: https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml/resolve/main/ggml-alpaca-7b-q4.bin

Notes: Better model = better results.
Yet, there are also restrictions. For example, 65B model 'alpaca-lora-65B.ggml.q5_1.bin' (5bit) = 49GB space; 51GB RAM Required.
Hopefully in the future we'll find even better ones. Also, there are different files (requirements) for models that will use only CPU or also GPU (and from which brand - AMD, NVIDIA).

To make the best use of your hardware - check available models.
'ggml-alpaca-7b-q4.bin' 7B model works without any need for the extra Graphic Card. It's light to start with.

Update path + Run (question) prompts from terminal

/Documents/Llama/llama.cpp$

make -j && ./main -m ./models/ggml-alpaca-7b-q4.bin -p "What is the best gift for my wife?" -n 512

Result:

Source: https://github.com/ggerganov/llama.cpp

It would be great to:
1. check for a better (Web)GUI (on top of the terminal).
2. Add persona, like

https://www.youtube.com/watch?v=nVC9D9fRyNU from https://discord.com/channels/1018992679893340160/1094185166060138547/threads/1094187855854719007

P.S. The easiest AI local installation is to download 'one-click-installer' from https://github.com/oobabooga/one-click-installers (and follow prompt messages).

For Ubuntu \ Terminal:

$ chmod +x start_linux.sh
$ ./start_linux.sh

Yet, now it's not a perfect world. My failed attempts included:

OobaBooga failed for my laptop hardware (no GPU found). Bug - reported. And it looks like the model I've selected could not work without NVIDIA graphic card
Dalai lama failed due to folder rights restrictions, and a few items versions compatibility issues. So I skipped it, even if it looked promising. https://github.com/cocktailpeanut/dalai

+ 2

Sergiusz Golec

4/30/24, 2:22 PM

In short, result are biased from the: model (for example 4GB Wikipedia.zip vs 120GB wiki.zip) and the software on top of it (like LLama.cpp). I'd like to have it without too many restrictions. For example I've tested Bing, ChatGPT, LLama, ... and some answers are considered to be impolite or not legal (in that region).

David

4/30/24, 2:57 PM

This reads like a list of suggests not a clear answer.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Install llama.cpp locally?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.