The video below shows how to download and run the Dolphin Llama 3 model.
Ollama Program.
First download the ollama program files from ollama.com.
Next, we need to pull the dolphin-llama3 model.
This process starts by opening two terminals, one for the server and the other for the program.
In the first terminal type ollama serve
This will make it so the base Ollama server program is running.
In the second terminal, type ollama run dolphin-llama3
Close both terminals and run the commands again to ensure the uncensored model is running.
You can type in a question that would typically be censored to see if it is working.
The program is now running from the primary hard drive and we will discuss how to transfer it to an external drive.
I will be transferring the program and model to a SanDisk 128 GB USB 3.0 flash drive
First right click the flash drive, select format and under file system, select NTFS. This will make it so we can transfer files larger than 4GB. If you are using a large hard drive you can skip this step as it will be NTFS by default. Note that if you do select the new format it will delete anything you previously had saved on the flash drive. So do this when first using a new flash drive.
Transfer the ollama files from C:\Ollama to the external hard drive or flash drive.
We also need to copy over the base ollama server program that interfaces the model.
To find where this is located you can type.
Get-Command ollama
In this case it was located here.
C:\Users\cody\AppData\Local\Programs\Ollama
We need to copy the files located within the ollama folder to the ollama folder on the external hard drive.
Now we can run the model from the external drive which could be the D:, E:, G:, H: or I: drive.
Run Dolphin Llama 3 From an External Drive.
Now we will run the program from the external drive.
Step 1:
To run the program from the external drive first open two PowerShell terminals.
In the first terminal, we will run the server. Start by changing the directory to the external drive
by typing cd h:\ and then pressing Enter.
cd h:\
To change the environmental variables and set the model path this command is used.
$env:OLLAMA_MODELS = “H:\ollama\models”
Then start the server with the serve command.
ollama\ollama.exe serve
Step 2:
In the second terminal, we will run the program.
Again change the directory to the external drive
cd h:\
Then type in the run command.
ollama\ollama.exe run dolphin-llama3
The first time this runs it might take a minute to initialize.
We can now type a question that would typically be censored to see if the model is working.
It gave an appropriate response so the model is not censored.
AnythingLLM Interface.
Step 1.
The first step is to run the server within the PowerShell the same way as before.
We will set the path and then enter the serve command.
Change directory
cd h:\
$env:OLLAMA_MODELS = “H:\ollama\models”
Then enter the serve command.
ollama\ollama.exe serve
Now go to AnythingLLM.com and download the program.
For the path use the external drive\anythingllm
It will take a few minutes for the program to install.
Before opening the program a .env file needs to be added within the anythingllm folder. Open a text document and save this code to the folder location as a .env file. Make sure the model path is correct and in this case, is set to the h drive.
OLLAMA_HOST=http://127.0.0.1:11434
LLM_PROVIDER=ollama
MODEL_BACKEND=dolphin-llama3
OLLAMA_MODEL_PATH=H:\Ollama\models
Now start the program.
Pick Ollama: Run LLMs locally on your own machine.
The Ollama model should be
dolphin-llama3:latest
Once the program opens you can type in a question that would typically be censored to make sure that it is working.
Open the program with just one click using a batch file.
Once installed you can run the program with one click if you add this one File 1: shown below to the external drive’s root folder. That means placing the file in the external drive’s main location, not in a folder.
File 1:
Make the file name: start.bat
@echo off
set DRIVE_LETTER=%~d0
set OLLAMA_MODELS=%DRIVE_LETTER%\ollama\models
echo Starting Ollama…
start “” %DRIVE_LETTER%\ollama\ollama.exe serve
:waitloop
rem Change the 11434 below to whatever port is actually used by ollama server
netstat -an | find “LISTENING” | find “:11434” >nul 2>&1
if errorlevel 1 (
timeout /t 1 /nobreak >nul
goto waitloop
)
echo Starting AnythingLLM…
start “” %DRIVE_LETTER%\anythingllm\AnythingLLM.exe
Once you place this file in the root folder just click start and it will open the server and AnythingLLM program. This makes it so you do not have to remember the run commands. This batch file code was provided by Alien Wise, from TAG Software in the Netherlands. Thank you very much for providing this code to help viewers more easily run the program. Also provided was the code below which makes it so when the USB drive is placed in the USB port, it automatically brings up a dialog, asking the user if they want to run Dolphin LLM.
File 2:
Make the file name: autorun.inf
shellexecute=start.bat
icon=customicon.ico
action=Start local Dolphin LLM
Also, you could make a custom icon. Adjust the autorun.inf file to match the icon name and location.
Note: There are other dolphin programs that can be downloaded from Ollama.com. There is a smaller TinyDolphin model that can be used when your computer or device has less than 8GB of RAM. The 8B parameter Dolphin Llama 3 model requires 5.9 GiB of RAM available. There is also a Dolphin 3 model which uses the newer Llama 3.1 model. The process to download and use the models is very similar.
Cody started the Global Science Network with the idea people should be focusing more time, energy, and resources on useful projects. He has a bachelor’s degree in aerospace engineering and a master’s degree in mechanical engineering. Cody has worked for the US federal government, a university, a large corporation, small businesses, and for himself. He has done human brain computer interface research and is currently working towards creating non-biological human consciousness.