원래는 arm server에서 ollama돌리려했지만…

firmware update하다가 부팅실패로 보류하고,

지난번에 까먹은 openwebui 사용법을 안 적어 둔것 같아서 살짝 다시 적어봅니다.

그냥 Gemini에게 알려달라면 다 알려주는 내용입니다.

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

그리고 브라우저에서 포트 3000으로 접근해서 사용하면 끝

http://localhost:3000

일단 귀찮으니, 그냥 복붙..

To run Ollama with GPU in Docker as a non-root user and access a GUI (Open WebUI) on Ubuntu or remotely, install NVIDIA Drivers and NVIDIA Container Toolkit. Add your user to the docker group, then run the Ollama container with --gpus all. The GUI is accessed via browser on http://localhost:3000. 

Step 1: Prerequisites & User Privileges

1. Install NVIDIA Drivers: Ensure your Ubuntu host has Nvidia drivers installed (nvidia-smi).
2. Install Docker & Toolkit: Install Docker and the NVIDIA Container Toolkit to allow Docker to use the GPU.
3. Setup User Privileges: Add your user to the docker group to run commands without sudo:bashsudo usermod -aG docker $USER # Log out and log back in for changes to take effect  

Step 2: Run Ollama in Docker (GPU) 

Run the Ollama container. Using a volume (-v) ensures models are saved when the container stops. 

bash

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama --restart always ollama/ollama:latest

Step 3: Run Ollama GUI (Open WebUI) 

Run the Open WebUI container to provide a ChatGPT-like interface. 

bash

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Step 4: Access the GUI (Local or Remote) 

• On Ubuntu Desktop: Open a browser and go to http://localhost:3000.
• From Remote Windows/Ubuntu: Open a browser and go to http://<UBUNTU_IP_ADDRESS>:3000.
  • Note: Ensure port 3000 is open in the Ubuntu firewall (sudo ufw allow 3000). 

Step 5: Start a Model 

1. In the WebUI, click “Get Started” to create an admin account.
2. In the top-left dropdown, click “Select a model” and type a model name (e.g., llama3 or phi3) to pull it.
3. Alternatively, pull via terminal: docker exec -it ollama ollama pull llama3. 

Key Configuration Tips

• Persistent Storage: The volumes ollama:/root/.ollama and open-webui:/app/backend/data store your models and chat history.
• Restart Policies: Both containers use --restart always to start automatically upon system reboot.
• Resource Management: If you have limited VRAM, consider running smaller models (7b or 8b) to prevent errors