1panel面板使用GPU启动ollama
使用1panel安装ollama,启动模型默认在CPU上跑，模型稍大，速度就很慢，以下是解决办法：
1 安装NVIDIA容器镜像
为了在docker容器中使用GPU加速，我们需要安装NVIDIA的容器镜像。
Centos系统安装
curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo yum-config-manager --enable nvidia-container-toolkit-experimental
sudo yum install -y nvidia-container-toolkit
Ubuntu系统安装
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
2.配置Docker容器使用NVIDIA
安装完容器镜像后，需要配置Docker以使用NVIDIA，并重启Docker服务：
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
3.安装ollama并配置GPU加速
在1Panel应用市场选择ollama进行默认安装。安装成功后，我们需要修改docker-compose文件以启用GPU加速。
在【已安装】中找到ollama，点击【参数】。
点击【编辑】->勾选【高级设置】->勾选【编辑compose文件】。
在docker-compose文件deploy下面添加reservations相关gpu配置：
networks:
1panel-network:
external: true
services:
ollama:
container_name: ${CONTAINER_NAME}
deploy:
resources:
limits:
cpus: ${CPUS}
memory: ${MEMORY_LIMIT}
reservations:
devices:
- capabilities:
- gpu
count: 1
driver: nvidia
image: ollama/ollama:0.13.0
labels:
createdBy: Apps
networks:
- 1panel-network
ports:
- ${HOST_IP}:${PANEL_APP_PORT_HTTP}:11434
restart: unless-stopped
tty: true
volumes:
- ./data:/root/.ollama
点击确认并查看日志，确认GPU加速是否生效。
第二种方法，手工方式
1、暂停ollama容器
2、删除ollama镜像
例如：
root@rsz-virtual-machine:/home/rsz# docker rmi ollama/ollama:0.5.4
Error response from daemon: conflict: unable to remove repository reference "ollama/ollama:0.5.4" (must force) - container 69e7be2e3973 is using its referenced image 325786c469da
root@rsz-virtual-machine:/home/rsz# docker stop 69e7be2e3973
69e7be2e3973
root@rsz-virtual-machine:/home/rsz# docker rm 69e7be2e3973
69e7be2e3973
root@rsz-virtual-machine:/home/rsz# docker rmi -f ollama/ollama:0.5.4
Untagged: ollama/ollama:0.5.4
Untagged: ollama/ollama@sha256:18bfb1d605604fd53dcad20d0556df4c781e560ebebcd923454d627c994a0e37
Deleted: sha256:325786c469dadaa0abfad3acdef17f684e7f159ca27ecd43b1851c8aec97ffc4
Deleted: sha256:9c73b4cbcbdcadcf670c1fc7c094b6508375c4f0795d99033e09335349666614
Deleted: sha256:b530b2475eddedc496ffb304e793e6e239d0abdfa873326bb77f4864684819a1
Deleted: sha256:dcfc835990dac945acfc4c00698cce52f8c3bee64a9ec21c7c26148c6b0ceed9
Deleted: sha256:2573e0d8158209ed54ab25c87bcdcb00bd3d2539246960a3d592a1c599d70465
root@rsz-virtual-machine:/home/rsz#
3、进入安装路径
4、查看路径
5、已GPU运行ollama得docker命令
其中/opt/1panel/apps/ollama为自己得安装路径
docker run --gpus all -d -v /opt/1panel/apps/ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama