Sign in Subscribe

Get up and running with your local AI

Get up and running with your own Local AI

In this series We will take a step by step process of how to create your own local AI server on your desktop or a headless network computer allowing you to experiment with LLMs, and generative models for free. There are a number of models that can be self hosted. I will be using Debian because that is what is on my computer and after some recent hardware changes its an opportunity to go over the process again.

Read next

Getting started with Docker

Getting started with Docker

Lets install Docker. this will allow you to use docker containers. In Debian, we will add the docker repository sudo apt-get update sudo apt-get install ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.

Getting started with Ollama

Getting started with Ollama

Now we will install Ollama. This is a tool that will allow you to interact with several LLM's. First we will need to make sure curl is installed sudo apt install curl Now we will install Ollama curl -fsSL https://ollama.com/install.sh | sh Now we can

Install CUDA Toolkit for your AI

Install CUDA Toolkit for your AI

First thing is to install NVIDIA CUDA Toolkit. but first lets just make sure the packages are up to date. sudo apt update && sudo apt upgrade If there were any updates it's a good idea to reboot to make sure there are no errors from updating