Open Source AI

Local AI means running a model on your own hardware — no API keys, no subscription, no data sent to a third party. The tradeoff: you need a capable machine, and the best local models are still a step behind the best cloud models.

When to go local

• Privacy: data stays on your machine — useful for sensitive codebases, client work, or regulated industries
• Cost: no per-token billing after the initial setup
• Offline: works without internet access
• Experimentation: try different models freely without usage limits

Ollama

The easiest way to run models locally. Ollama is a CLI tool that downloads and runs open source models with a single command:

# install Ollama, then:
ollama run llama3.2
ollama run mistral
ollama run qwen2.5-coder   # good for code

Models are downloaded automatically on first run. Ollama also exposes a local API compatible with the OpenAI API format, so many tools can use it as a drop-in backend.

Best for: getting started quickly; running models from the terminal.

LM Studio

A desktop app (Mac, Windows, Linux) for downloading and running local models via a GUI. Includes a chat interface and a local server you can point other tools at.

Best for: people who prefer a visual interface over the terminal; trying out models without writing any config.

Notable models

Model	Made by	Good at
Llama 3.x	Meta	General purpose; strong baseline
Mistral / Mixtral	Mistral AI	Fast; good instruction following
Qwen 2.5 Coder	Alibaba	Code generation and completion
Phi-3 / Phi-4	Microsoft	Strong performance at small size
Gemma 2	Google	Efficient; good for constrained hardware

Model capabilities improve rapidly. Check ollama.com/library for the current list and download counts (a rough proxy for quality and community trust).

Hardware requirements

Local models need RAM or VRAM proportional to their size. A rough guide:

7B model   → 8GB RAM minimum, 16GB comfortable
13B model  → 16GB RAM minimum
70B model  → 48GB+ RAM, or a high-end GPU

Smaller models (3B–7B) run on most modern laptops. Larger models need a desktop with a good GPU or Apple Silicon (M2/M3 Pro/Max/Ultra with unified memory).