Gemma2+Ollama:FreeLocalAICodeGenerationGuide

📋 Quick Stats

• Difficulty: Intermediate
• Time commitment: 30-60 minutes (depends heavily on your internet speed for model downloads)
• What you'll need:
  • A decent CPU (Intel i5/AMD Ryzen 5 or better)
  • At least 16GB RAM (32GB+ is definitely recommended if you plan on using bigger models like Gemma 2 27B, trust me on this)
  • A dedicated GPU with 8GB+ VRAM (NVIDIA with CUDA or AMD ROCm support will fly; Apple Silicon Macs get a pass with their unified memory)
  • Some basic command-line chops
  • Admin/sudo access for installation
• Tested on: macOS (Apple Silicon & Intel), Linux (x86_64, ARM64), Windows (WSL2 gives the best experience, but a native installer exists)

Getting Ollama onto Your Machine

Alright, first things first: you need Ollama. This is the bedrock that lets you fire up models like Gemma 2 on your local machine. It wraps up all the complex model execution details, including tapping into your hardware acceleration, so you can just talk to your AI via a simple CLI or API.

⚠️ Seriously, Check Your Specs: I've seen too many developers (and, fine, me once or twice) try to run these models on underpowered machines. It's a recipe for excruciatingly slow inference or, worse, a full system crash. For gemma2:9b, you'll want at least 16GB RAM. If you're eyeing gemma2:27b, prepare for 30-32GB. Don't skimp here unless you enjoy watching your machine crawl.

1. Install Ollama

Get the right Ollama application for your operating system. It provides the runtime and the command-line tool you’ll use.

macOS (Apple Silicon & Intel)

1. Download: Head over to the official Ollama website and grab the macOS application.

   # This just opens the download page, you still need to click download.
   open https://ollama.com/download/macos
   

2. Install: Open the downloaded .dmg file. You know the drill: drag that Ollama app right into your Applications folder.
3. Run: Launch Ollama from Applications. It runs as a background service. You'll see a tiny llama icon chilling in your menu bar once it’s up.

Linux (x86_64, ARM64)

1. Install: Use their official curl script. This thing handles everything: downloading Ollama, putting it in your PATH, and setting up a systemd service. Pretty neat.

   # For Linux x86_64
   curl -fsSL https://ollama.com/install.sh | sh
   

   # For Linux ARM64 (think Raspberry Pi 4/5, Jetson Nano boards)
   curl -fsSL https://ollama.com/install.sh | sh
   

   ⚠️ Permission Headaches?: If you hit permission errors, you probably need sudo. Try sudo -E sh -c 'curl -fsSL https://ollama.com/install.sh | sh'. Just be mindful when you run stuff with sudo.

Windows (Native Installer)

1. Download: Get the Windows installer from the official Ollama website.

   # Again, opens the page, you do the clicking.
   start https://ollama.com/download/windows
   

2. Install: Run the .exe file you just downloaded. Follow the prompts. Ollama should kick off its background service automatically.

Verify the Installation: Open a new terminal or command prompt. Let's see if Ollama is playing nice and is in your system's PATH.

ollama --version

You should see something like this:

ollama version is 0.1.X  # The exact version number will obviously be different.

If it blew up:

• macOS/Windows: Did you actually launch the Ollama application? Make sure it’s running in the background. Sometimes just restarting your terminal fixes it.
• Linux: Classic Linux troubleshooting: check your PATH (echo $PATH). Is /usr/local/bin in there? If not, restart your shell or add it manually. Also, verify the service with systemctl status ollama. Might be a simple service not running.

Picking Your Gemma Model for Code Generation

Choosing the right Gemma model is a balancing act, really. You want decent code generation without turning your machine into a snail. The original video might have thrown around "Gemma 4," but let's be realistic: we're living in April 2026, and Google's Gemma 2 models are the ones we can actually get our hands on today. They're a solid step up from the first Gemma series in terms of reasoning and code abilities. For most setups, I recommend gemma2:9b. It hits a sweet spot between being smart enough to be useful and not hogging all your resources. If you've got a beast of a machine, by all means, go for gemma2:27b – it'll give you even better performance.

⚠️ The Dreaded OOM Error: Don't underestimate the VRAM/RAM requirements. I’ve wasted hours debugging "mystery" system slowdowns only to realize I was trying to cram a 27B model into 16GB of RAM. The gemma2:9b model typically needs about 10-12GB of VRAM or system RAM (if your GPU isn't cutting it or you're on integrated graphics). The gemma2:27b? That's a whopping 30-32GB. Pick a model that respects your hardware, or you'll be staring at "out of memory" errors or waiting an eternity for results.

2. Pull the Gemma 2 Model

Now that Ollama's set up, let's grab the gemma2:9b model. This command downloads the quantized model weights and all the config files Ollama needs to run Gemma 2 locally.

ollama pull gemma2:9b

You'll see a progress bar like this (don't worry, I've truncated the output, it's a long download):

pulling manifest
pulling 000000000000... 100% | || 
... (many more lines of pulling) ...
pulling 000000000000... 100% | || 

Related Reading

• The Agentic Shift: How AI-Driven Development is Replacing Junior Tasks in 2026
• Claude Code 2.0: Practical Guide for Developers
• Build a Full Stack Gen AI Web App: React, Node, JWT, Gemini