Errors
- Actual figure not found for this caption (check if figures have appropriate styles): Create Python environment
- Code listing found, but no header specified. Listing cannot be added:
ollama run gemma4:12b
Welcome to the era of AI-assisted coding. If you're a developer who loves the speed of thought but hates the friction of syntax, you've likely flirted with AI coding assistants. But today, we're moving beyond the “chat-box-in-the-browser” model.
I recently wrote an article about AI-assisted development with Gemini, https://www.codemag.com/Article/266021/AI-Assisted-Development-with-Gemini. While I used Gemini, I generically talked about Claude and Codex as side-by-side examples but I chose to use Gemini because you can get started with it for free. Of course, like most things, the more you pay, the better it gets.
A long time ago, we used to write code using assembly language. The people who wrote such code were brilliant. The programs they wrote did simple things, but getting a computer to do what it did, and explaining that to the computer via a language such as assembly, was hard.
Then along came languages like Fortran and Pascal. Gurus predicted the death of programmers. I mean, why would you ever bother to write complex assembler code when you could just write Fortran? The leaky abstraction to assembler remained, but the world moved to languages such as Fortran, Pascal, and then C, C++. We still used arcane editors like vi, and some people still do. I don't know why, but it takes a special mad man to code C++ in vi. In fact, it takes a special mad man to write C++ anyway. You can imagine how impressed I was when I saw Borland C++.
From the DOS prompt, I could launch a program, and it had menus. No need to remember complex commands, I could just navigate complex actions via menus. You know what was truly amazing? I could scroll up, and down in my long code files. The gurus once again predicted the death of the programmer. The leaky abstraction to assembler still remained, as did the leaky abstraction to writing C++ in vi.
Then came GUI-based operating systems, high-resolution screens, and IntelliSense. No need to remember the exact method, or even documentation was available as tooltips. You get my drift, right? The gurus predicted the death of the programmer once again. The programmer is still around.
This time it feels different. AI models allow me to describe what I want in natural language. In fact, I regularly do complex work that would otherwise involve a team of developers and months of work. I achieve that with a band of AI agents working on my machine in a matter of hours. Not only am I faster, but I am also producing better quality work. Once again, the gurus are predicting the demise of the programmer. Once again, the gurus are wrong.
Programmers won't be obsolete; we are going to build taller skyscrapers with better tools.
Local Models
Most of my AI-driven workflows rely on a model hosted by a third-party company like Google, Anthropic, or OpenAI. Over the past couple of years, these models have become astonishingly good. We have moved from “wow that's an impressive vector db search,” to “wow, AI is so good, it is practically reading my mind.” But a lot of such work relying on externally hosted models has some really huge drawbacks.
The first is privacy. Using cloud-hosted models means your data leaves your machine. This may not be a big deal if you are writing the next small utility, but this is a huge problem if you are using AI to sift through data that has national security concerns or commercial competitive edge. Frequently, local laws simply prevent data from going outside your country's boundary. How much do you trust OpenAI, Anthropic, or Google with such data?
Second is speed. Sure, the internet these days is pretty fast, but when I'm running a multi-hour multi-turn workflow, I have to imagine, would this be faster if I was sitting inside Anthropic's or OpenAI's or Google's data centers? I can literally see tokens incrementing, requests being sent, delays, me waiting for response and yes, sometimes I do run into 529s because I maxed out the APIs or my quotas.
Third is cost. Even if I just say, “Thanks Claude,” I end up paying for the tokens to do that. This seems unfair, but it is what it is. And let's be honest, while there is no doubt that AI has made me more productive, there are times when I just have AI do something simple that I could have done manually, because either I'm lazy or AI is just better at it. For instance, if I'm working on a complex set of stacked PRs, and the develop branch has moved along, it is just far easier for me to say, “hey AI, rebase all these PRs from develop, merge and fix all the problems, run unit tests, and push to git and monitor my CI checks.” Could I have done this manually? Sure. And it would have saved me some bucks too. It's a bit like getting a car for the first time: you stop walking as much, and eventually the extra convenience catches up with you. AI can end up costing a lot. Just ask all the companies that have put caps on AI budgets for their developers. I guess their developers are getting too fat.
And fourth is the issue of customization. I'm frequently deep researching a problem across documents, code, conversations, emails, internet knowledge, and more, or writing loads of code in stacked PRs. Doing all this work means I am going deeper into a problem area, and that means I am generating a larger and larger context as I go. Think of the context window as the model's active, working memory (RAM) during a single prompt-and-response cycle. If a model has a short context window, it suffers from “short-term memory loss”—as you feed it new code, it completely forgets the files you showed it five minutes ago. When I'm using cloud-based models, I have limited ability to tweak this context window. All I can do is pick a different model. And then I have to work with the limits of that model.
Imagine if we flipped the problem around, if I had a single model to choose from, and I could pick how many tokens I wanted to consume in one go; now I can super optimize my interaction with the model in a way that cloud-based models won't allow me to. I can customize my consumption of context, up or down, with each request.
I can get around all these problems if I use local models. That means no privacy issues. My queries are on my computer so no issues of speed. I can download the model and use it forever, so no additional cost besides initial hardware expense. And I can customize my context window to the limit of the model, based on the hardware I am on. So, for instance, if I have a high-end Mac with 128GB, I can consume 256K of token context window from Gemma 4 12B, but if I have only 16GB of unified memory, I can choose to consume less.
As you know, the AI field is growing by leaps and bounds. I wake up every day and there is a new technological revolution. A few months ago, I wrote this article (https://www.codemag.com/Article/2503031/Offline-AI-Image-Generation) where I used some models from Hugging Face and generated some impressive images. Back then, I checked out if there were general purpose models that could generate images, documents, transcribe my audio commands, do analysis on a multitude of inputs and help me write code. There were none. There were specialized models doing specialized things, and in retrospect they were impressive for their time of six months ago, but sometimes the images they generated were downright weird or funny.
I'm glad to say, things have changed. At the time of writing this article, we have capable, off-the-shelf hardware in the form of M4 or M5 Macs or Macbook Pros, and Microsoft Surface Ultra with Nvidia RTX has also started entering this field (likely more vendors coming soon). And we have models such as Qwen 3.5-9B or Qwen 3.5-14B or Gemma 4 12B that have impressive agentic coding capabilities, along with other tasks they can handle.
Also, both models are available for you to use using the Apache 2.0 license. If you have capable hardware, you can start using them right away.
In this article, I am going to write a complete, production-ready article showcasing exactly how to build an application entirely offline. I will hook up Gemma 4 12B inside Visual Studio Code using Python, set up a local inference server, and step-by-step build a local developer dashboard—complete with automated code review, markdown documentation generation, and structural verification tools.
So, grab a coffee, start terminal, and let's get started.
What Is Gemma 4 12B?
Gemma 4 12B (hereafter, Gemma) is the “Goldilocks” of the AI world. It is a 12-billion parameter model from Google DeepMind that is small enough to run on a standard laptop with 16GB of RAM, yet smart enough to reason through complex Python logic and system architecture. It slots comfortably between ultra-light edge models and massive datacenter architectures. With 12 billion parameters, it is exceptionally dense, highly efficient, and tuned specifically to punch well above its weight class in logic, reasoning, and coding task compliance.
Google explicitly highlights coding and agentic workflows as core targeted use cases in the official Gemma 4 model card. Because it is optimized for developer workflows and designed to run locally, it fits right into the “vibe coding” ethos—where you describe your goals in high-level natural language, let the AI handle the heavy structural lifting, and maintain a fast, iterative flow.
Gemma is built for agents. Google explicitly optimized Gemma for agentic tool use and multi-step reasoning. It integrates smoothly with local developer agent harnesses like OpenCode (https://opencode.ai/) or Claude Code (https://github.com/anthropics/claude-code), allowing it to execute code, test it, and iterate on it autonomously.
Gemma has a massive 256K context window. Vibe coding relies heavily on feeding entire codebases, file structures, and error logs into the model. With a 256K token context, you don't have to meticulously trim your files before handing them over to the model.
Gemma has low-latency and executes locally. It features a novel, encoder-free “unified architecture” where images and audio pipe directly into the decoder, letting Gemma operate with a much lighter memory footprint. You can run a quantized version (like a 4-bit GGUF via Ollama or llama.cpp) right on a consumer laptop with 16GB of RAM, giving you fast, offline generation without API costs or rate limits.
In my experiments with Gemma, I've found its capabilities to be quite impressive. I dropped a UI mockup or a screenshot of a layout bug directly into my agent pipeline alongside a text prompt and asked Gemma to write the code to match or fix it.
Setting Up Your Workspace
You can use Gemma via llama.cpp or Ollama (https://ollama.com). The first thing I am going to do is download and install Ollama by running the command below.
curl -fsSL https://ollama.com/install.sh | sh
Ollama is an open-source framework that allows you to run large language models (LLMs)—like Llama 3 or Mistral—directly on your own computer. It acts as a lightweight, user-friendly wrapper around local AI engines, letting you download and use models offline without relying on cloud servers.
Ollama runs as a background application on your machine (available for Windows, Mac, and Linux). Using a simple command line interface (CLI) or third-party visual interfaces, you can type a command like ollama run llama3, and the software will handle the download, setup, and processing right on your local hardware. You can see the models available for use with Ollama here https://ollama.com/search. I wrote about Ollama in a previous CODE Magazine article (https://www.codemag.com/Article/2411031/AI-with-No-Internet-Connection) and Wei-Meng Lee has a great Ollama article here: https://www.codemag.com/Article/264031/Running-Large-Language-Models-Locally-Using-Ollama.
Once Ollama is installed on your machine, run the command on terminal as shown below.
Running this command will pull the model weights to your hard drive, as shown in Figure 1.
Set Up Python in VS Code
While you can use any coding surface or programming language, I will use VS Code and Python. Install VS Code from https://code.visualstudio.com and install the Python extensions shown in Figure 2.
Also, I have Python 3.12.3 installed on my machine.
Now create a directory on your machine. I called mine VibeStudio and I opened that folder in VS Code.
In that folder, I create a requirements.txt and add the following lines in it.
ollama
rich
Next in VS Code, press CMD-P and choose to create a Python environment as shown in Figure 3.
Also go ahead and create an index.py file at the same location as requirements.txt.
Now we are ready to start building our VibeStudio.
Building the Application
This article will show you how to write an application in a few steps that helps you with vibe coding. I am going to keep things simple. I'll write the application that accepts an input, and based on that input, it generates code. You can make this as fancy as you like. For instance, nothing stops you from editing code that goes sentient, or most likely, just goes haywire like a bad version of OpenClaw.
Setup and Configuration
First, we need to import our libraries and define the “Vibe” we want the AI to have. As Listing 1 shows, we set the temperature low (0.2) because code requires precision, not creative poetry. As you can see, this code is the foundation and setup layer for an interactive, AI-powered development tool. It doesn't process files or chat with you just yet; instead, it imports the necessary tools, declares a blueprint (class) for the application, and configures exactly how the AI should behave.
Listing 1: Setting temperature
import os
import ollama
from rich.console import Console
from rich.markdown import Markdown
# The Console object helps us print
# beautiful text to the terminal
console = Console()
class VibeStudio:
def __init__(self):
self.model = "gemma4:12b"
# Tell Gemma 4 to be accurate and literal
self.options = {
"temperature": 0.2,
"top_p": 0.9,
}
At the very top we have a few imports.
import os: Grabs Python's built-in tool for talking to your computer's operating system (used later to read files, look through folders, etc.).
import ollama: Imports the library that connects Python to Ollama—the engine running the Gemma 4 AI model locally on your hard drive.
from rich.console import Console & from rich.markdown import Markdown: Pulls in parts of a popular library called rich. Instead of printing boring, plain white text to your terminal, rich allows the program to display styled text, colors, tables, and cleanly formatted Markdown text (like headers, bullet points, and code blocks).
Then, we create an object called console. Think of it as an upgraded version of Python's standard print() function. Whenever the code wants to show you a colorful message, a loading spinner, or a nicely formatted block of code, it will route it through this console object.
And finally, we create a class called VibeStudio, which is essentially a blueprint or container. It groups together all the data and actions (functions) that belong to our AI assistant tool, so everything stays clean, organized, and isolated.
Then in the constructor, we tell the program exactly which local AI model it should wake up and talk to when you ask a question. We give it some options, specifically, temperature: 0.2: This controls the AI's “creativity.” A temperature of 1.0 is highly creative (great for writing stories). Dropping it all the way down to 0.2 forces the AI to be highly focused, literal, and deterministic—exactly what you want for coding, where a “creative” guess usually breaks the syntax. And top_p: 0.9: works alongside the temperature to discard highly unlikely words, ensuring that Gemma 4 sticks to the most reliable, standard coding patterns.
Reading Your Project
Vibe coding requires the AI to “see” your work. Using the code in Listing 2, the AI crawls your folders and ignores the “trash” (like __pycache__ or .git folders). This code acts like a digital vacuum cleaner for your project. Its sole job is to crawl through all the folders in your current workspace, skip over the folders full of technical junk, find every Python file, open them up, read everything written inside them, and bundle it all up into a neat list so the AI can look at your entire project at once.
Listing 2: Reading files
def get_workspace_files(self):
files_to_read = []
for root, dirs, files in os.walk("."):
# Filter out junk
if any(x in root for x in [".git", "venv", "__pycache__"]):
continue
for name in files:
if name.endswith(".py"):
path = os.path.join(root, name)
with open(path, "r") as f:
files_to_read.append({"name": name, "content": f.read()})
return files_to_read
Once Python has finished exploring every single corner of your project directory, it returns the completed basket of the relevant parts of your codebase minus the trash supporting files it doesn't care about. Now, your main program has a neat, searchable map containing the live contents of your entire codebase, ready to hand over to Gemma.
The “Vibe” Prompt
Listing 3 shows the core logic of the application. We take your natural language request (the “vibe”) and give it to Gemma along with your code. This function is the “brain” of the application. It's the moment where your project files, your specific request, and the AI model all meet to produce an answer. It acts like a high-level manager: it gathers all the relevant documents, briefs the expert (Gemma), and asks Gemma to solve your problem.
Listing 3: The vibe prompt
def process_vibe(self, user_input):
code_context = self.get_workspace_files()
# We build a 'System Prompt' to tell Gemma 4 its job
system_instructions = "You are a Vibe Coding assistant.
Your goal is to take high-level user requests and
provide perfect Python code improvements
based on the current workspace."
# We bundle the code files into a single string
context_string = "\n".join(
[f"FILE {f['name']}:\n{f['content']}" for f in code_context]
)
full_prompt = f"CONTEXT:\n
{context_string}\n\nUSER REQUEST: {user_input}"
# Call Gemma 4 locally
response = ollama.generate(
model=self.model,
system=system_instructions,
prompt=full_prompt,
options=self.options
)
return response['response']
First it gets the workspace files using the function in Listing 2. Then it sets the “Expert persona” a bit like this: system_instructions = "You are a Vibe Coding assistant…".
This is the system prompt. It's like giving the AI a job description before it starts working. By telling it exactly who it is (a “Vibe Coding assistant”) and what its goal is (“provide perfect Python code improvements”), you ensure it stays professional and focused on code rather than trying to write poetry or tell jokes.
Then we give it the context string. Think of this as the briefing document. Here, the code takes that “basket” of files (created in Listing 2) and turns it into one long, continuous text string. It adds labels (like FILE main.py:) before each file's content so the AI knows exactly where one file ends and the next begins.
And then we build a full prompt, which will be sent to AI. This line "full_prompt = …" builds the final “message” that will be sent to the AI. It puts the entire project's code at the top (CONTEXT) and your specific instruction at the bottom (USER REQUEST). Because Gemma 4 12B has a massive 256,000-token window, it can “read” this entire combined message—even if your project is quite large—without getting confused or losing the thread.
When we have the full prompt ready, we talk to the oracle, in our case Gemma, using the ollama.generate method. And we hand back the answer.
In other words, this function formats your entire project into a readable document, attaches your question to the bottom, and hits the “Send” button to the local AI engine.
Running the Application
Great! Your application is done. Now let's run it. From VS Code, press F5, and pick “Python debugger” to run your vibe.py file. You should see an output like Figure 4.
Let's give your vibe coder app an interesting problem to solve. Interviewers love asking LeetCode problems. I used Google and found an interesting LeetCode problem.
Given a signed 32-bit integer x, return x with its digits reversed. If reversing x causes the value to go outside the signed 32-bit integer range [-231, 231 - 1], then return 0. Assume the environment does not allow you to store 64-bit integers (signed or unsigned).
Before you give it to your vibe coder, give it some thought. How long will it take for you to solve this? Who cares! I'll just give it to Mr. Vibey.
As soon as I paste the LeetCode problem text in terminal, I can see that my local GPU started working hard. This can be seen in Figure 5.
I can also hear the fan spin up, even though I have a relatively powerful machine (M4 Max with 128GB of unified memory). So, let's just say, this isn't replacing OpenAI or Anthropic just yet. But, in about 15 seconds, it gave me the output you can see in Listing 4.
Listing 4: Vibe coded output
def reverse_integer(x: int) -> int:
"""
Reverses the digits of a 32-bit signed integer.
If the reversed value exceeds the 32-bit signed integer range, returns 0.
"""
# Define the limits for a 32-bit signed integer
MIN_INT = -(2**31)
MAX_INT = 2**31 - 1
res = 0
# Work with the absolute value to simplify modulo/division logic
# and handle the sign at the end.
num = abs(x)
sign = -1 if x < 0 else 1
while num > 0:
# Extract the last digit
digit = num % 10
# Append it to our result
res = (res * 10) + digit
# Remove the last digit from the original number
num //= 10
# Apply the original sign
res *= sign
# Check if the result is within the valid 32-bit signed integer range
if res < MIN_INT or res > MAX_INT:
return 0
return res
# Example usage:
# print(reverse_integer(123)) # Output: 321
# print(reverse_integer(-123)) # Output: -321
# print(reverse_integer(120)) # Output: 21
# print(reverse_integer(1534236465)) # Output: 0 (overflows)
Take a deep breath. You just communicated your intent in plain English, and the locally running model generated code for you? Why bother learning Python anymore? In fact, why even bother with an IDE, or Fortran or Pascal or assembler? You get my drift, right?
Gemma is also incredibly good at parsing audio text and pictures. I encourage you to take this application further and do some DSL-specific stuff, like point it to a database, and have it query the database via natural language. See it generate SQL commands underneath the hood. How about log analytics, or Splunk queries?
Or my favorite thing to dunk on, git.
How do I revert the last two git commits that I have already pushed to develop?
I am really interested in learning what Gemma says here. Gemma thought for about 15 seconds and gave me the output shown in Figure 6.
I am blown away. Just one year ago, if you had told me something like this could run locally on my Mac, I would not have believed you.
Making Code Changes
I think I should share the full working code with you. Now, I could do things like write a readme.md or add a .gitignore, but hey, I have a shiny new toy, our vibe coder. Let me just use that.
I gave vibe coder a prompt:
Add a suitable readme.md to this project.
It generated a nice readme.md. You can see the full readme in my shared code at https://github.com/maliksahil/vibestudio/blob/main/`readme.md` so you be the judge about how good or bad it is.
Then, I gave it a prompt:
Suggest a suitable .gitignore for this project.
I thought you wouldn't believe me about how good this local vibe coder is, so I am including a screenshot of the vibe coder application's output in Figure 7.
Vibe coder generated a perfect .gitignore for me. I didn't have to explain that this was a Python project, or what are the best practices for Python, or what the heck a .gitignore file is. In fact, it is coaching me on what to do.
I think I'm ready to share this with you. So, I pushed it to a public repo here: https://github.com/maliksahil/vibestudio. Feel free to fork, enhance, and let me know what you do with it.
Summary
Vibe coding isn't about writing less code; it's about raising your abstraction level to act as an architect rather than an assembly line programmer. By combining the local execution safety of VS Code with the reasoning power of Gemma 4 12B, you get a zero-cost, private, hyper-responsive software factory running right on your desk.
You should look at vibe coding not as a flashy buzzword, but as a fundamental shift in software cognitive load. In traditional programming, your brain acts as a compiler and syntax checker. You spend significant mental energy ensuring braces match, tracking imports, and looking up API signatures. Vibe coding shifts your responsibility up the stack. You become a systems director. You manage intent, edge cases, and architectural boundaries, while Gemma 4 12B manages the keystrokes.
There are many possibilities of enhancing this further.
You could create multi-agent swarms. Our VibeStudio app used a single instance of Gemma to read files and answer prompts. The immediate next level is agentic delegation. Instead of one AI doing everything, you use Gemma's strong tool-use capabilities to spin up a collaborative team on your machine. An architect agent, a developer agent, the QA tester agent, all working in tandem with each other, against each other, like a team, making the end product better.
You could go for true multimodal vibe coding. Because Gemma features a unified architecture that natively understands vision without needing heavy, separate image encoders, you aren't limited to text prompts. Imagine dropping a hand-drawn napkin sketch of a user interface, or a screenshot of a broken CSS layout, straight into your local workspace. You can vibe prompt: “Look at this layout bug image, check my styles.css file in the context window, and fix the alignment.” The model bridges the gap between visual intent and raw code implementation entirely offline.
You could integrate it with MCP servers, have it write or use custom skills, or even write MCP servers for you.
Create a VS Code extension that constantly looks at your typed characters, so as you write code, Gemma will generate unit tests, find edge cases, suggest improvements, and act as a predictive pair programmer.
The true promise of local vibe coding with models like Gemma 4 12B is the elimination of creative friction. Software creation becomes democratized down to the speed of your thoughts, protected entirely by the privacy boundaries of your own local hardware.
Exciting times ahead. Until next time, happy vibing.
