Using Local AI Offline

This page = setup (while there is internet) + cheat sheet for an emergency. Example software: Ollama (free, Windows/macOS/Linux). Alternatives with a graphical interface: LM Studio, GPT4All.

Basic principle

Local models run completely without internet — but only if the program and the model files have already been downloaded. Set it up and test it now, not during a blackout!

Setting up Ollama (once, requires internet)

1. Download the installer from ollama.com/download and install it. Ollama then runs in the background (icon in the notification area) and is available on the PC at http://localhost:11434.
2. Open Command Prompt/PowerShell and download a model: ollama pull <model> (examples below).
3. Test: ollama run <model> → ask a question → exit with /bye.
4. Offline test: disconnect Wi-Fi/the network cable and ask another question.
5. Where models are stored on Windows: %USERPROFILE%\.ollama\models. Short of space on C:? Set the environment variable OLLAMA_MODELS to another folder and restart Ollama. This folder can also be copied via USB stick to a PC without internet.

Which model? (guide values)

Rule of thumb from the Ollama documentation: at least 8 GB of RAM for 7B models, 16 GB for 13B, 32 GB for 33B ("B" = billion parameters). A graphics card with plenty of memory makes answers much faster, but is not required.

• Recommendation: one medium all-round model + one small fast one for battery operation.
• For questions in languages other than English, choose a multilingual model and test it beforehand.
• Model names and versions change constantly — current list at ollama.com/library.

Immediate check (while the internet is still available)

• [ ] At least one model available locally? (ollama list or LM Studio → My Models)
• [ ] A large/medium all-round model (knowledge/advice) + a small fast one (battery operation) available?
• [ ] Does the AI start without a network? Test: Wi-Fi off → ask a question
• [ ] Is this offline base accessible to the AI? (folder wissen\ usable as context)

Ollama — quick commands

ollama list # show installed models ollama pull <model> # download a model (requires internet) ollama run <model> # start a chat ollama ps # show models currently loaded ollama serve # start the server manually (if it isn't running in the background) /bye # end the chat

Sharing the AI with all devices on the local Wi-Fi

Router on (power!) = the LAN works without internet.

1. On the AI computer, set the environment variable OLLAMA_HOST to 0.0.0.0 (Windows: "Edit environment variables for your account"), then quit Ollama (icon in the notification area) and start it again.
2. Windows Firewall: allow incoming connections on port 11434 — only on your own, trusted network!
3. Note the computer's IP address: ipconfig → IPv4 address, e.g. 192.168.x.x → write it here: ……
4. Other devices: in a chat app or web interface, enter http://<IP>:11434 as the Ollama server. Install the web interface (e.g. Open WebUI) or app beforehand — that only works with internet.

Sensible use in an emergency

Well suited: - Explaining things: "How do I apply a pressure bandage?" (cross-check with this base!) - Translating, writing texts, calculating, planning ("Ration 6 L of water for 2 people over 4 days") - Improvising: "What can I build/cook from X and Y?" - Querying this knowledge base: copy the contents of an MD file into the prompt + ask your question

Caution (risk of hallucination!): - Medication doses, chemical mixtures, electrical installations → always cross-check with the files in wissen\ or the package leaflet - Local facts (addresses, frequencies, phone numbers) — the model doesn't know them reliably → that's what this base is for

Prompt template for emergency questions

You are a precise emergency adviser. Answer briefly, in steps, and state safety warnings first. If you are unsure, say so explicitly. Context: power cut, no internet, household in <COUNTRY>. Question: <QUESTION>

Saving power when using AI

• Small model (3–8B) instead of a large one: a fraction of the consumption, sufficient for everyday questions
• Collect questions, bundle them into one session, then put the laptop on standby
• Laptop instead of desktop (a desktop uses 5–10× more power)
• Rule of thumb: laptop AI session ~50–100 W → with a 1000 Wh power station ~10–15 h of use