Run the AI on Your Own Computer, and the Book Stays There.
Shiro’s AI can run entirely on your computer, with a bundled llama.cpp or through Ollama or LM Studio. Requests stay on 127.0.0.1, so you can write in a cabin with the Wi-Fi off. The catch is hardware: the catalogue runs from 4 GB of memory to 64 GB, and the small models are weaker than the big ones, and much weaker than the best hosted ones.
Three Ways to Run One
Shiro’s own. The desktop app ships with llama.cpp. Open You, then Shiro AI, then Local models, and it shows your chip and memory and recommends one model for writing and one for quick tasks. Downloads come from Hugging Face, can be paused and resumed, and are checked against a checksum when they finish. Nothing downloads or starts until you ask.
Ollama or LM Studio. If you already run either, Shiro picks up their models while they’re open.
Your own file. Paste a model name or link from Hugging Face, or point to a file on your disk.
What Stays on the Machine
A local request goes to 127.0.0.1 and comes back. “What Shiro sent” says so in as many words: “Stayed on this computer”. The app’s test suite includes a browser test that fails if the desktop client makes a request off localhost while browsing home, the lenses and settings. A local project with a local model never needs the network, and you can read the source if you’d rather not take our word for it.
Two things can undo that, both your choice. Models are set per task, so a key on one task sends that task’s text to that provider. And a cloud project lives on our server whichever model you use. The download is a request to Hugging Face, with nothing of yours in it.
What Your Hardware Buys
The memory is what the model needs at its default size, and the download is what it takes on your disk.
| Memory | Model | Download | What it’s for |
|---|---|---|---|
| 4 GB | Gemma 4 E2B | 2.6 GB | Renames, titles, tags, summaries |
| 8 GB | Gemma 4 E4B | 4.3 GB | The above, plus quick chat |
| 24 GB | Gemma 4 26B | 13.6 GB | Drafting, rewriting and chat |
| 32 GB | Gemma 4 31B | 16.8 GB | Long scenes and revision |
| 32 GB | Qwen 3.6 35B | 19 GB | Long context and structure |
| 64 GB | Mistral Small 4 | 68 GB | Best in the catalogue for French |
Take the first two rows at their word. A 4 GB model will name your conversations and summarise a chapter. It won’t write a scene you’d show anyone. The 26B is the default for writing, and it’s a file more than twenty thousand times the size of the novel it’s helping with.
Speed depends on your machine and we won’t promise a number. Try one short scene before you plan a week around it.
What You Give Up
Models of 32 GB and up work as agents and can read further into the book with tools. Below that, a model gets a fixed pack of context: the project card, your notes for it, up to 3 World entries and the text around your caret. No local model runs Max mode today, and images and video aren’t generated locally at all.
If that’s too little for a given job, you can hand that one job to a hosted model or your own key and keep the rest here. Or leave the AI off, which costs nothing and has excellent privacy.
The Questions People Ask
How much memory do I need to run AI for writing?
About 24 GB for a model we’d trust with drafting and rewriting. At 4 to 8 GB you get titles, tags, summaries and quick chat, and prose you probably won’t keep. Shiro looks at your computer and recommends what fits.
Can I use the models I already have in Ollama or LM Studio?
Yes. Shiro can use them while the app is open. You can also add a model from Hugging Face or a file on your disk.
Does it work offline?
Yes, once the model is on your computer. The download is the one step that needs a connection, and it only brings a file down.
Is a local model as good as a hosted one?
Usually not, and small ones aren’t close. Local models also can’t run Max mode today. What you get in exchange is that nothing leaves the machine, which for some books is the whole point.
Does local AI cost anything?
Not from Shiro, and there’s no per-word charge from anybody. You pay in disk space, memory and patience.