KnightOS
Personal · single-user · localhostpersonal ai assistant
A self-hosted, voice-capable assistant for macOS that pairs a chat interface with an agentic layer able to drive the machine and pull live data.
- lines, solo
- ~3,900lines, solo
- REST / SSE endpoints
- 17REST / SSE endpoints
- routed OS actions
- 34routed OS actions
- intent patterns
- 60+intent patterns
Cloud assistants can hold a conversation but cannot touch your machine, and the ones that can are usually thin shells around a terminal. I wanted one process that could do both: talk properly and actually operate macOS, while staying usable when the network or the cloud provider fails. Everything runs on my own hardware.
- 01
Deterministic intent routing instead of LLM tool-calling
key callSystem commands never reach the model. A routing engine resolves natural language against 60+ prioritised regex patterns to 34 concrete actions first; only genuinely open-ended input falls through to an LLM. Asking a model whether to open an app costs a network round-trip and can pick the wrong action from a plausible-looking set. Neither is acceptable for something you say out loud and expect to happen. Routing is O(patterns), runs locally, and is testable without a model in the loop.
- 02
Hybrid cloud/local model routing with automatic fallback
Gemini 2.0 Flash handles the primary conversational path; a failed cloud call fails over to local Ollama rather than surfacing an error. Model selection is task-specialised: a coder model for code generation, a vision model for local image understanding, so quality does not get traded away for a single general endpoint. The assistant degrades rather than stops.
- 03
SSE for token streaming, not WebSockets
Once a prompt is submitted the traffic is one-directional, server to client. SSE is a plain HTTP response with built-in reconnection, so there is no second protocol to run, proxy, or keep alive. Tokens stream as they are produced.
- 04
Two-tier persistent user modelling
Hard facts are pulled synchronously by regex the moment they appear, cheap and instant. A second asynchronous LLM pass runs off the critical path to mine personality, preferences and mood. Both feed a system prompt composed fresh per conversation, so the profile keeps deepening without adding latency to any single reply.
- 05
AppleScript as the OS control surface
App launching, volume and brightness, media transport, Spotlight file search, Calendar, Mail and notifications all go through AppleScript, the sanctioned automation path on macOS, which keeps the permission model honest and the action layer readable.
- 06
Multimodal ingestion and a generation studio alongside the chat
PDF, DOCX, XLSX, CSV and images are ingested into the conversation. A generation studio covers text-to-image, text-to-video and neural TTS. A separate news-intelligence dashboard aggregates five RSS feeds and plots geo-tagged articles on an interactive world map.
- Node.js
- Express 5
- React 19
- SSE streaming
- Gemini API
- Ollama
- AppleScript
- Web Speech API
Runs as a daily driver on one machine. It is deliberately a single-user localhost tool. There is no multi-tenancy, no hosted deployment and no auth layer, because none of those were the point. What it demonstrates is the routing, fallback and user-modelling architecture holding up under real daily use.





