Docs: 9router + Hermes Agent local AI stack with Telegram (9router routes model via combo)

2026-08-11 17:50:23 +02:00
commit da4b918e68
7 changed files with 332 additions and 0 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -0,0 +1,12 @@
+# OS / editor junk
+.DS_Store
+*.swp
+*~
+.idea/
+.vscode/
+
+# never commit real secrets
+.env
+*.env
+*_APIKEY
+*_KEY
--- a/README.md
+++ b/README.md
@@ -0,0 +1,227 @@
+# 9router + Hermes Agent — Local AI Stack with Telegram
+
+Run a **self-hosted AI assistant** on a single Ubuntu server:
+
+```
+Telegram  ──►  Hermes Agent (gateway)
+                    │  (model = "free", base_url = 9router)
+                    ▼
+                 9router  (http://127.0.0.1:20128/v1)
+                    │  combo "free" (fallback routing)
+                    ├─► ollama/gpt-oss:120b
+                    └─► ollama/minimax-m3
+```
+
+**Key design point:** Hermes does NOT pick the model. It sends every request to
+9router with `model: "free"`, and **9router itself chooses** which free model to
+use (via a *combo* with `fallback` strategy). This keeps model selection
+centralized in the router.
+
+---
+
+## Architecture
+
+| Component | Runs as | Port | Notes |
+|-----------|---------|------|-------|
+| **9router** | user `ubuntu` | `127.0.0.1:20128` | AI router, Ollama connected, combo `free` |
+| **Hermes Agent** | user `ubuntu` | (gateway) | systemd **user** service + linger |
+| **Telegram bot** | via Hermes gateway | — | bot token + allowed user in `.env` |
+
+Both bind to `127.0.0.1` only (not public). To open the 9router dashboard from
+your laptop use an SSH tunnel:
+
+```bash
+ssh -L 20128:127.0.0.1:20128 ubuntu@<server-ip>
+# then open http://localhost:20128 in your browser
+```
+
+---
+
+## Prerequisites
+
+- Ubuntu 24.04 LTS server, user `ubuntu` with sudo
+- SSH access: `ssh ubuntu@<server-ip>`
+- An **Ollama API key** (Ollama Cloud) — free tier works
+- A **Telegram bot token** (from @BotFather) and your Telegram **user id**
+
+---
+
+## 1. Install 9router
+
+```bash
+# as ubuntu
+sudo apt-get update
+sudo apt-get install -y build-essential python3 make g++ curl ca-certificates gnupg
+curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
+sudo apt-get install -y nodejs
+sudo npm install -g 9router
+9router --version   # expect 0.5.50
+```
+
+### Start 9router (headless, non-root)
+
+9router needs the `-t` flag in headless mode or it crash-loops. Run as `ubuntu`:
+
+```bash
+cd /home/ubuntu
+9router -t -n --host 127.0.0.1 --port 20128 >> /home/ubuntu/9router.log 2>&1 < /dev/null &
+```
+
+Verify:
+
+```bash
+ss -ltnp | grep 20128
+curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:20128/v1/models   # 200
+```
+
+See [`files/start-9router.sh`](files/start-9router.sh) for a detached wrapper
+(suitable for `@reboot` cron).
+
+---
+
+## 2. Connect providers & create the combo
+
+9router's HTTP API requires a **cookie** auth (login via password), not a Bearer
+token. Get the dashboard password from the login screen (default set during first
+run), then:
+
+```bash
+# login -> saves cookie
+curl -c /tmp/ck.txt -X POST http://127.0.0.1:20128/api/auth/login \
+  -H "Content-Type: application/json" -d '{"password":"YOUR_DASHBOARD_PASSWORD"}'
+
+# connect Ollama (free provider, needs API key)
+curl -b /tmp/ck.txt -X POST http://127.0.0.1:20128/api/providers \
+  -H "Content-Type: application/json" \
+  -d '{"provider":"ollama","authType":"apiKey","apiKey":"YOUR_OLLAMA_KEY"}'
+
+# create combo "free" (9router will route through these)
+curl -b /tmp/ck.txt -X POST http://127.0.0.1:20128/api/combos \
+  -H "Content-Type: application/json" \
+  -d '{"name":"free","kind":"fallback","models":["ollama/gpt-oss:120b","ollama/minimax-m3"]}'
+```
+
+> **Note:** the combo's model id is just its **name** (`free`), not `combo/free`.
+> Working Ollama models observed: `ollama/gpt-oss:120b`, `ollama/minimax-m3`.
+> Others (`glm-4.7-flash`, `kimi-k2.5`, `glm-5`, `minimax-m2.5`) returned
+> 404/410 — avoid them.
+
+---
+
+## 3. Install Hermes Agent
+
+```bash
+curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
+# non-interactive variant used here:
+#   bash /tmp/hermes-install.sh --non-interactive --skip-setup
+source ~/.bashrc
+hermes --version
+```
+
+---
+
+## 4. Configure Hermes (model → 9router, Telegram)
+
+### `config.yaml`
+
+```yaml
+model:
+  default: "free"            # combo name in 9router — NOT a raw model id
+  provider: "custom"
+  base_url: "http://127.0.0.1:20128/v1"
+  api_key: "sk-...9router-api-key..."   # from 9router dashboard API section
+
+messaging:
+  telegram:
+    enabled: true
+    token: "TELEGRAM_BOT_TOKEN"
+    allowed_user_ids:
+      - "YOUR_TELEGRAM_USER_ID"
+    home_channel: telegram
+    deliver: telegram
+```
+
+### `.env` (required for the gateway to see Telegram)
+
+The gateway systemd unit does **not** auto-load `.env` — you must add
+`EnvironmentFile=` to the unit (see [`files/hermes-gateway.service`](files/hermes-gateway.service)).
+
+```bash
+TELEGRAM_BOT_TOKEN=1234567890:AAxxxxxxx
+TELEGRAM_ALLOWED_USERS=8937088575
+TELEGRAM_HOME_CHANNEL=8937088575
+```
+
+Then reload & restart:
+
+```bash
+systemctl --user daemon-reload
+systemctl --user restart hermes-gateway
+```
+
+---
+
+## 5. Enable auto-start (survives logout & reboot)
+
+```bash
+hermes gateway install      # creates ~/.config/systemd/user/hermes-gateway.service
+loginctl enable-linger ubuntu   # gateway keeps running after SSH logout
+systemctl --user enable hermes-gateway
+systemctl --user start hermes-gateway
+systemctl --user is-active hermes-gateway   # -> active
+```
+
+For 9router auto-start at boot, use an `@reboot` cron (see `files/start-9router.sh`):
+
+```bash
+# as root crontab (wrapper drops privileges to ubuntu via runuser)
+@reboot /usr/local/bin/start-9router.sh
+```
+
+---
+
+## 6. Verify
+
+```bash
+# Telegram works?
+hermes send -t telegram "Halo from Hermes via 9router!"
+
+# Agent loop actually calls 9router -> combo -> model?
+hermes -z "Balas satu kata: OK"
+# -> should print OK, proving the request flowed Hermes -> 9router -> Ollama
+```
+
+---
+
+## Files in this repo
+
+| File | Purpose |
+|------|---------|
+| `README.md` | This document |
+| `files/start-9router.sh` | Detached 9router launcher (for `@reboot` cron) |
+| `files/hermes-gateway.service` | systemd user unit (with `EnvironmentFile` for `.env`) |
+| `files/config.yaml` | Example Hermes config (model → 9router combo) |
+| `files/.env.example` | Example Telegram env vars |
+| `files/setup-9router.sh` | One-shot: connect Ollama + create combo `free` |
+
+---
+
+## Troubleshooting
+
+| Symptom | Cause | Fix |
+|---------|-------|-----|
+| 9router crash-loops on boot | Missing `-t` flag (waits for TTY) | Always start with `-t` |
+| `http://<ip>:20128` unreachable | Bound to `127.0.0.1`, not public | Use SSH tunnel `-L 20128:127.0.0.1:20128` |
+| Hermes: `model 'glm-4.7-flash' not found` | Model id not in Ollama | Use combo `free` or `ollama/gpt-oss:120b` |
+| Gateway: `No messaging platforms enabled` | `.env` not loaded by unit | Add `EnvironmentFile=%h/.hermes/.env` to unit |
+| Gateway: `No env user allowlists` | `TELEGRAM_ALLOWED_USERS` missing | Set it in `.env` |
+| 404 on `/api/providers` | Using Bearer instead of cookie | Login first, reuse cookie jar |
+
+---
+
+## Security notes
+
+- 9router & Hermes bind `127.0.0.1` only — not exposed to the internet.
+- Dashboard/API key, Ollama key, Telegram token are secrets — keep them out of
+  this repo (use `.env.example`, never commit real values).
+- Change the default SSH password (`!1Ubuntu123`) or use SSH keys only.
--- a/files/.env.example
+++ b/files/.env.example
@@ -0,0 +1,5 @@
+# Telegram credentials for Hermes gateway
+# Copy to /home/ubuntu/.hermes/.env and chmod 600
+TELEGRAM_BOT_TOKEN=REPLACE_WITH_BOT_TOKEN
+TELEGRAM_ALLOWED_USERS=REPLACE_WITH_USER_ID
+TELEGRAM_HOME_CHANNEL=REPLACE_WITH_USER_ID
--- a/files/config.yaml
+++ b/files/config.yaml
@@ -0,0 +1,14 @@
+model:
+  default: "free"            # 9router combo name — 9router picks the model
+  provider: "custom"
+  base_url: "http://127.0.0.1:20128/v1"
+  api_key: "REPLACE_WITH_9ROUTER_API_KEY"   # from 9router dashboard → API section
+
+messaging:
+  telegram:
+    enabled: true
+    token: "REPLACE_WITH_TELEGRAM_BOT_TOKEN"
+    allowed_user_ids:
+      - "REPLACE_WITH_TELEGRAM_USER_ID"
+    home_channel: telegram
+    deliver: telegram
--- a/files/hermes-gateway.service
+++ b/files/hermes-gateway.service
@@ -0,0 +1,30 @@
+[Unit]
+Description=Hermes Agent Gateway - Messaging Platform Integration
+After=network-online.target
+Wants=network-online.target
+StartLimitIntervalSec=0
+
+[Service]
+Type=simple
+ExecStart=/home/ubuntu/.hermes/hermes-agent/venv/bin/python -m hermes_cli.main gateway run
+WorkingDirectory=/home/ubuntu/.hermes
+Environment="PATH=/home/ubuntu/.hermes/hermes-agent/venv/bin:/home/ubuntu/.hermes/hermes-agent/node_modules/.bin:/usr/bin:/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
+Environment="VIRTUAL_ENV=/home/ubuntu/.hermes/hermes-agent/venv"
+Environment="HERMES_HOME=/home/ubuntu/.hermes"
+# REQUIRED: gateway reads Telegram creds from .env — without this line the
+# platform stays disabled ("No messaging platforms enabled").
+EnvironmentFile=%h/.hermes/.env
+Restart=always
+RestartSec=5
+RestartForceExitStatus=75
+RestartPreventExitStatus=78
+KillMode=mixed
+KillSignal=SIGTERM
+ExecReload=/bin/kill -USR1 $MAINPID
+ExecStopPost=-/home/ubuntu/.hermes/hermes-agent/venv/bin/python -m gateway.cgroup_cleanup
+TimeoutStopSec=60
+StandardOutput=journal
+StandardError=journal
+
+[Install]
+WantedBy=default.target
--- a/files/setup-9router.sh
+++ b/files/setup-9router.sh
@@ -0,0 +1,29 @@
+#!/bin/sh
+# setup-9router.sh — connect Ollama + create combo "free" on a running 9router.
+# Usage:  ./setup-9router.sh <dashboard_password> <ollama_api_key>
+# Requires 9router already listening on http://127.0.0.1:20128
+set -e
+
+PASS="$1"
+OLLAMA_KEY="$2"
+BASE=http://127.0.0.1:20128
+CK=/tmp/9r-ck.txt
+
+[ -z "$PASS" ] && { echo "usage: $0 <dashboard_password> <ollama_api_key>"; exit 1; }
+[ -z "$OLLAMA_KEY" ] && { echo "usage: $0 <dashboard_password> <ollama_api_key>"; exit 1; }
+
+echo "==> login"
+curl -s -c "$CK" -X POST "$BASE/api/auth/login" \
+  -H "Content-Type: application/json" -d "{\"password\":\"$PASS\"}" >/dev/null
+
+echo "==> connect Ollama"
+curl -s -b "$CK" -X POST "$BASE/api/providers" \
+  -H "Content-Type: application/json" \
+  -d "{\"provider\":\"ollama\",\"authType\":\"apiKey\",\"apiKey\":\"$OLLAMA_KEY\"}"
+
+echo "==> create combo 'free' (fallback gpt-oss:120b + minimax-m3)"
+curl -s -b "$CK" -X POST "$BASE/api/combos" \
+  -H "Content-Type: application/json" \
+  -d '{"name":"free","kind":"fallback","models":["ollama/gpt-oss:120b","ollama/minimax-m3"]}'
+
+echo "==> done. Hermes should use model: \"free\""
--- a/files/start-9router.sh
+++ b/files/start-9router.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+# Detached 9router launcher for @reboot cron.
+# Run as root crontab; drops privileges to user 'ubuntu' via runuser.
+# Uses absolute paths (cron @reboot has minimal PATH).
+SETSID=/usr/bin/setsid
+RUNUSER=/usr/sbin/runuser
+BIN=/usr/local/bin/9router
+USER=ubuntu
+LOG=/home/ubuntu/9router.log
+
+if [ "$(id -u)" -eq 0 ]; then
+  exec $SETSID $RUNUSER -u $USER -- $BIN -t -n --host 127.0.0.1 --port 20128 >> "$LOG" 2>&1 < /dev/null
+else
+  exec $SETSID $BIN -t -n --host 127.0.0.1 --port 20128 >> "$LOG" 2>&1 < /dev/null
+fi