Docs: 9router + Hermes Agent local AI stack with Telegram (9router routes model via combo)
This commit is contained in:
cania
227
README.md
Normal file
227
README.md
Normal file
@@ -0,0 +1,227 @@
|
||||
# 9router + Hermes Agent — Local AI Stack with Telegram
|
||||
|
||||
Run a **self-hosted AI assistant** on a single Ubuntu server:
|
||||
|
||||
```
|
||||
Telegram ──► Hermes Agent (gateway)
|
||||
│ (model = "free", base_url = 9router)
|
||||
▼
|
||||
9router (http://127.0.0.1:20128/v1)
|
||||
│ combo "free" (fallback routing)
|
||||
├─► ollama/gpt-oss:120b
|
||||
└─► ollama/minimax-m3
|
||||
```
|
||||
|
||||
**Key design point:** Hermes does NOT pick the model. It sends every request to
|
||||
9router with `model: "free"`, and **9router itself chooses** which free model to
|
||||
use (via a *combo* with `fallback` strategy). This keeps model selection
|
||||
centralized in the router.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
| Component | Runs as | Port | Notes |
|
||||
|-----------|---------|------|-------|
|
||||
| **9router** | user `ubuntu` | `127.0.0.1:20128` | AI router, Ollama connected, combo `free` |
|
||||
| **Hermes Agent** | user `ubuntu` | (gateway) | systemd **user** service + linger |
|
||||
| **Telegram bot** | via Hermes gateway | — | bot token + allowed user in `.env` |
|
||||
|
||||
Both bind to `127.0.0.1` only (not public). To open the 9router dashboard from
|
||||
your laptop use an SSH tunnel:
|
||||
|
||||
```bash
|
||||
ssh -L 20128:127.0.0.1:20128 ubuntu@<server-ip>
|
||||
# then open http://localhost:20128 in your browser
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ubuntu 24.04 LTS server, user `ubuntu` with sudo
|
||||
- SSH access: `ssh ubuntu@<server-ip>`
|
||||
- An **Ollama API key** (Ollama Cloud) — free tier works
|
||||
- A **Telegram bot token** (from @BotFather) and your Telegram **user id**
|
||||
|
||||
---
|
||||
|
||||
## 1. Install 9router
|
||||
|
||||
```bash
|
||||
# as ubuntu
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y build-essential python3 make g++ curl ca-certificates gnupg
|
||||
curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
|
||||
sudo apt-get install -y nodejs
|
||||
sudo npm install -g 9router
|
||||
9router --version # expect 0.5.50
|
||||
```
|
||||
|
||||
### Start 9router (headless, non-root)
|
||||
|
||||
9router needs the `-t` flag in headless mode or it crash-loops. Run as `ubuntu`:
|
||||
|
||||
```bash
|
||||
cd /home/ubuntu
|
||||
9router -t -n --host 127.0.0.1 --port 20128 >> /home/ubuntu/9router.log 2>&1 < /dev/null &
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
ss -ltnp | grep 20128
|
||||
curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:20128/v1/models # 200
|
||||
```
|
||||
|
||||
See [`files/start-9router.sh`](files/start-9router.sh) for a detached wrapper
|
||||
(suitable for `@reboot` cron).
|
||||
|
||||
---
|
||||
|
||||
## 2. Connect providers & create the combo
|
||||
|
||||
9router's HTTP API requires a **cookie** auth (login via password), not a Bearer
|
||||
token. Get the dashboard password from the login screen (default set during first
|
||||
run), then:
|
||||
|
||||
```bash
|
||||
# login -> saves cookie
|
||||
curl -c /tmp/ck.txt -X POST http://127.0.0.1:20128/api/auth/login \
|
||||
-H "Content-Type: application/json" -d '{"password":"YOUR_DASHBOARD_PASSWORD"}'
|
||||
|
||||
# connect Ollama (free provider, needs API key)
|
||||
curl -b /tmp/ck.txt -X POST http://127.0.0.1:20128/api/providers \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"provider":"ollama","authType":"apiKey","apiKey":"YOUR_OLLAMA_KEY"}'
|
||||
|
||||
# create combo "free" (9router will route through these)
|
||||
curl -b /tmp/ck.txt -X POST http://127.0.0.1:20128/api/combos \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name":"free","kind":"fallback","models":["ollama/gpt-oss:120b","ollama/minimax-m3"]}'
|
||||
```
|
||||
|
||||
> **Note:** the combo's model id is just its **name** (`free`), not `combo/free`.
|
||||
> Working Ollama models observed: `ollama/gpt-oss:120b`, `ollama/minimax-m3`.
|
||||
> Others (`glm-4.7-flash`, `kimi-k2.5`, `glm-5`, `minimax-m2.5`) returned
|
||||
> 404/410 — avoid them.
|
||||
|
||||
---
|
||||
|
||||
## 3. Install Hermes Agent
|
||||
|
||||
```bash
|
||||
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
|
||||
# non-interactive variant used here:
|
||||
# bash /tmp/hermes-install.sh --non-interactive --skip-setup
|
||||
source ~/.bashrc
|
||||
hermes --version
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Configure Hermes (model → 9router, Telegram)
|
||||
|
||||
### `config.yaml`
|
||||
|
||||
```yaml
|
||||
model:
|
||||
default: "free" # combo name in 9router — NOT a raw model id
|
||||
provider: "custom"
|
||||
base_url: "http://127.0.0.1:20128/v1"
|
||||
api_key: "sk-...9router-api-key..." # from 9router dashboard API section
|
||||
|
||||
messaging:
|
||||
telegram:
|
||||
enabled: true
|
||||
token: "TELEGRAM_BOT_TOKEN"
|
||||
allowed_user_ids:
|
||||
- "YOUR_TELEGRAM_USER_ID"
|
||||
home_channel: telegram
|
||||
deliver: telegram
|
||||
```
|
||||
|
||||
### `.env` (required for the gateway to see Telegram)
|
||||
|
||||
The gateway systemd unit does **not** auto-load `.env` — you must add
|
||||
`EnvironmentFile=` to the unit (see [`files/hermes-gateway.service`](files/hermes-gateway.service)).
|
||||
|
||||
```bash
|
||||
TELEGRAM_BOT_TOKEN=1234567890:AAxxxxxxx
|
||||
TELEGRAM_ALLOWED_USERS=8937088575
|
||||
TELEGRAM_HOME_CHANNEL=8937088575
|
||||
```
|
||||
|
||||
Then reload & restart:
|
||||
|
||||
```bash
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user restart hermes-gateway
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Enable auto-start (survives logout & reboot)
|
||||
|
||||
```bash
|
||||
hermes gateway install # creates ~/.config/systemd/user/hermes-gateway.service
|
||||
loginctl enable-linger ubuntu # gateway keeps running after SSH logout
|
||||
systemctl --user enable hermes-gateway
|
||||
systemctl --user start hermes-gateway
|
||||
systemctl --user is-active hermes-gateway # -> active
|
||||
```
|
||||
|
||||
For 9router auto-start at boot, use an `@reboot` cron (see `files/start-9router.sh`):
|
||||
|
||||
```bash
|
||||
# as root crontab (wrapper drops privileges to ubuntu via runuser)
|
||||
@reboot /usr/local/bin/start-9router.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Verify
|
||||
|
||||
```bash
|
||||
# Telegram works?
|
||||
hermes send -t telegram "Halo from Hermes via 9router!"
|
||||
|
||||
# Agent loop actually calls 9router -> combo -> model?
|
||||
hermes -z "Balas satu kata: OK"
|
||||
# -> should print OK, proving the request flowed Hermes -> 9router -> Ollama
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Files in this repo
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `README.md` | This document |
|
||||
| `files/start-9router.sh` | Detached 9router launcher (for `@reboot` cron) |
|
||||
| `files/hermes-gateway.service` | systemd user unit (with `EnvironmentFile` for `.env`) |
|
||||
| `files/config.yaml` | Example Hermes config (model → 9router combo) |
|
||||
| `files/.env.example` | Example Telegram env vars |
|
||||
| `files/setup-9router.sh` | One-shot: connect Ollama + create combo `free` |
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| 9router crash-loops on boot | Missing `-t` flag (waits for TTY) | Always start with `-t` |
|
||||
| `http://<ip>:20128` unreachable | Bound to `127.0.0.1`, not public | Use SSH tunnel `-L 20128:127.0.0.1:20128` |
|
||||
| Hermes: `model 'glm-4.7-flash' not found` | Model id not in Ollama | Use combo `free` or `ollama/gpt-oss:120b` |
|
||||
| Gateway: `No messaging platforms enabled` | `.env` not loaded by unit | Add `EnvironmentFile=%h/.hermes/.env` to unit |
|
||||
| Gateway: `No env user allowlists` | `TELEGRAM_ALLOWED_USERS` missing | Set it in `.env` |
|
||||
| 404 on `/api/providers` | Using Bearer instead of cookie | Login first, reuse cookie jar |
|
||||
|
||||
---
|
||||
|
||||
## Security notes
|
||||
|
||||
- 9router & Hermes bind `127.0.0.1` only — not exposed to the internet.
|
||||
- Dashboard/API key, Ollama key, Telegram token are secrets — keep them out of
|
||||
this repo (use `.env.example`, never commit real values).
|
||||
- Change the default SSH password (`!1Ubuntu123`) or use SSH keys only.
|
||||
Reference in New Issue
Block a user