Initial commit: Hermes Agent skills collection
- Trading skills (OKX, dividend, lottery, quantitative) - Creative skills (ASCII art, diagrams, video) - Development skills (GitHub, debugging, TDD) - Research skills (arXiv, blog monitoring) - Productivity skills (email, documents, notes) - MCP integration skills - Custom user skills
This commit is contained in:
Hermes Skills Manager
@@ -0,0 +1,557 @@
|
||||
---
|
||||
name: telegram-message-forwarder
|
||||
description: >
|
||||
Deploy and manage Telegram message forwarding between channels/groups.
|
||||
Use when user asks to forward messages from one TG channel to another,
|
||||
set up a message relay, monitor channels, or says "消息转发器", "bind频道",
|
||||
"转发到我的群".
|
||||
trigger:
|
||||
- telegram forwarder
|
||||
- 消息转发器
|
||||
- bind频道
|
||||
- 转发频道
|
||||
- channel forwarder
|
||||
- message relay
|
||||
---
|
||||
|
||||
# Telegram Message Forwarder
|
||||
|
||||
Deploy Heavrnl/TelegramForwarder — the most feature-rich open-source TG forwarder (1144+ ⭐).
|
||||
|
||||
## Why This One
|
||||
|
||||
| Project | Stars | Bind Command | GUI | Filters | AI | Docker |
|
||||
|---------|-------|-------------|-----|---------|-----|--------|
|
||||
| **Heavrnl/TelegramForwarder** | 1144 | ✅ `/bind` | ✅ Web + Inline KB | ✅ Keyword/Regex | ✅ Multi-provider | ✅ |
|
||||
| aahnik/tgcf | 1600 | ❌ Config-only | ✅ Web | ✅ Plugin-based | ❌ | ✅ |
|
||||
| MrMissx/Telegram_Forwarder | 636 | ❌ JSON config | ❌ | ✅ Basic | ❌ | ❌ |
|
||||
| GentlemanHu/TelegramForwarder | 36 | ❌ Inline KB only | ✅ Inline KB | ✅ Basic | ❌ | ❌ |
|
||||
|
||||
**Heavrnl wins** because it has `/bind` command, inline keyboard GUI, regex replace, AI processing, RSS, and multi-platform push (Apprise).
|
||||
|
||||
## Quick Deploy (Docker)
|
||||
|
||||
```bash
|
||||
cd ~
|
||||
git clone https://github.com/Heavrnl/TelegramForwarder.git
|
||||
cd TelegramForwarder
|
||||
cp .env.example .env
|
||||
# Edit .env with required values (see below)
|
||||
docker-compose run -it telegram-forwarder # first run: interactive for phone verification
|
||||
# CTRL+C after verification
|
||||
# Edit docker-compose.yml: set stdin_open: false, tty: false
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
**⚠️ ghcr.io pulls often fail with "unexpected EOF"** on this server. Use Chinese Docker mirrors:
|
||||
- `docker.1ms.run` — 1ms acceleration (already configured in `/etc/docker/daemon.json`)
|
||||
- `ghcr.nju.edu.cn` — NJU mirror for ghcr.io images
|
||||
If pulls still fail, retry once (transient network issues) or use `network_mode: host` with Mihomo proxy.
|
||||
|
||||
## Required .env Values
|
||||
|
||||
| Variable | Source | Notes |
|
||||
|----------|--------|-------|
|
||||
| `API_ID` | https://my.telegram.org/apps | Same as TG_API_ID in .bashrc |
|
||||
| `API_HASH` | https://my.telegram.org/apps | Same as TG_API_HASH in .bashrc |
|
||||
| `BOT_TOKEN` | @BotFather `/newbot` | Must be DIFFERENT from Hermes bot token |
|
||||
| `PHONE_NUMBER` | User's TG phone | Format: `+86...` |
|
||||
| `USER_ID` | @userinfobot | Numeric user ID |
|
||||
|
||||
**⚠️ BOT_TOKEN conflict**: Cannot reuse Hermes's bot token. Need a separate bot from @BotFather.
|
||||
|
||||
## Usage (Bot Commands)
|
||||
|
||||
```
|
||||
/bind https://t.me/channel_name # Bind source channel to current group
|
||||
/bind "频道名称" # Bind by name
|
||||
/settings # Open settings GUI (inline keyboard)
|
||||
/add keyword1 keyword2 # Add filter keywords
|
||||
/remove_keyword keyword1 # Remove keywords
|
||||
/replace regex_pattern # Add regex replacement
|
||||
/list_keyword # List current keywords
|
||||
/list_replace # List current replacements
|
||||
```
|
||||
|
||||
### Workflow
|
||||
1. Create a Telegram group/channel as the destination
|
||||
2. Add the bot to that group as admin
|
||||
3. In the group, send `/bind https://t.me/source_channel`
|
||||
4. Configure filters via `/settings` or `/add`
|
||||
5. Messages auto-forward with filters applied
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Multi-source**: Bind multiple sources to one destination
|
||||
- **Keyword filter**: Whitelist/blacklist mode
|
||||
- **Regex replace**: Clean up formatting (e.g., remove `**`, fix links)
|
||||
- **Media filter**: Filter by type (photo, video, audio, document)
|
||||
- **AI processing**: Use OpenAI/Claude/Gemini/DeepSeek/Qwen for message rewriting
|
||||
- **RSS**: Generate RSS feed from forwarded messages
|
||||
- **Apprise push**: Forward to email, Slack, Discord, webhooks, etc.
|
||||
- **Sync rules**: Mirror settings across multiple bind rules
|
||||
- **Time-based filter**: Forward only during specific hours
|
||||
|
||||
## Proxy Setup (Required for This Server)
|
||||
|
||||
This server **cannot reach Telegram directly**. Telethon (used by TelegramForwarder) needs explicit SOCKS5 proxy configuration — standard `HTTP_PROXY` env vars are NOT enough for MTProto connections.
|
||||
|
||||
### Step-by-step
|
||||
|
||||
1. **Patch `main.py`** to add SOCKS5 proxy support for Telethon:
|
||||
|
||||
```python
|
||||
# Add after 'from telethon import TelegramClient, types':
|
||||
try:
|
||||
import socks
|
||||
HAS_SOCKS = True
|
||||
except ImportError:
|
||||
HAS_SOCKS = False
|
||||
|
||||
# Replace client creation:
|
||||
proxy_config = None
|
||||
if os.getenv('HTTP_PROXY') or os.getenv('http_proxy'):
|
||||
proxy_addr = os.getenv('HTTP_PROXY') or os.getenv('http_proxy')
|
||||
import re
|
||||
m = re.match(r'https?://([\d.]+):(\d+)', proxy_addr)
|
||||
if m and HAS_SOCKS:
|
||||
proxy_host, proxy_port = m.group(1), int(m.group(2))
|
||||
proxy_config = (socks.SOCKS5, proxy_host, proxy_port)
|
||||
|
||||
user_client = TelegramClient('./sessions/user', api_id, api_hash, proxy=proxy_config)
|
||||
bot_client = TelegramClient('./sessions/bot', api_id, api_hash, proxy=proxy_config)
|
||||
```
|
||||
|
||||
2. **Create `Dockerfile.custom`** (base image lacks PySocks):
|
||||
|
||||
```dockerfile
|
||||
FROM heavrnl/telegramforwarder:latest
|
||||
RUN pip install --no-cache-dir PySocks
|
||||
COPY main.py /app/main.py
|
||||
```
|
||||
|
||||
3. **Update `docker-compose.yml`** — use `network_mode: host` + proxy env:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
telegram-forwarder:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.custom
|
||||
container_name: telegram-forwarder
|
||||
restart: unless-stopped
|
||||
network_mode: host
|
||||
environment:
|
||||
- HTTP_PROXY=http://127.0.0.1:7890
|
||||
- HTTPS_PROXY=http://127.0.0.1:7890
|
||||
- http_proxy=http://127.0.0.1:7890
|
||||
- https_proxy=http://127.0.0.1:7890
|
||||
- NO_PROXY=localhost,127.0.0.1
|
||||
volumes:
|
||||
- ./db:/app/db
|
||||
- ./.env:/app/.env
|
||||
- ./logs:/app/logs
|
||||
- ./sessions:/app/sessions
|
||||
- ./temp:/app/temp
|
||||
- ./ufb/config:/app/ufb/config
|
||||
- ./config:/app/config
|
||||
- ./rss/data:/app/rss/data
|
||||
- ./rss/media:/app/rss/media
|
||||
stdin_open: true
|
||||
tty: true
|
||||
```
|
||||
|
||||
4. **Build and run**:
|
||||
```bash
|
||||
docker-compose build
|
||||
docker-compose run --rm telegram-forwarder # interactive for phone code
|
||||
# After verification, set stdin_open: false, tty: false
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
**⚠️ Phone verification should be done by the user via SSH**, not by the agent. Verification codes expire in ~30 seconds, and agent latency through `process(action='submit')` is too slow. Tell the user to SSH in and run the interactive command themselves. If 2FA is enabled, they'll also need their Telegram two-step verification password.
|
||||
|
||||
### Pitfalls
|
||||
|
||||
- **🔴 CRITICAL: Competing gateway processes steal bot token polling** — If another process (old OpenClaw Gateway, duplicate Hermes instance, test script) is using the same Telegram bot token for `getUpdates`, the Hermes adapter will get stuck in permanent polling conflict. **Symptom**: Gateway shows `telegram: connected` in state, but `getUpdates` via API returns 0 results WITHOUT a 409 conflict (meaning no one is actually polling). No "✓ telegram connected" in logs. Repeated `Telegram polling conflict (1/5)` warnings. Group messages NEVER processed. **Root cause**: Two processes share the same bot token — Telegram only allows ONE `getUpdates` session per token. **Diagnosis**: `ps aux | grep gateway | grep -v grep` — look for stale processes (e.g., old `openclaw ... gateway --port 18789` from before migration to Hermes). **Fix**: Kill the stale process first, THEN restart Hermes: `kill <stale_pid> && systemctl --user restart hermes-gateway`. **Also check**: `getUpdates` returning 0 without 409 conflict means NO adapter is polling — this is the key diagnostic signal. If you get 409, the adapter IS polling (normal). If you get 0, nobody is polling (broken).
|
||||
- **🔴 CRITICAL: Gateway restart causes Telegram polling conflict** — When `hermes gateway restart` is called, the old Telegram polling session may NOT release on Telegram's servers. The new adapter gets stuck in a permanent conflict loop: `Conflict: terminated by other getUpdates request; make sure that only one bot instance is running`. **Symptom**: Gateway restarts but group messages are never processed, zero Telegram adapter logs. **Root cause**: `hermes gateway restart` sends SIGTERM → systemd restarts immediately → old polling session still active → new adapter can't connect. **Fix**: NEVER use `hermes gateway restart`. Instead, use the three-step process:
|
||||
```bash
|
||||
systemctl --user stop hermes-gateway # stop
|
||||
sleep 30 # wait for Telegram session to expire
|
||||
systemctl --user start hermes-gateway # start fresh
|
||||
```
|
||||
Or from inside the gateway, kill the PID to trigger systemd auto-restart:
|
||||
```bash
|
||||
kill <gateway_pid> # SIGTERM, systemd Restart=always kicks in
|
||||
```
|
||||
But even the kill approach may hit the conflict if systemd restarts too fast. The safest approach is stop → wait → start. **This bug means group messages may have NEVER worked** — every time the user restarted the gateway, the conflict silently killed the adapter.
|
||||
- **Group migrated to supergroup (silent failure)**: Telegram sometimes migrates a basic group to a supergroup, changing its ID. The old chat ID becomes `deactivated=True` and forwarding fails silently with `An invalid Peer was used... bots cannot start conversations`. To diagnose: `bot.get_entity(old_id)` — the returned object will have `deactivated=True` and `migrated_to=InputChannel(channel_id=NEW_ID, access_hash=...)`. The new supergroup ID in Telethon format is `-100` + `channel_id` (e.g., channel_id=3966251111 → `-1003966251111`). Update BOTH the forwarder DB AND Hermes's `free_response_channels` in `config.yaml`. If `hermes gateway restart` is refused from inside the gateway, ask the user to run it from a separate shell.
|
||||
- **Bot-to-bot messaging FAILS**: If the forwarding target is a bot (e.g., Hermes Bot), Telethon throws `Bots can't send messages to other bots`. Telegram prohibits bot→bot messaging. **Fix**: Forward to the GROUP where the Hermes bot resides, NOT directly to the bot. The bot reads messages from the group as a member. If there's no group yet, create one, add both bots, and forward there. The `chats` table column is `telegram_chat_id` (NOT `chat_id`).
|
||||
- **Bot Privacy Mode blocks group messages (CRITICAL)**: Telegram bots have **privacy mode ON by default**. A bot with privacy mode can only see: (1) commands (/start), (2) messages that @mention it, (3) replies to its messages. It CANNOT see regular group messages — even with `free_response_channels` configured in Hermes. **Symptom**: Forwarder sends messages to the group successfully, but Hermes never responds. **Fix**: Open @BotFather → `/mybots` → select the Hermes bot → **Bot Settings** → **Group Privacy** → **Turn off**. **After turning off privacy mode, kick the bot from the group and re-add it** — the change doesn't apply to bots already in a group. Verify with API: `can_read_all_group_messages` must be `true`. Must be done for BOTH the Hermes bot AND the forwarder bot if both need to read group messages.
|
||||
- **Testing: use human account, not bot**: When testing the forwarding → Hermes chain, send test messages from a **human Telegram account**, NOT from the forwarder bot. Hermes may filter or ignore messages sent by other bots in groups. Use the Telethon **user client** (`sessions/user`) to send test messages to the source channel, or have the user manually post in the group. Bot-sent test messages can falsely appear as "Hermes not receiving" when the real issue is bot-message filtering.
|
||||
- **Telethon ignores HTTP_PROXY**: Telethon uses raw TCP sockets for MTProto, not HTTP. You MUST configure proxy at the Telethon client level via the `proxy` parameter. The `HTTP_PROXY` env var alone does NOT work.
|
||||
- **`network_mode: host`** lets the container reach `127.0.0.1:7890` on the host. Without it, you'd need `extra_hosts: ["host.docker.internal:host-gateway"]` and use that hostname instead.
|
||||
- **Mihomo mixed-port 7890** supports both HTTP and SOCKS5 on the same port. Use SOCKS5 for Telethon.
|
||||
- **Session database locked**: If you see `sqlite3.OperationalError: database is locked`, clean `./sessions/*.session*` files and retry. Caused by unclean shutdown.
|
||||
- **git clone needs proxy**: `http_proxy=http://127.0.0.1:7890 https_proxy=http://127.0.0.1:7890 git clone ...`
|
||||
- **Phone verification**: First run MUST be interactive (`docker-compose run --rm`, not `up -d`). The code is sent to the user's Telegram. After verification, session file persists in `./sessions/`.
|
||||
- **Custom image required**: Modifying `requirements.txt` locally doesn't affect the pre-built Docker image. You must build a custom image with `Dockerfile.custom` that installs PySocks AND copies the patched `main.py`.
|
||||
|
||||
### Hermes Bot Integration (Forwarded Messages → Agent Processing)
|
||||
|
||||
When the user wants forwarded messages to be **processed by Hermes** (e.g., auto-trade on signals), set up the forwarding chain:
|
||||
|
||||
1. **Create a destination group** (e.g., "Telegram消息转发") — this is the meeting point
|
||||
2. **Add the forwarder bot** (MsgForwarder) to the group as admin
|
||||
3. **Add the Hermes bot** to the group as admin
|
||||
4. **⚠️ DISABLE PRIVACY MODE on the Hermes bot** via @BotFather → `/mybots` → select bot → Bot Settings → Group Privacy → Turn off. **This step is mandatory** — without it, Hermes cannot see group messages and `free_response_channels` has no effect. **After disabling privacy mode, you MUST kick the bot from the group and re-add it** — the privacy mode change does not take effect for bots already in a group. Verify with `getMe` API: `can_read_all_group_messages` must be `true`.
|
||||
5. **Configure `free_response_chats`** in `~/.hermes/config.yaml`:
|
||||
```yaml
|
||||
telegram:
|
||||
extra:
|
||||
free_response_chats: '<destination_group_chat_id>'
|
||||
```
|
||||
This lets Hermes respond to messages in that group **without requiring @mention**.
|
||||
6. **Restart Hermes** for config to take effect. Run `hermes gateway restart` from a separate shell (not from inside the gateway).
|
||||
|
||||
**⚠️ CRITICAL PITFALL**: The config key is `free_response_chats` inside `telegram.extra`, NOT `free_response_channels` at the top level! The Hermes source code reads `self.config.extra.get("free_response_chats")`. Wrong key = silently ignored, bot only responds to @mentions.
|
||||
|
||||
```yaml
|
||||
telegram:
|
||||
extra:
|
||||
free_response_chats: '-100XXXXXXXXXX' # ← correct key
|
||||
# free_response_channels: '...' ← WRONG, ignored by code
|
||||
```
|
||||
|
||||
**⚠️ Pitfall**: `require_mention: true` (default) means Hermes ignores group messages unless @mentioned. Without correct `free_response_chats`, forwarded messages are silently ignored.
|
||||
|
||||
**⚠️ Pitfall**: After changing bot privacy mode in BotFather, you must kick the bot from the group and re-add it for the change to take effect.
|
||||
|
||||
**⚠️ Pitfall**: Don't reuse the Hermes bot token as the forwarder's BOT_TOKEN — they'll conflict. Use a separate @BotFather bot for the forwarder.
|
||||
|
||||
**⚠️ Pitfall**: If the destination group was ever migrated from a basic group to a supergroup, the chat ID changed. See "Group migrated to supergroup" pitfall above. Update BOTH the forwarder DB and `free_response_channels`.
|
||||
|
||||
### End-to-End Verification
|
||||
|
||||
After setting up the forwarding → Hermes chain, verify each link:
|
||||
|
||||
1. **Forwarder listening**: `docker logs telegram-forwarder 2>&1 | grep "收到新消息"` — should show messages from source channel
|
||||
2. **Forwarder sending**: `docker logs telegram-forwarder 2>&1 | grep "已发送"` — should show successful sends to target group
|
||||
3. **Hermes receiving**: `hermes logs 2>&1 | grep "inbound message"` — should show messages from the group (chat ID should match the group, not the user DM)
|
||||
4. **Hermes responding**: Check the group for Hermes's reply
|
||||
|
||||
**Common failure chain**:
|
||||
- Forwarder logs show "收到新消息" but no "已发送" → keyword filter issue or invalid Peer (check group ID)
|
||||
- Forwarder logs show "已发送" but Hermes shows no inbound from group → privacy mode still on, or `free_response_channels` wrong
|
||||
- Hermes shows inbound but no response → check `require_mention` and `free_response_channels`
|
||||
|
||||
**⚠️ For testing, use a human account** (not the forwarder bot) to send messages to the source channel or directly to the group. Bot-sent messages may be filtered by Hermes.
|
||||
|
||||
**⚠️ STRICT REGEX SILENTLY BLOCKS ALL SIGNALS (2026-06-25 discovered)**: If the whitelist keyword is a strict regex like `【币种】[\\s\\S]*【方向】[\\s\\S]*【仓位】`, ANY message that doesn't contain ALL those keywords in order gets silently dropped. The forwarder logs show "未匹配到普通白名单关键词，不转发" but there's NO alert to the user. **This happened in production** — 39 signals were blocked over 8 hours before the user noticed. **Root cause**: the source channel changed its message format (added `【field】: value` with colons, changed trader name format), but the forwarder filter still required the old keyword pattern. **Lesson**: Always use `.*` for agent-side filtering. If you must filter at the forwarder level, add monitoring: periodically check `docker logs telegram-forwarder | grep -c "未匹配"` to detect mass-blocked messages.
|
||||
|
||||
**⚠️ Agent-Side Filtering Pattern (Verified 2026-06-25 in production)**: Set forwarder to pass ALL messages (`.*` whitelist) and let the Hermes agent classify via `channel_prompts`. This session proved this pattern works: forwarder passes everything, agent prompt classifies messages into A(交易信号)/B(确认取消)/C(平仓)/D(非交易) and only processes A/B/C. If source format changes, only the agent prompt needs updating, not the forwarder database. Full template in `okx-auto-position` skill's `references/channel-prompts-template.md`.
|
||||
|
||||
**⚠️ Privacy mode off but bot still can't see group messages**: If `getMe` shows `can_read_all_group_messages: true` but Hermes still doesn't receive group messages after re-adding the bot and restarting the gateway, try these steps in order:
|
||||
1. Verify the bot is actually IN the group: use `iter_participants` via Telethon or `getChatMember` API
|
||||
2. Check `allowed_chats` in Hermes config — if set, the group ID must be listed
|
||||
3. Have a **human** (not a bot) send a message in the group — bot-to-bot messages are filtered by Telegram itself
|
||||
4. Try `@Mike94_HermesBot hello` (explicit @mention) to test if the bot receives mentioned messages at all
|
||||
5. If @mentions work but regular messages don't, the issue is likely a Telegram API propagation delay for the privacy mode change — wait 5-10 minutes and try again
|
||||
6. As a last resort, create a brand new bot via @BotFather with privacy mode off from the start
|
||||
|
||||
### Direct Database Keyword Management
|
||||
|
||||
When bot commands aren't accessible (e.g., from agent), manage keywords directly in SQLite.
|
||||
|
||||
**⚠️ `sqlite3` CLI is NOT available in the container.** Always use Python's `sqlite3` module inside the container, OR copy the DB out to host:
|
||||
|
||||
**Option A: Edit from host (recommended for complex changes)**:
|
||||
```bash
|
||||
docker cp telegram-forwarder:/app/db/forward.db /tmp/forward.db
|
||||
sqlite3 /tmp/forward.db "DELETE FROM keywords WHERE rule_id=1 AND is_blacklist=0;"
|
||||
sqlite3 /tmp/forward.db "INSERT INTO keywords (rule_id, keyword, is_regex, is_blacklist) VALUES (1, '.*', 1, 0);"
|
||||
docker cp /tmp/forward.db telegram-forwarder:/app/db/forward.db
|
||||
docker restart telegram-forwarder
|
||||
```
|
||||
|
||||
**Option B: Edit inside container (Python)**:
|
||||
```bash
|
||||
docker exec telegram-forwarder python3 -c "
|
||||
import sqlite3
|
||||
conn = sqlite3.connect('/app/db/forward.db')
|
||||
c = conn.cursor()
|
||||
c.execute('INSERT INTO keywords (rule_id, keyword, is_regex, is_blacklist) VALUES (1, \\\"未知\\\", 0, 1)')
|
||||
conn.commit()
|
||||
conn.close()
|
||||
"
|
||||
docker restart telegram-forwarder
|
||||
```
|
||||
|
||||
**⚠️ Option A pitfall**: `docker cp` out while container is running works for READING, but WRITING back may give `attempt to write a readonly database` if SQLite WAL mode is active. Safer to stop the container first, copy back, then start. Or use `docker exec` with Python (Option B) which handles the lock correctly.
|
||||
|
||||
**⚠️ The DB file is locked while the container runs.** If you need to edit from the host (not via `docker exec`), stop the container first: `docker stop telegram-forwarder`, edit `./db/forward.db`, then `docker start telegram-forwarder`. Writing from host while container is up gives `attempt to write a readonly database`.
|
||||
|
||||
**DB Schema**:
|
||||
- `forward_rules` — source→dest mappings, filter modes, templates
|
||||
- `keywords` — `(id, rule_id, keyword, is_regex, is_blacklist)` where `is_blacklist=1` means skip matching messages, `is_regex=1` enables regex mode
|
||||
- `replace_rules` — regex find/replace patterns
|
||||
- `chats` — `(id, telegram_chat_id, name, current_add_id)` — column is `telegram_chat_id`, NOT `chat_id`
|
||||
- **Multiple white keywords = OR logic** (any match triggers forwarding). For AND (all must match), use a single regex. See "Whitelist Regex" section below.
|
||||
|
||||
### Whitelist Regex: Let Everything Through (Recommended for Agent-Side Filtering)
|
||||
|
||||
**When the downstream agent (Hermes) handles message classification**, set the forwarder to pass ALL messages and let the agent decide what to process. This is more robust than maintaining regex patterns in the forwarder — if the source channel's message format changes, you only need to update the agent's prompt, not the forwarder's database.
|
||||
|
||||
```bash
|
||||
# Set whitelist regex to .*=match everything
|
||||
docker cp telegram-forwarder:/app/db/forward.db /tmp/forward.db
|
||||
sqlite3 /tmp/forward.db "DELETE FROM keywords WHERE rule_id=1 AND is_blacklist=0;"
|
||||
sqlite3 /tmp/forward.db "INSERT INTO keywords (rule_id, keyword, is_regex, is_blacklist) VALUES (1, '.*', 1, 0);"
|
||||
docker cp /tmp/forward.db telegram-forwarder:/app/db/forward.db
|
||||
docker restart telegram-forwarder
|
||||
```
|
||||
|
||||
**Agent-Side Filtering** via `channel_prompts` in Hermes config.yaml:
|
||||
```
|
||||
A类(交易信号) → 处理
|
||||
B类(确认/取消) → 处理
|
||||
C类(平仓) → 处理
|
||||
D类(非交易消息) → 忽略，不回复
|
||||
```
|
||||
|
||||
**Cross-platform push from channel_prompts** (`send_message` is NOT agent-callable — use `hermes send` instead):
|
||||
|
||||
The Hermes internal `send_message` tool is **intentionally NOT registered as an agent-callable model tool**. Old channel_prompts that reference `send_message` will silently fail. The correct approach is to use `terminal("hermes send -t qqbot 'message'")` in the prompt:
|
||||
|
||||
```yaml
|
||||
telegram:
|
||||
channel_prompts:
|
||||
'-1003966251111': '你是交易信号处理器。解析【币种】【方向】等字段。然后通过 terminal("hermes send -t qqbot '"'"'信号摘要+建议操作'"'"'") 推送到QQ。不要在群里回复。'
|
||||
```
|
||||
|
||||
See the `cross-platform-push` skill for full details on `hermes send` vs cronjob deliver methods.
|
||||
|
||||
### Whitelist Regex: Require Multiple Keywords (AND Logic — Only If Filtering at Forwarder Level)
|
||||
|
||||
**⚠️ Only use this approach if you need to filter at the forwarder level.** For agent-side filtering, use `.*` above.
|
||||
|
||||
To forward ONLY messages containing ALL of several keywords (e.g., 【币种】【方向】【仓位】):
|
||||
|
||||
**⚠️ CRITICAL PITFALL: Do NOT use lookahead syntax `(?=.*A)(?=.*B)(?=.*C)` with Chinese brackets `【】`!**
|
||||
|
||||
The TelegramForwarder regex engine fails silently on lookahead patterns with Chinese full-width brackets. The log shows "未匹配到普通白名单关键词" even though the keywords are clearly present. Use **sequential matching** instead:
|
||||
|
||||
```bash
|
||||
docker exec telegram-forwarder python3 -c "
|
||||
import sqlite3
|
||||
conn = sqlite3.connect('/app/db/forward.db')
|
||||
c = conn.cursor()
|
||||
|
||||
# Switch rule to WHITELIST mode
|
||||
c.execute(\"UPDATE forward_rules SET forward_mode='WHITELIST', add_mode='WHITELIST' WHERE id=1\")
|
||||
|
||||
# Clear old keywords
|
||||
c.execute('DELETE FROM keywords WHERE rule_id = 1')
|
||||
|
||||
# ✅ CORRECT: Sequential match with [\s\S]* (matches newlines)
|
||||
regex = '【币种】[\\s\\S]*【方向】[\\s\\S]*【仓位】'
|
||||
c.execute('INSERT INTO keywords (rule_id, keyword, is_regex, is_blacklist) VALUES (1, ?, 1, 0)', (regex,))
|
||||
|
||||
# Add blacklist: exclude messages with '未知'
|
||||
c.execute(\"INSERT INTO keywords (rule_id, keyword, is_regex, is_blacklist) VALUES (1, '未知', 0, 1)\")
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
\"
|
||||
docker restart telegram-forwarder
|
||||
```
|
||||
|
||||
**Key rules**:
|
||||
|
||||
| Approach | Pattern | Result |
|
||||
|----------|---------|--------|
|
||||
| ❌ Lookahead (broken) | `(?=.*【币种】)(?=.*【方向】)(?=.*【仓位】)` | 未匹配，不转发 |
|
||||
| ✅ Sequential (works) | `【币种】[\s\S]*【方向】[\s\S]*【仓位】` | 匹配成功 |
|
||||
| ❌ Dot-star (newline issue) | `【币种】.*【方向】.*【仓位】` | 不匹配跨行消息 |
|
||||
|
||||
**Why `[\s\S]*` not `.*`**: Messages have newlines between fields. `.` doesn't match `\n` by default. `[\s\S]` matches any character including newlines.
|
||||
|
||||
**Why not separate keywords**: Multiple white keywords in the `keywords` table are evaluated with **OR logic** — matching ANY one keyword triggers forwarding. To require ALL keywords, you MUST use a single regex pattern. See the filter chain source at `/app/utils/common.py` lines ~428-460.
|
||||
|
||||
**Keyword order matters**: The sequential pattern assumes keywords appear in a fixed order (e.g., 币种→方向→仓位). If the source channel always formats messages in the same order (which trading signal channels typically do), this is fine. If order varies, you'd need multiple regex patterns joined with `|` for each permutation.
|
||||
|
||||
## Adding a New Source Channel (Post-Deploy)
|
||||
|
||||
When you need to bind an additional Telegram source to an existing forwarder (same target group), here is the complete process:
|
||||
|
||||
### Step-by-Step
|
||||
|
||||
**1. Resolve the entity** (find its Telegram chat ID):
|
||||
|
||||
Run INSIDE the container while it is running (entity resolution does not need the DB):
|
||||
|
||||
```bash
|
||||
docker exec telegram-forwarder python3 -c "
|
||||
import os, asyncio, socks
|
||||
os.environ['API_ID'] = '37679203'
|
||||
os.environ['API_HASH'] = '514a2ba8b7898365c9070da4739e8deb'
|
||||
from telethon import TelegramClient
|
||||
from telethon.tl.functions.channels import JoinChannelRequest
|
||||
|
||||
async def resolve():
|
||||
client = TelegramClient('./sessions/user', int(os.environ['API_ID']), os.environ['API_HASH'],
|
||||
proxy=(socks.SOCKS5, '127.0.0.1', 7890))
|
||||
await client.start()
|
||||
entity = await client.get_entity('shipanbtc') # ← replace with target username
|
||||
print(f'type={type(entity).__name__} id={entity.id} title={getattr(entity, \"title\", \"\")}')
|
||||
await client(JoinChannelRequest(entity))
|
||||
print('Joined OK')
|
||||
await client.disconnect()
|
||||
|
||||
asyncio.run(resolve())
|
||||
"
|
||||
```
|
||||
|
||||
Output example: `type=Channel id=3432963646 title=链上监控 | 魏神 麻吉 内幕哥`
|
||||
|
||||
**⚠️ Proxy format**: Telethon needs `(socks.SOCKS5, '127.0.0.1', 7890)` — use `socks.SOCKS5`, NOT the raw number `5`.
|
||||
|
||||
**2. Stop container** (to release session DB lock):
|
||||
|
||||
```bash
|
||||
docker stop telegram-forwarder
|
||||
```
|
||||
|
||||
**⚠️ Session DB lock**: Trying to create a Telethon client that uses the same session file while the container is running fails with `database is locked`. Always stop the container before making DB changes, or resolve the entity first (which can run while the container is up).
|
||||
|
||||
**3. Add chat entry + forward rule**:
|
||||
|
||||
```bash
|
||||
docker start telegram-forwarder # needed so docker exec works
|
||||
docker exec telegram-forwarder python3 -c "
|
||||
import os, asyncio, socks, sqlite3
|
||||
os.environ['API_ID'] = '37679203'
|
||||
os.environ['API_HASH'] = '514a2ba8b7898365c9070da4739e8deb'
|
||||
from telethon import TelegramClient
|
||||
from telethon.tl.functions.channels import JoinChannelRequest
|
||||
|
||||
async def resolve_and_bind():
|
||||
client = TelegramClient('./sessions/user', int(os.environ['API_ID']), os.environ['API_HASH'],
|
||||
proxy=(socks.SOCKS5, '127.0.0.1', 7890))
|
||||
await client.start()
|
||||
|
||||
entity = await client.get_entity('shipanbtc') # ← replace
|
||||
try: await client(JoinChannelRequest(entity))
|
||||
except: pass
|
||||
entity2 = await client.get_entity('shipanbtc')
|
||||
entity_id_str = str(entity2.id)
|
||||
print(f'Entity: id={entity_id_str} title={getattr(entity2, \"title\", \"\")}')
|
||||
|
||||
# Update DB
|
||||
conn = sqlite3.connect('/app/db/forward.db')
|
||||
c = conn.cursor()
|
||||
existing = c.execute('SELECT id FROM chats WHERE telegram_chat_id=?', (entity_id_str,)).fetchone()
|
||||
if existing:
|
||||
new_chat_id = existing[0]
|
||||
print(f'Chat exists: id={new_chat_id}')
|
||||
else:
|
||||
max_id = c.execute('SELECT MAX(id) FROM chats').fetchone()[0] or 0
|
||||
new_chat_id = max_id + 1
|
||||
c.execute('INSERT INTO chats (id, telegram_chat_id, name) VALUES (?, ?, ?)',
|
||||
(new_chat_id, entity_id_str, getattr(entity2, 'title', 'New Source')))
|
||||
print(f'Inserted chat: id={new_chat_id}')
|
||||
|
||||
existing_rule = c.execute('SELECT id FROM forward_rules WHERE source_chat_id=? AND target_chat_id=2', (new_chat_id,)).fetchone()
|
||||
if existing_rule:
|
||||
print(f'Rule exists: id={existing_rule[0]}')
|
||||
else:
|
||||
c.execute('PRAGMA table_info(forward_rules)')
|
||||
cols = [r[1] for r in c.fetchall()]
|
||||
existing = dict(zip(cols, c.execute('SELECT * FROM forward_rules WHERE id=1').fetchone()))
|
||||
insert_cols = [k for k in cols if k != 'id']
|
||||
vals = [existing[k] if k != 'source_chat_id' else new_chat_id for k in insert_cols]
|
||||
col_str = ', '.join(insert_cols); ph = ', '.join(['?'] * len(vals))
|
||||
c.execute(f'INSERT INTO forward_rules ({col_str}) VALUES ({ph})', vals)
|
||||
new_rule_id = c.lastrowid
|
||||
for kw in c.execute('SELECT keyword, is_regex, is_blacklist FROM keywords WHERE rule_id=1').fetchall():
|
||||
c.execute('INSERT INTO keywords (rule_id, keyword, is_regex, is_blacklist) VALUES (?, ?, ?, ?)',
|
||||
(new_rule_id, kw[0], kw[1], kw[2]))
|
||||
print(f'Created rule id={new_rule_id}')
|
||||
|
||||
conn.commit(); conn.close()
|
||||
print('Done')
|
||||
await client.disconnect()
|
||||
|
||||
asyncio.run(resolve_and_bind())
|
||||
"
|
||||
docker stop telegram-forwarder
|
||||
```
|
||||
|
||||
**4. Restart**:
|
||||
|
||||
```bash
|
||||
docker start telegram-forwarder
|
||||
sleep 3
|
||||
docker logs telegram-forwarder --tail 5
|
||||
# Expect: "用户客户端已启动" + "机器人客户端已启动"
|
||||
```
|
||||
|
||||
**5. Verify**:
|
||||
|
||||
```bash
|
||||
docker exec telegram-forwarder python3 -c "
|
||||
import sqlite3
|
||||
conn = sqlite3.connect('/app/db/forward.db')
|
||||
c = conn.cursor()
|
||||
for r in c.execute('''SELECT fr.id, c1.name as source, c2.name as target, fr.enable_rule
|
||||
FROM forward_rules fr LEFT JOIN chats c1 ON fr.source_chat_id=c1.id
|
||||
LEFT JOIN chats c2 ON fr.target_chat_id=c2.id''').fetchall():
|
||||
print(f'{\"✅\" if r[3] else \"❌\"} Rule#{r[0]}: {r[1]} -> {r[2]}')
|
||||
conn.close()
|
||||
"
|
||||
```
|
||||
|
||||
### DB Schema (Forward Rules)
|
||||
|
||||
```sql
|
||||
chats: (id INTEGER, telegram_chat_id VARCHAR, name VARCHAR, current_add_id VARCHAR)
|
||||
forward_rules: (id INTEGER, source_chat_id INTEGER->chats.id, target_chat_id INTEGER->chats.id,
|
||||
forward_mode VARCHAR, add_mode VARCHAR, enable_rule BOOLEAN, ...)
|
||||
keywords: (id INTEGER, rule_id INTEGER->forward_rules.id, keyword TEXT,
|
||||
is_regex BOOLEAN, is_blacklist BOOLEAN)
|
||||
```
|
||||
|
||||
Key relationships:
|
||||
- `forward_rules.source_chat_id` → `chats.id` (not `telegram_chat_id` directly)
|
||||
- `forward_rules.target_chat_id` → `chats.id`
|
||||
- `keywords.rule_id` → `forward_rules.id`
|
||||
- `chats.telegram_chat_id` holds the resolved numeric entity ID (e.g. `3805472665`)
|
||||
- Public channels: positive integers, Groups/supergroups: negative (e.g. `-1003966251111`)
|
||||
|
||||
### Alternative: Direct DB Edit (Container Stopped)
|
||||
|
||||
When you already know the entity ID:
|
||||
|
||||
```bash
|
||||
docker stop telegram-forwarder
|
||||
docker cp telegram-forwarder:/app/db/forward.db /tmp/forward.db
|
||||
# Edit /tmp/forward.db with sqlite3
|
||||
docker cp /tmp/forward.db telegram-forwarder:/app/db/forward.db
|
||||
docker start telegram-forwarder
|
||||
```
|
||||
|
||||
**⚠️ Copy the DB while container is stopped**. Writing from host while container runs gives `attempt to write a readonly database` (SQLite WAL lock).
|
||||
|
||||
### Pitfalls
|
||||
|
||||
- **Session DB lock**: The forwarder keeps `sessions/user` locked while running. Any Telethon client using the same session file fails with `database is locked`. Resolve entity first (which does not need the DB), or stop the container before connecting.
|
||||
- **JoinChannelRequest required**: `get_entity` gives you the entity ID, but the user client does not receive messages until it joins via `JoinChannelRequest`. For public channels this subscribes the client.
|
||||
- **Multiple white keywords = OR logic**: Separate keyword rows are evaluated as OR. To require ALL keywords, use a single regex row with `[\\s\\S]*` between terms (sequential match, not lookahead).
|
||||
- **Proxy format**: Telethon proxy parameter is `(socks.SOCKS5, '127.0.0.1', 7890)`. Using raw `5` gives `ValueError: Unknown proxy protocol type: 5`.
|
||||
|
||||
## References
|
||||
|
||||
- Repo: https://github.com/Heavrnl/TelegramForwarder
|
||||
- Docker Hub: https://hub.docker.com/r/heavrnl/telegramforwarder
|
||||
- **Diagnostic script**: `scripts/diagnose_chain.py` — run inside container to check DB config, entity resolution, send permissions, and Hermes config in one shot.
|
||||
- **Debugging guide**: `references/debugging-guide.md` — systematic step-by-step for "信号没收到" complaints. Follow the chain: source → forwarder → group → Hermes → user.
|
||||
@@ -0,0 +1,251 @@
|
||||
# Signal Forwarding Debugging Guide
|
||||
|
||||
Systematic approach to diagnose "信号没收到" (signal not received) complaints.
|
||||
|
||||
## The Chain
|
||||
|
||||
```
|
||||
Source Channel → TelegramForwarder → Target Group → Hermes → User DM
|
||||
(1) (2) (3) (4) (5)
|
||||
```
|
||||
|
||||
Failure at ANY link breaks the chain. Debug from left to right.
|
||||
|
||||
## Step 1: Forwarder Receiving Messages?
|
||||
|
||||
```bash
|
||||
docker logs telegram-forwarder 2>&1 | grep "收到新消息.*实盘监控" | tail -5
|
||||
```
|
||||
|
||||
- **Has entries** → Forwarder is receiving from source. Go to Step 2.
|
||||
- **No entries** → Source channel not bound, or Telethon session expired. Check DB:
|
||||
```bash
|
||||
docker exec telegram-forwarder python3 -c "
|
||||
import sqlite3
|
||||
conn = sqlite3.connect('/app/db/forward.db')
|
||||
for r in conn.execute('SELECT id, telegram_chat_id, name FROM chats'): print(r)
|
||||
conn.close()"
|
||||
```
|
||||
|
||||
## Step 2: Keyword Matching?
|
||||
|
||||
```bash
|
||||
docker logs telegram-forwarder 2>&1 | grep -E "正则匹配成功|未匹配" | tail -5
|
||||
```
|
||||
|
||||
- `正则匹配成功` → Match succeeded. Go to Step 3.
|
||||
- `未匹配到普通白名单关键词` → Signal format changed, or regex doesn't match.
|
||||
- Check the actual signal text: `docker logs telegram-forwarder 2>&1 | grep "内容:" | tail -5`
|
||||
- Common issue: signal uses different brackets (e.g., `[币种]` vs `【币种】`)
|
||||
- Common issue: multi-line signals — `.*` doesn't match `\n`, use `[\s\S]*` instead
|
||||
|
||||
## Step 3: Forwarder Sending to Group?
|
||||
|
||||
```bash
|
||||
docker logs telegram-forwarder 2>&1 | grep -E "已发送|成功获取目标" | tail -5
|
||||
```
|
||||
|
||||
- `消息已发送到: XXX` → Forward succeeded. Go to Step 4.
|
||||
- No send logs after match → Check for errors:
|
||||
```bash
|
||||
docker logs telegram-forwarder 2>&1 | grep -iE "error|invalid Peer|banned|forbidden" | tail -5
|
||||
```
|
||||
- `An invalid Peer was used` → Group migrated to supergroup, ID changed
|
||||
- `Bots can't send messages` → Trying to send to a bot, not a group
|
||||
|
||||
## Step 4: Hermes Receiving from Group?
|
||||
|
||||
**FIRST: Check for competing gateway processes** (most common hidden cause):
|
||||
```bash
|
||||
ps aux | grep "gateway\|openclaw" | grep -v grep
|
||||
```
|
||||
If you see an old `openclaw ... gateway` or duplicate `hermes gateway` process → **KILL IT FIRST**. Two processes sharing one bot token = permanent polling failure. See "Polling Conflict — Scenario A" below.
|
||||
|
||||
Then check polling status:
|
||||
```bash
|
||||
# Check for polling conflict
|
||||
journalctl --user -u hermes-gateway --since "10 min ago" --no-pager | grep -i "conflict"
|
||||
# If found → see "Polling Conflict" section below
|
||||
|
||||
# Check if "✓ telegram connected" appears in logs
|
||||
journalctl --user -u hermes-gateway --since "5 min ago" --no-pager | grep "✓ telegram"
|
||||
# If NOT found → adapter didn't connect (check for competing processes or conflict)
|
||||
|
||||
# Check gateway.log for inbound messages from the group
|
||||
grep "inbound message" ~/.hermes/logs/gateway.log | grep "1003966251111" | tail -5
|
||||
```
|
||||
|
||||
- **Polling conflict found** → Stop gateway, wait 30s, start fresh (see "Polling Conflict" section)
|
||||
- **Has entries** → Hermes receives group messages. Go to Step 5.
|
||||
- **No entries, no conflict** → Hermes can't see group messages. Check:
|
||||
1. Bot privacy mode: `curl -s --proxy http://127.0.0.1:7890 "https://api.telegram.org/bot<TOKEN>/getMe" | grep can_read_all_group_messages`
|
||||
- Must be `true`. If `false`, disable in @BotFather and **kick+re-add** bot to group.
|
||||
2. `free_response_chats` config (NOT `free_response_channels`):
|
||||
```bash
|
||||
grep -A5 "extra:" ~/.hermes/config.yaml | grep free_response
|
||||
```
|
||||
Must be under `telegram.extra.free_response_chats`, not top-level.
|
||||
3. Check if bot is actually in the group.
|
||||
|
||||
## Step 5: Hermes Processing and Responding?
|
||||
|
||||
```bash
|
||||
grep "1003966251111" ~/.hermes/logs/gateway.log | grep -E "response ready|Suppressing" | tail -5
|
||||
```
|
||||
|
||||
- `response ready` → Hermes processed and responded. Check delivery.
|
||||
- `inbound` but no `response` → Agent failed or timed out. Check errors.log:
|
||||
```bash
|
||||
tail -20 ~/.hermes/logs/errors.log
|
||||
```
|
||||
|
||||
## Quick One-Liner Diagnostic
|
||||
|
||||
```bash
|
||||
echo "=== Forwarder ===" && \
|
||||
docker logs telegram-forwarder 2>&1 | grep -cE "正则匹配成功|已发送" && \
|
||||
echo "=== Hermes ===" && \
|
||||
grep -c "inbound.*1003966251111" ~/.hermes/logs/gateway.log 2>/dev/null && \
|
||||
echo "=== Errors ===" && \
|
||||
tail -3 ~/.hermes/logs/errors.log 2>/dev/null
|
||||
```
|
||||
|
||||
## Log Pattern Reference
|
||||
|
||||
| Pattern | Meaning | Where |
|
||||
|---------|---------|-------|
|
||||
| `收到新消息 来自聊天: XXX` | Forwarder received message | forwarder logs |
|
||||
| `正则匹配成功` | Keyword regex matched | forwarder logs |
|
||||
| `未匹配到普通白名单关键词` | Keyword didn't match | forwarder logs |
|
||||
| `所有白名单条件都满足，允许转发` | Forwarding allowed | forwarder logs |
|
||||
| `消息已发送到: XXX` | Message sent to target | forwarder logs |
|
||||
| `过滤器识别到机器人消息，忽略处理` | Bot message filtered (normal) | forwarder logs |
|
||||
| `inbound message: platform=telegram chat=XXX` | Hermes received message | gateway.log |
|
||||
| `response ready: platform=telegram chat=XXX` | Hermes responded | gateway.log |
|
||||
| `Suppressing normal final send` | Response already delivered via stream | gateway.log |
|
||||
|
||||
## Common Scenarios
|
||||
|
||||
### "昨天能收到，今天收不到"
|
||||
- Check if source channel stopped sending (forwarder logs)
|
||||
- Check if Telethon session expired (container restart fixes this)
|
||||
- Check if signal format changed
|
||||
|
||||
### "有信号进来但没收到你的消息"
|
||||
- Could be timing: signal hasn't arrived yet, or arrived but user expected Hermes response
|
||||
- Quick check: `docker logs telegram-forwarder 2>&1 | grep "收到新消息.*实盘监控" | tail -1` — compare timestamp to current time. If >30min stale, the signal may not have reached the forwarder.
|
||||
- If forwarder received and forwarded but no Hermes response: check `tail -20 ~/.hermes/logs/gateway.log | grep inbound` — if no recent group messages, the bot may not be receiving (privacy mode).
|
||||
- **Bot message filtering**: Hermes may ignore messages sent by other bots. Always test with a human account, not the forwarder bot.
|
||||
- **⚠️ Check for competing gateway processes**: `ps aux | grep "gateway\|openclaw" | grep -v grep` — if an old OpenClaw Gateway or duplicate Hermes process is running, it steals the bot token polling session and Hermes NEVER receives any group messages. This was the actual root cause for a user whose group messages had NEVER worked for over a week.
|
||||
|
||||
### "信号在频道里能看到，但没转发到群"
|
||||
- Step 2 failure: keyword mismatch
|
||||
- Check actual signal text vs regex pattern
|
||||
|
||||
### "信号转发到群了，但Hermes没回复"
|
||||
- Step 4 failure: privacy mode or config
|
||||
- Most common: bot privacy mode is ON
|
||||
|
||||
### "重启了 gateway 还是不处理群消息"（Polling Conflict — 两种原因）
|
||||
|
||||
**This is the #1 hidden cause.** There are TWO distinct polling conflict scenarios:
|
||||
|
||||
#### Scenario A: Competing gateway processes (most dangerous)
|
||||
|
||||
Another process is using the same bot token for `getUpdates`. This causes PERMANENT failure — no amount of restarting Hermes will help until the other process is killed.
|
||||
|
||||
**Diagnosis**:
|
||||
```bash
|
||||
# 1. Check for competing processes
|
||||
ps aux | grep "gateway\|openclaw" | grep -v grep
|
||||
# Look for: node .../openclaw/dist/index.js gateway --port 18789
|
||||
# Or: another hermes gateway run
|
||||
|
||||
# 2. Test getUpdates directly (bypass adapter)
|
||||
python3 -c "
|
||||
import json, urllib.request
|
||||
with open('/home/openclaw/.hermes/.env') as f:
|
||||
for line in f:
|
||||
if line.startswith('TELEGRAM_BOT_TOKEN'):
|
||||
token = line.strip().split('=',1)[1]
|
||||
break
|
||||
proxy = urllib.request.ProxyHandler({'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'})
|
||||
opener = urllib.request.build_opener(proxy)
|
||||
try:
|
||||
req = urllib.request.Request(f'https://api.telegram.org/bot{token}/getUpdates?timeout=1')
|
||||
resp = opener.open(req, timeout=5)
|
||||
data = json.loads(resp.read())
|
||||
print(f'getUpdates: {len(data.get(\"result\", []))} updates (no conflict = nobody polling)')
|
||||
except Exception as e:
|
||||
if '409' in str(e):
|
||||
print('409 Conflict — adapter IS polling (normal)')
|
||||
else:
|
||||
print(f'Error: {e}')
|
||||
"
|
||||
```
|
||||
|
||||
**Key diagnostic signal**:
|
||||
- `getUpdates` returns 0 without 409 conflict → **NOBODY is polling** → competing process consumed and dropped updates
|
||||
- `getUpdates` returns 409 → adapter IS polling (conflict is expected, this is normal)
|
||||
- `getUpdates` returns updates → adapter is NOT polling AND competing process dropped off → restart Hermes
|
||||
|
||||
**Fix**: Kill the stale process FIRST, then restart Hermes:
|
||||
```bash
|
||||
kill <stale_pid>
|
||||
systemctl --user restart hermes-gateway
|
||||
```
|
||||
|
||||
**Verify**: `journalctl --user -u hermes-gateway --since "1 min ago" | grep "✓ telegram connected"` — must appear after restart.
|
||||
|
||||
#### Scenario B: `hermes gateway restart` race condition
|
||||
|
||||
The restart command sends SIGTERM and immediately starts a new process. Telegram's servers keep the old polling session alive for ~30 seconds. The new process tries to poll → conflict → retry loop.
|
||||
|
||||
**Diagnosis**:
|
||||
```bash
|
||||
journalctl --user -u hermes-gateway --since "5 min ago" --no-pager | grep -i "conflict"
|
||||
```
|
||||
|
||||
**Fix**:
|
||||
```bash
|
||||
systemctl --user stop hermes-gateway
|
||||
sleep 30 # critical: wait for Telegram session to expire
|
||||
systemctl --user start hermes-gateway
|
||||
```
|
||||
|
||||
#### How to tell which scenario
|
||||
|
||||
```bash
|
||||
# Check how long the gateway has been running
|
||||
systemctl --user status hermes-gateway | grep "Active:"
|
||||
# If running for days/weeks but group messages NEVER worked → Scenario A
|
||||
# If just restarted and immediately getting conflicts → Scenario B
|
||||
|
||||
# Check for stale processes
|
||||
ps aux | grep "gateway" | grep -v grep | grep -v $(pgrep -f "hermes_cli.main gateway")
|
||||
# If you see old processes → Scenario A
|
||||
```
|
||||
|
||||
## Gateway Logs (Binary Format — Use strings!)
|
||||
|
||||
Gateway logs at `~/.hermes/logs/gateway.log` are **binary format**. Use `strings` to extract:
|
||||
|
||||
```bash
|
||||
strings ~/.hermes/logs/gateway.log | tail -30
|
||||
strings ~/.hermes/logs/gateway.log | grep "inbound message" | tail -10
|
||||
strings ~/.hermes/logs/gateway.log | grep "<group_chat_id>" | tail -10
|
||||
strings ~/.hermes/logs/gateway.log | grep "Connected to"
|
||||
```
|
||||
|
||||
## ⚠️ Testing: Use Human Account
|
||||
|
||||
**Never test with bot-sent messages.** Bots cannot see their own messages via getUpdates. Always use a human Telegram account to send test messages to the group or source channel.
|
||||
|
||||
## Agent-Side Filtering Architecture
|
||||
|
||||
When the user says "收所有的消息，在处理消息这边来处理规则过滤":
|
||||
1. Forwarder: `.*` whitelist (pass everything)
|
||||
2. Agent channel_prompts: classify messages as A(signal)/B(confirm)/C(close)/D(ignore)
|
||||
3. Only process A/B/C, silently ignore D
|
||||
|
||||
Benefits: source format changes only require updating the agent prompt, not the forwarder database.
|
||||
@@ -0,0 +1,36 @@
|
||||
# Diagnostic: check if a Telegram group has been migrated to a supergroup
|
||||
# Usage: docker exec telegram-forwarder python3 <this-script>
|
||||
#
|
||||
# When a group migrates to a supergroup, the old ID becomes deactivated.
|
||||
# The new ID is the channel_id from migrated_to, prefixed with -100.
|
||||
|
||||
from telethon import TelegramClient
|
||||
import asyncio, socks
|
||||
|
||||
async def check_migration(chat_id: int):
|
||||
proxy = (socks.SOCKS5, '127.0.0.1', 7890)
|
||||
# Use bot client for checking
|
||||
bot = TelegramClient('./sessions/bot', API_ID, API_HASH, proxy=proxy)
|
||||
await bot.start()
|
||||
|
||||
try:
|
||||
entity = await bot.get_entity(chat_id)
|
||||
print(f'Entity type: {type(entity).__name__}')
|
||||
print(f'ID: {entity.id}')
|
||||
print(f'Title: {getattr(entity, "title", "N/A")}')
|
||||
if hasattr(entity, 'deactivated') and entity.deactivated:
|
||||
print(f'⚠️ DEACTIVATED - group was migrated!')
|
||||
if hasattr(entity, 'migrated_to') and entity.migrated_to:
|
||||
new_id = entity.migrated_to.channel_id
|
||||
print(f'New channel_id: {new_id}')
|
||||
print(f'New Telethon ID: -100{new_id}')
|
||||
print(f'Hermes free_response_channels should be: "-100{new_id}"')
|
||||
else:
|
||||
print('✅ Active - no migration detected')
|
||||
except Exception as e:
|
||||
print(f'Error: {e}')
|
||||
|
||||
await bot.disconnect()
|
||||
|
||||
# Example: check old group ID
|
||||
# asyncio.run(check_migration(5586190970))
|
||||
@@ -0,0 +1,145 @@
|
||||
# Hermes + TelegramForwarder Group Integration
|
||||
|
||||
## Problem
|
||||
Forwarder sends messages to a group, but Hermes bot (also in the group) ignores them.
|
||||
|
||||
## Root Causes (check ALL of these)
|
||||
|
||||
| # | Cause | Symptom | Fix |
|
||||
|---|-------|---------|-----|
|
||||
| 0 | **Competing gateway process** (old OpenClaw, duplicate Hermes) | Adapter shows "connected" but `getUpdates` returns 0 without 409 conflict. No "✓ telegram connected" in logs. Group messages NEVER worked. | `ps aux \| grep gateway` → kill stale process → restart Hermes |
|
||||
| 1 | Bot Privacy Mode is ON | Bot receives 0 group messages | @BotFather → /mybots → Bot Settings → Group Privacy → Turn off |
|
||||
| 2 | Wrong config key `free_response_channels` | Bot sees @mentions but ignores regular messages | Use `extra.free_response_chats` (NOT top-level `free_response_channels`) |
|
||||
| 3 | Group migrated to supergroup | `An invalid Peer was used` in forwarder logs | Update chat ID (see below) |
|
||||
| 4 | Privacy mode not propagated | `can_read_all_group_messages=true` but bot still can't see messages | Kick bot from group, re-add it |
|
||||
|
||||
**⚠️ Check #0 FIRST.** A stale gateway process is invisible — the state file says "connected", no errors appear, but the adapter never actually polls. This was the actual root cause for a user whose group messages had NEVER worked for over a week (old OpenClaw Gateway PID 900 running since June 22). Always run `ps aux | grep gateway | grep -v grep` before investigating other causes.
|
||||
|
||||
**⚠️ Privacy mode is the #1 cause.** Even with `free_response_chats` correctly configured, if privacy mode is ON, the bot literally cannot see group messages. Fix privacy mode FIRST.
|
||||
|
||||
## Step-by-step
|
||||
|
||||
### 0. Check for Competing Gateway Processes (DO THIS FIRST)
|
||||
```bash
|
||||
ps aux | grep "gateway\|openclaw" | grep -v grep
|
||||
```
|
||||
If you see any `openclaw ... gateway` or duplicate `hermes gateway` processes → **kill them immediately**. They steal the bot token polling session.
|
||||
|
||||
Verify nobody else is polling:
|
||||
```bash
|
||||
python3 -c "
|
||||
import json, urllib.request
|
||||
with open('/home/openclaw/.hermes/.env') as f:
|
||||
for line in f:
|
||||
if line.startswith('TELEGRAM_BOT_TOKEN'):
|
||||
token = line.strip().split('=',1)[1]
|
||||
break
|
||||
proxy = urllib.request.ProxyHandler({'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'})
|
||||
opener = urllib.request.build_opener(proxy)
|
||||
try:
|
||||
req = urllib.request.Request(f'https://api.telegram.org/bot{token}/getUpdates?timeout=1')
|
||||
resp = opener.open(req, timeout=5)
|
||||
print(f'getUpdates: {len(json.loads(resp.read()).get(\"result\",[]))} updates — nobody polling!' )
|
||||
except Exception as e:
|
||||
print(f'409 Conflict — adapter IS polling (normal)' if '409' in str(e) else f'Error: {e}')
|
||||
"
|
||||
```
|
||||
- 0 updates without 409 → broken (check for competing processes)
|
||||
- 409 Conflict → normal (adapter is polling)
|
||||
|
||||
### 1. Disable Bot Privacy Mode (MANDATORY)
|
||||
Open Telegram → @BotFather:
|
||||
```
|
||||
/mybots → select Hermes bot → Bot Settings → Group Privacy → Turn off
|
||||
```
|
||||
Do this for the Hermes bot. If the forwarder bot also needs to read messages in the group, do it for that bot too.
|
||||
|
||||
**After disabling, kick the bot from the group and re-add it.** The change does not apply to bots already in a group.
|
||||
|
||||
Verify: `curl -s --proxy http://127.0.0.1:7890 "https://api.telegram.org/bot<TOKEN>/getMe"` → `can_read_all_group_messages` must be `true`.
|
||||
|
||||
### 2. Find the destination group's chat_id
|
||||
```bash
|
||||
cd ~/TelegramForwarder
|
||||
docker exec telegram-forwarder python3 -c "
|
||||
import sqlite3
|
||||
conn = sqlite3.connect('/app/db/forward.db')
|
||||
c = conn.cursor()
|
||||
c.execute('SELECT id, telegram_chat_id, name FROM chats')
|
||||
for ch in c.fetchall():
|
||||
print(ch)
|
||||
conn.close()
|
||||
"
|
||||
```
|
||||
The column is `telegram_chat_id` (NOT `chat_id`).
|
||||
|
||||
**⚠️ Supergroup IDs**: If the group was migrated, the old ID is stale. Use `-100` + `channel_id` format. To find the real channel_id:
|
||||
```python
|
||||
entity = await bot.get_entity(old_id)
|
||||
# If entity has migrated_to:
|
||||
new_id = -100 * entity.migrated_to.channel_id # e.g., -1003966251111
|
||||
```
|
||||
|
||||
### 3. Edit Hermes config (CORRECT KEY!)
|
||||
`~/.hermes/config.yaml` — add inside the `telegram:` → `extra:` section:
|
||||
```yaml
|
||||
telegram:
|
||||
extra:
|
||||
free_response_chats: '-1003966251111' # ← CORRECT key
|
||||
# free_response_channels: '...' ← WRONG — code ignores this!
|
||||
```
|
||||
|
||||
**⚠️ CRITICAL**: The Hermes source code reads `self.config.extra.get("free_response_chats")`. The top-level `free_response_channels` is a DIFFERENT field that is NOT read by the group-message processing logic. Using the wrong key = silently ignored, bot only responds to @mentions.
|
||||
|
||||
Multiple groups: comma-separated `'-1003966251111,-1001234567890'`
|
||||
|
||||
### 4. Restart Hermes
|
||||
```bash
|
||||
hermes gateway restart # from a separate shell, NOT inside the gateway
|
||||
```
|
||||
If refused ("Refusing to restart from inside the gateway"), ask the user to SSH in and run it.
|
||||
|
||||
## Config Field Reference
|
||||
| Field | Location | Default | Purpose |
|
||||
|-------|----------|---------|---------|
|
||||
| `require_mention` | `telegram:` | `true` | If true, bot only responds when @mentioned in groups |
|
||||
| `free_response_chats` | `telegram:extra:` | `''` | Comma-separated chat_ids exempt from @mention requirement |
|
||||
| `free_response_channels` | `telegram:` | `''` | **NOT used by group message processing** — legacy field, ignore |
|
||||
| `allowed_chats` | `telegram:` | `''` | If set, ONLY these chats can interact with the bot (whitelist) |
|
||||
|
||||
## Signal Forwarding with channel_prompts
|
||||
|
||||
When you want Hermes to process group messages (e.g., trading signals) and forward results to a user DM instead of replying in the group:
|
||||
|
||||
```yaml
|
||||
telegram:
|
||||
channel_prompts:
|
||||
"-1003966251111": "你是交易信号处理器。收到群里的交易信号后，解析【币种】【方向】【仓位】，然后用send_message工具发送到telegram:656685714（用户DM），格式简洁：信号摘要+当前价格+建议操作+风险提示。不要在群里回复。"
|
||||
```
|
||||
|
||||
This configures a per-group system prompt. The agent in that group session will follow these instructions instead of the default behavior.
|
||||
|
||||
## End-to-End Verification
|
||||
|
||||
After setting up the forwarding → Hermes chain, verify each link:
|
||||
|
||||
1. **Forwarder listening**: `docker logs telegram-forwarder 2>&1 | grep "收到新消息"` — should show messages from source channel
|
||||
2. **Forwarder sending**: `docker logs telegram-forwarder 2>&1 | grep "已发送"` — should show successful sends to target group
|
||||
3. **Hermes receiving**: `hermes logs 2>&1 | grep "inbound message"` — should show messages from the group (chat ID should match the group, not the user DM)
|
||||
4. **Hermes responding**: Check the group for Hermes's reply
|
||||
|
||||
**Common failure chain**:
|
||||
- Forwarder logs show "收到新消息" but no "已发送" → keyword filter issue or invalid Peer (check group ID)
|
||||
- Forwarder logs show "已发送" but Hermes shows no inbound from group → privacy mode still on, or `free_response_chats` wrong key
|
||||
- Hermes shows inbound but no response → check `require_mention` and `free_response_chats`
|
||||
|
||||
**⚠️ For testing, use a human account** (not the forwarder bot) to send messages to the source channel or directly to the group. Bot-sent messages may be filtered by Hermes.
|
||||
|
||||
## Pitfalls
|
||||
- `free_response_chats` takes string values, not integers: `'-1003966251111'` not `-1003966251111`
|
||||
- Config changes require Hermes restart to take effect
|
||||
- If `allowed_chats` is set (non-empty), the group must also be in that list
|
||||
- The forwarder's `handle_mode: FORWARD` uses Telegram's native forward (shows "Forwarded from"). If user wants clean copies, change to `COPY` mode.
|
||||
- **Don't test by sending directly to the target group** — this bypasses the forwarding chain and proves nothing. Test by sending to the SOURCE group and verifying the full pipeline: source → forwarder → target → Hermes response.
|
||||
- **Bot-to-bot messaging FAILS**: Telegram prohibits bot→bot messaging. Forward to a GROUP where both bots reside, not directly to a bot.
|
||||
- **Session database locked**: If you see `sqlite3.OperationalError: database is locked`, stop the container first, edit DB, then restart.
|
||||
@@ -0,0 +1,200 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Diagnose TelegramForwarder → Hermes forwarding chain.
|
||||
|
||||
Run inside the container:
|
||||
docker exec telegram-forwarder python3 /app/scripts/diagnose_chain.py
|
||||
|
||||
Or from host (container must be stopped):
|
||||
cd ~/TelegramForwarder && docker run --rm -v $(pwd)/sessions:/app/sessions \
|
||||
-v $(pwd)/db:/app/db --network host \
|
||||
-e HTTP_PROXY=http://127.0.0.1:7890 \
|
||||
telegramforwarder-telegram-forwarder python3 /app/scripts/diagnose_chain.py
|
||||
"""
|
||||
|
||||
import sqlite3
|
||||
import asyncio
|
||||
import socks
|
||||
|
||||
DB_PATH = '/app/db/forward.db'
|
||||
|
||||
def check_db():
|
||||
"""Check forwarder database configuration."""
|
||||
print("=" * 50)
|
||||
print("1. DATABASE CONFIG")
|
||||
print("=" * 50)
|
||||
conn = sqlite3.connect(DB_PATH)
|
||||
c = conn.cursor()
|
||||
|
||||
print("\n--- Chats ---")
|
||||
for row in c.execute('SELECT id, telegram_chat_id, name FROM chats'):
|
||||
print(f" {row}")
|
||||
|
||||
print("\n--- Forward Rules ---")
|
||||
for row in c.execute('SELECT id, source_chat_id, target_chat_id, forward_mode, enable_rule FROM forward_rules'):
|
||||
print(f" Rule {row[0]}: source={row[1]} → target={row[2]} | mode={row[3]} | enabled={row[4]}")
|
||||
|
||||
print("\n--- Keywords ---")
|
||||
for row in c.execute('SELECT * FROM keywords'):
|
||||
print(f" {row}")
|
||||
|
||||
conn.close()
|
||||
|
||||
async def check_entity_resolution():
|
||||
"""Test if bot can resolve the target group entity."""
|
||||
print("\n" + "=" * 50)
|
||||
print("2. ENTITY RESOLUTION TEST")
|
||||
print("=" * 50)
|
||||
|
||||
try:
|
||||
from telethon import TelegramClient
|
||||
except ImportError:
|
||||
print(" SKIP: telethon not available")
|
||||
return
|
||||
|
||||
import os
|
||||
api_id = os.getenv('API_ID')
|
||||
api_hash = os.getenv('API_HASH')
|
||||
if not api_id or not api_hash:
|
||||
print(" SKIP: API_ID/API_HASH not set")
|
||||
return
|
||||
|
||||
proxy = (socks.SOCKS5, '127.0.0.1', 7890)
|
||||
bot = TelegramClient('./sessions/bot', int(api_id), api_hash, proxy=proxy)
|
||||
await bot.start()
|
||||
|
||||
conn = sqlite3.connect(DB_PATH)
|
||||
c = conn.cursor()
|
||||
targets = c.execute('SELECT id, telegram_chat_id, name FROM chats').fetchall()
|
||||
conn.close()
|
||||
|
||||
for chat_id, tg_id, name in targets:
|
||||
try:
|
||||
entity = await bot.get_entity(int(tg_id) if not tg_id.startswith('-') else tg_id)
|
||||
deactivated = getattr(entity, 'deactivated', False)
|
||||
migrated = getattr(entity, 'migrated_to', None)
|
||||
if deactivated and migrated:
|
||||
new_id = f"-100{migrated.channel_id}"
|
||||
print(f" ❌ {name} ({tg_id}): DEACTIVATED → migrated to {new_id}")
|
||||
else:
|
||||
print(f" ✅ {name} ({tg_id}): OK - {type(entity).__name__}")
|
||||
except Exception as e:
|
||||
print(f" ❌ {name} ({tg_id}): {e}")
|
||||
|
||||
await bot.disconnect()
|
||||
|
||||
async def check_send_permission():
|
||||
"""Test if bot can send to the target group."""
|
||||
print("\n" + "=" * 50)
|
||||
print("3. SEND PERMISSION TEST")
|
||||
print("=" * 50)
|
||||
|
||||
try:
|
||||
from telethon import TelegramClient
|
||||
except ImportError:
|
||||
print(" SKIP: telethon not available")
|
||||
return
|
||||
|
||||
import os
|
||||
api_id = os.getenv('API_ID')
|
||||
api_hash = os.getenv('API_HASH')
|
||||
if not api_id or not api_hash:
|
||||
print(" SKIP: API_ID/API_HASH not set")
|
||||
return
|
||||
|
||||
proxy = (socks.SOCKS5, '127.0.0.1', 7890)
|
||||
bot = TelegramClient('./sessions/bot', int(api_id), api_hash, proxy=proxy)
|
||||
await bot.start()
|
||||
|
||||
conn = sqlite3.connect(DB_PATH)
|
||||
c = conn.cursor()
|
||||
rules = c.execute('SELECT id, target_chat_id FROM forward_rules WHERE enable_rule=1').fetchall()
|
||||
conn.close()
|
||||
|
||||
for rule_id, target_id in rules:
|
||||
conn = sqlite3.connect(DB_PATH)
|
||||
c = conn.cursor()
|
||||
tg_id = c.execute('SELECT telegram_chat_id FROM chats WHERE id=?', (target_id,)).fetchone()
|
||||
conn.close()
|
||||
if not tg_id:
|
||||
print(f" ❌ Rule {rule_id}: target chat {target_id} not found in chats table")
|
||||
continue
|
||||
|
||||
tg_id = tg_id[0]
|
||||
try:
|
||||
await bot.send_message(int(tg_id) if not tg_id.startswith('-') else tg_id, '🤖 Forward chain diagnostic test')
|
||||
print(f" ✅ Rule {rule_id}: can send to {tg_id}")
|
||||
except Exception as e:
|
||||
print(f" ❌ Rule {rule_id}: cannot send to {tg_id}: {e}")
|
||||
|
||||
await bot.disconnect()
|
||||
|
||||
def check_hermes_config():
|
||||
"""Check if Hermes free_response_chats matches."""
|
||||
print("\n" + "=" * 50)
|
||||
print("4. HERMES CONFIG CHECK")
|
||||
print("=" * 50)
|
||||
|
||||
import os
|
||||
config_path = os.path.expanduser('~/.hermes/config.yaml')
|
||||
if not os.path.exists(config_path):
|
||||
print(" SKIP: ~/.hermes/config.yaml not found")
|
||||
return
|
||||
|
||||
with open(config_path) as f:
|
||||
content = f.read()
|
||||
|
||||
# Extract telegram section
|
||||
in_telegram = False
|
||||
in_extra = False
|
||||
found_correct_key = False
|
||||
found_wrong_key = False
|
||||
|
||||
for line in content.split('\n'):
|
||||
stripped = line.strip()
|
||||
if stripped == 'telegram:':
|
||||
in_telegram = True
|
||||
continue
|
||||
if in_telegram and line and not line.startswith(' ') and not line.startswith('\t'):
|
||||
in_telegram = False
|
||||
in_extra = False
|
||||
if in_telegram:
|
||||
if stripped == 'extra:':
|
||||
in_extra = True
|
||||
continue
|
||||
if in_extra and not line.startswith(' ') and not line.startswith('\t') and stripped:
|
||||
in_extra = False
|
||||
|
||||
# CORRECT key: telegram.extra.free_response_chats
|
||||
if in_extra and 'free_response_chats' in line:
|
||||
val = line.split(':')[-1].strip().strip("'\"")
|
||||
found_correct_key = True
|
||||
if val:
|
||||
print(f" ✅ free_response_chats = {val}")
|
||||
else:
|
||||
print(f" ❌ free_response_chats is EMPTY — Hermes will ignore group messages")
|
||||
|
||||
# WRONG key: telegram.free_response_channels (legacy, ignored by code)
|
||||
if not in_extra and 'free_response_channels' in line:
|
||||
val = line.split(':')[-1].strip().strip("'\"")
|
||||
found_wrong_key = True
|
||||
if val:
|
||||
print(f" ⚠️ free_response_channels = {val} (WRONG KEY — ignored by Hermes code!)")
|
||||
else:
|
||||
print(f" ⚠️ free_response_channels is empty (also wrong key)")
|
||||
|
||||
if not found_correct_key:
|
||||
print(f" ❌ free_response_chats NOT FOUND in telegram.extra — add it!")
|
||||
if found_wrong_key and not found_correct_key:
|
||||
print(f" 💡 Move free_response_channels value to telegram.extra.free_response_chats")
|
||||
|
||||
async def main():
|
||||
check_db()
|
||||
await check_entity_resolution()
|
||||
await check_send_permission()
|
||||
check_hermes_config()
|
||||
print("\n" + "=" * 50)
|
||||
print("DIAGNOSIS COMPLETE")
|
||||
print("=" * 50)
|
||||
|
||||
if __name__ == '__main__':
|
||||
asyncio.run(main())
|
||||
Reference in New Issue
Block a user