Initial commit: Hermes Agent skills collection

- Trading skills (OKX, dividend, lottery, quantitative) - Creative skills (ASCII art, diagrams, video) - Development skills (GitHub, debugging, TDD) - Research skills (arXiv, blog monitoring) - Productivity skills (email, documents, notes) - MCP integration skills - Custom user skills
2026-07-05 02:31:15 -04:00
commit 6770bc9b9d
908 changed files with 239614 additions and 0 deletions
@@ -0,0 +1,251 @@
+# Signal Forwarding Debugging Guide
+
+Systematic approach to diagnose "信号没收到" (signal not received) complaints.
+
+## The Chain
+
+```
+Source Channel → TelegramForwarder → Target Group → Hermes → User DM
+     (1)              (2)               (3)          (4)       (5)
+```
+
+Failure at ANY link breaks the chain. Debug from left to right.
+
+## Step 1: Forwarder Receiving Messages?
+
+```bash
+docker logs telegram-forwarder 2>&1 | grep "收到新消息.*实盘监控" | tail -5
+```
+
+- **Has entries** → Forwarder is receiving from source. Go to Step 2.
+- **No entries** → Source channel not bound, or Telethon session expired. Check DB:
+  ```bash
+  docker exec telegram-forwarder python3 -c "
+  import sqlite3
+  conn = sqlite3.connect('/app/db/forward.db')
+  for r in conn.execute('SELECT id, telegram_chat_id, name FROM chats'): print(r)
+  conn.close()"
+  ```
+
+## Step 2: Keyword Matching?
+
+```bash
+docker logs telegram-forwarder 2>&1 | grep -E "正则匹配成功|未匹配" | tail -5
+```
+
+- `正则匹配成功` → Match succeeded. Go to Step 3.
+- `未匹配到普通白名单关键词` → Signal format changed, or regex doesn't match.
+  - Check the actual signal text: `docker logs telegram-forwarder 2>&1 | grep "内容:" | tail -5`
+  - Common issue: signal uses different brackets (e.g., `[币种]` vs `【币种】`)
+  - Common issue: multi-line signals — `.*` doesn't match `\n`, use `[\s\S]*` instead
+
+## Step 3: Forwarder Sending to Group?
+
+```bash
+docker logs telegram-forwarder 2>&1 | grep -E "已发送|成功获取目标" | tail -5
+```
+
+- `消息已发送到: XXX` → Forward succeeded. Go to Step 4.
+- No send logs after match → Check for errors:
+  ```bash
+  docker logs telegram-forwarder 2>&1 | grep -iE "error|invalid Peer|banned|forbidden" | tail -5
+  ```
+  - `An invalid Peer was used` → Group migrated to supergroup, ID changed
+  - `Bots can't send messages` → Trying to send to a bot, not a group
+
+## Step 4: Hermes Receiving from Group?
+
+**FIRST: Check for competing gateway processes** (most common hidden cause):
+```bash
+ps aux | grep "gateway\|openclaw" | grep -v grep
+```
+If you see an old `openclaw ... gateway` or duplicate `hermes gateway` process → **KILL IT FIRST**. Two processes sharing one bot token = permanent polling failure. See "Polling Conflict — Scenario A" below.
+
+Then check polling status:
+```bash
+# Check for polling conflict
+journalctl --user -u hermes-gateway --since "10 min ago" --no-pager | grep -i "conflict"
+# If found → see "Polling Conflict" section below
+
+# Check if "✓ telegram connected" appears in logs
+journalctl --user -u hermes-gateway --since "5 min ago" --no-pager | grep "✓ telegram"
+# If NOT found → adapter didn't connect (check for competing processes or conflict)
+
+# Check gateway.log for inbound messages from the group
+grep "inbound message" ~/.hermes/logs/gateway.log | grep "1003966251111" | tail -5
+```
+
+- **Polling conflict found** → Stop gateway, wait 30s, start fresh (see "Polling Conflict" section)
+- **Has entries** → Hermes receives group messages. Go to Step 5.
+- **No entries, no conflict** → Hermes can't see group messages. Check:
+  1. Bot privacy mode: `curl -s --proxy http://127.0.0.1:7890 "https://api.telegram.org/bot<TOKEN>/getMe" | grep can_read_all_group_messages`
+     - Must be `true`. If `false`, disable in @BotFather and **kick+re-add** bot to group.
+  2. `free_response_chats` config (NOT `free_response_channels`):
+     ```bash
+     grep -A5 "extra:" ~/.hermes/config.yaml | grep free_response
+     ```
+     Must be under `telegram.extra.free_response_chats`, not top-level.
+  3. Check if bot is actually in the group.
+
+## Step 5: Hermes Processing and Responding?
+
+```bash
+grep "1003966251111" ~/.hermes/logs/gateway.log | grep -E "response ready|Suppressing" | tail -5
+```
+
+- `response ready` → Hermes processed and responded. Check delivery.
+- `inbound` but no `response` → Agent failed or timed out. Check errors.log:
+  ```bash
+  tail -20 ~/.hermes/logs/errors.log
+  ```
+
+## Quick One-Liner Diagnostic
+
+```bash
+echo "=== Forwarder ===" && \
+docker logs telegram-forwarder 2>&1 | grep -cE "正则匹配成功|已发送" && \
+echo "=== Hermes ===" && \
+grep -c "inbound.*1003966251111" ~/.hermes/logs/gateway.log 2>/dev/null && \
+echo "=== Errors ===" && \
+tail -3 ~/.hermes/logs/errors.log 2>/dev/null
+```
+
+## Log Pattern Reference
+
+| Pattern | Meaning | Where |
+|---------|---------|-------|
+| `收到新消息 来自聊天: XXX` | Forwarder received message | forwarder logs |
+| `正则匹配成功` | Keyword regex matched | forwarder logs |
+| `未匹配到普通白名单关键词` | Keyword didn't match | forwarder logs |
+| `所有白名单条件都满足，允许转发` | Forwarding allowed | forwarder logs |
+| `消息已发送到: XXX` | Message sent to target | forwarder logs |
+| `过滤器识别到机器人消息，忽略处理` | Bot message filtered (normal) | forwarder logs |
+| `inbound message: platform=telegram chat=XXX` | Hermes received message | gateway.log |
+| `response ready: platform=telegram chat=XXX` | Hermes responded | gateway.log |
+| `Suppressing normal final send` | Response already delivered via stream | gateway.log |
+
+## Common Scenarios
+
+### "昨天能收到，今天收不到"
+- Check if source channel stopped sending (forwarder logs)
+- Check if Telethon session expired (container restart fixes this)
+- Check if signal format changed
+
+### "有信号进来但没收到你的消息"
+- Could be timing: signal hasn't arrived yet, or arrived but user expected Hermes response
+- Quick check: `docker logs telegram-forwarder 2>&1 | grep "收到新消息.*实盘监控" | tail -1` — compare timestamp to current time. If >30min stale, the signal may not have reached the forwarder.
+- If forwarder received and forwarded but no Hermes response: check `tail -20 ~/.hermes/logs/gateway.log | grep inbound` — if no recent group messages, the bot may not be receiving (privacy mode).
+- **Bot message filtering**: Hermes may ignore messages sent by other bots. Always test with a human account, not the forwarder bot.
+- **⚠️ Check for competing gateway processes**: `ps aux | grep "gateway\|openclaw" | grep -v grep` — if an old OpenClaw Gateway or duplicate Hermes process is running, it steals the bot token polling session and Hermes NEVER receives any group messages. This was the actual root cause for a user whose group messages had NEVER worked for over a week.
+
+### "信号在频道里能看到，但没转发到群"
+- Step 2 failure: keyword mismatch
+- Check actual signal text vs regex pattern
+
+### "信号转发到群了，但Hermes没回复"
+- Step 4 failure: privacy mode or config
+- Most common: bot privacy mode is ON
+
+### "重启了 gateway 还是不处理群消息"（Polling Conflict — 两种原因）
+
+**This is the #1 hidden cause.** There are TWO distinct polling conflict scenarios:
+
+#### Scenario A: Competing gateway processes (most dangerous)
+
+Another process is using the same bot token for `getUpdates`. This causes PERMANENT failure — no amount of restarting Hermes will help until the other process is killed.
+
+**Diagnosis**:
+```bash
+# 1. Check for competing processes
+ps aux | grep "gateway\|openclaw" | grep -v grep
+# Look for: node .../openclaw/dist/index.js gateway --port 18789
+# Or: another hermes gateway run
+
+# 2. Test getUpdates directly (bypass adapter)
+python3 -c "
+import json, urllib.request
+with open('/home/openclaw/.hermes/.env') as f:
+    for line in f:
+        if line.startswith('TELEGRAM_BOT_TOKEN'):
+            token = line.strip().split('=',1)[1]
+            break
+proxy = urllib.request.ProxyHandler({'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'})
+opener = urllib.request.build_opener(proxy)
+try:
+    req = urllib.request.Request(f'https://api.telegram.org/bot{token}/getUpdates?timeout=1')
+    resp = opener.open(req, timeout=5)
+    data = json.loads(resp.read())
+    print(f'getUpdates: {len(data.get(\"result\", []))} updates (no conflict = nobody polling)')
+except Exception as e:
+    if '409' in str(e):
+        print('409 Conflict — adapter IS polling (normal)')
+    else:
+        print(f'Error: {e}')
+"
+```
+
+**Key diagnostic signal**:
+- `getUpdates` returns 0 without 409 conflict → **NOBODY is polling** → competing process consumed and dropped updates
+- `getUpdates` returns 409 → adapter IS polling (conflict is expected, this is normal)
+- `getUpdates` returns updates → adapter is NOT polling AND competing process dropped off → restart Hermes
+
+**Fix**: Kill the stale process FIRST, then restart Hermes:
+```bash
+kill <stale_pid>
+systemctl --user restart hermes-gateway
+```
+
+**Verify**: `journalctl --user -u hermes-gateway --since "1 min ago" | grep "✓ telegram connected"` — must appear after restart.
+
+#### Scenario B: `hermes gateway restart` race condition
+
+The restart command sends SIGTERM and immediately starts a new process. Telegram's servers keep the old polling session alive for ~30 seconds. The new process tries to poll → conflict → retry loop.
+
+**Diagnosis**:
+```bash
+journalctl --user -u hermes-gateway --since "5 min ago" --no-pager | grep -i "conflict"
+```
+
+**Fix**:
+```bash
+systemctl --user stop hermes-gateway
+sleep 30  # critical: wait for Telegram session to expire
+systemctl --user start hermes-gateway
+```
+
+#### How to tell which scenario
+
+```bash
+# Check how long the gateway has been running
+systemctl --user status hermes-gateway | grep "Active:"
+# If running for days/weeks but group messages NEVER worked → Scenario A
+# If just restarted and immediately getting conflicts → Scenario B
+
+# Check for stale processes
+ps aux | grep "gateway" | grep -v grep | grep -v $(pgrep -f "hermes_cli.main gateway")
+# If you see old processes → Scenario A
+```
+
+## Gateway Logs (Binary Format — Use strings!)
+
+Gateway logs at `~/.hermes/logs/gateway.log` are **binary format**. Use `strings` to extract:
+
+```bash
+strings ~/.hermes/logs/gateway.log | tail -30
+strings ~/.hermes/logs/gateway.log | grep "inbound message" | tail -10
+strings ~/.hermes/logs/gateway.log | grep "<group_chat_id>" | tail -10
+strings ~/.hermes/logs/gateway.log | grep "Connected to"
+```
+
+## ⚠️ Testing: Use Human Account
+
+**Never test with bot-sent messages.** Bots cannot see their own messages via getUpdates. Always use a human Telegram account to send test messages to the group or source channel.
+
+## Agent-Side Filtering Architecture
+
+When the user says "收所有的消息，在处理消息这边来处理规则过滤":
+1. Forwarder: `.*` whitelist (pass everything)
+2. Agent channel_prompts: classify messages as A(signal)/B(confirm)/C(close)/D(ignore)
+3. Only process A/B/C, silently ignore D
+
+Benefits: source format changes only require updating the agent prompt, not the forwarder database.
@@ -0,0 +1,36 @@
+# Diagnostic: check if a Telegram group has been migrated to a supergroup
+# Usage: docker exec telegram-forwarder python3 <this-script>
+#
+# When a group migrates to a supergroup, the old ID becomes deactivated.
+# The new ID is the channel_id from migrated_to, prefixed with -100.
+
+from telethon import TelegramClient
+import asyncio, socks
+
+async def check_migration(chat_id: int):
+    proxy = (socks.SOCKS5, '127.0.0.1', 7890)
+    # Use bot client for checking
+    bot = TelegramClient('./sessions/bot', API_ID, API_HASH, proxy=proxy)
+    await bot.start()
+
+    try:
+        entity = await bot.get_entity(chat_id)
+        print(f'Entity type: {type(entity).__name__}')
+        print(f'ID: {entity.id}')
+        print(f'Title: {getattr(entity, "title", "N/A")}')
+        if hasattr(entity, 'deactivated') and entity.deactivated:
+            print(f'⚠️  DEACTIVATED - group was migrated!')
+            if hasattr(entity, 'migrated_to') and entity.migrated_to:
+                new_id = entity.migrated_to.channel_id
+                print(f'New channel_id: {new_id}')
+                print(f'New Telethon ID: -100{new_id}')
+                print(f'Hermes free_response_channels should be: "-100{new_id}"')
+        else:
+            print('✅ Active - no migration detected')
+    except Exception as e:
+        print(f'Error: {e}')
+
+    await bot.disconnect()
+
+# Example: check old group ID
+# asyncio.run(check_migration(5586190970))
@@ -0,0 +1,145 @@
+# Hermes + TelegramForwarder Group Integration
+
+## Problem
+Forwarder sends messages to a group, but Hermes bot (also in the group) ignores them.
+
+## Root Causes (check ALL of these)
+
+| # | Cause | Symptom | Fix |
+|---|-------|---------|-----|
+| 0 | **Competing gateway process** (old OpenClaw, duplicate Hermes) | Adapter shows "connected" but `getUpdates` returns 0 without 409 conflict. No "✓ telegram connected" in logs. Group messages NEVER worked. | `ps aux \| grep gateway` → kill stale process → restart Hermes |
+| 1 | Bot Privacy Mode is ON | Bot receives 0 group messages | @BotFather → /mybots → Bot Settings → Group Privacy → Turn off |
+| 2 | Wrong config key `free_response_channels` | Bot sees @mentions but ignores regular messages | Use `extra.free_response_chats` (NOT top-level `free_response_channels`) |
+| 3 | Group migrated to supergroup | `An invalid Peer was used` in forwarder logs | Update chat ID (see below) |
+| 4 | Privacy mode not propagated | `can_read_all_group_messages=true` but bot still can't see messages | Kick bot from group, re-add it |
+
+**⚠️ Check #0 FIRST.** A stale gateway process is invisible — the state file says "connected", no errors appear, but the adapter never actually polls. This was the actual root cause for a user whose group messages had NEVER worked for over a week (old OpenClaw Gateway PID 900 running since June 22). Always run `ps aux | grep gateway | grep -v grep` before investigating other causes.
+
+**⚠️ Privacy mode is the #1 cause.** Even with `free_response_chats` correctly configured, if privacy mode is ON, the bot literally cannot see group messages. Fix privacy mode FIRST.
+
+## Step-by-step
+
+### 0. Check for Competing Gateway Processes (DO THIS FIRST)
+```bash
+ps aux | grep "gateway\|openclaw" | grep -v grep
+```
+If you see any `openclaw ... gateway` or duplicate `hermes gateway` processes → **kill them immediately**. They steal the bot token polling session.
+
+Verify nobody else is polling:
+```bash
+python3 -c "
+import json, urllib.request
+with open('/home/openclaw/.hermes/.env') as f:
+    for line in f:
+        if line.startswith('TELEGRAM_BOT_TOKEN'):
+            token = line.strip().split('=',1)[1]
+            break
+proxy = urllib.request.ProxyHandler({'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'})
+opener = urllib.request.build_opener(proxy)
+try:
+    req = urllib.request.Request(f'https://api.telegram.org/bot{token}/getUpdates?timeout=1')
+    resp = opener.open(req, timeout=5)
+    print(f'getUpdates: {len(json.loads(resp.read()).get(\"result\",[]))} updates — nobody polling!'  )
+except Exception as e:
+    print(f'409 Conflict — adapter IS polling (normal)' if '409' in str(e) else f'Error: {e}')
+"
+```
+- 0 updates without 409 → broken (check for competing processes)
+- 409 Conflict → normal (adapter is polling)
+
+### 1. Disable Bot Privacy Mode (MANDATORY)
+Open Telegram → @BotFather:
+```
+/mybots → select Hermes bot → Bot Settings → Group Privacy → Turn off
+```
+Do this for the Hermes bot. If the forwarder bot also needs to read messages in the group, do it for that bot too.
+
+**After disabling, kick the bot from the group and re-add it.** The change does not apply to bots already in a group.
+
+Verify: `curl -s --proxy http://127.0.0.1:7890 "https://api.telegram.org/bot<TOKEN>/getMe"` → `can_read_all_group_messages` must be `true`.
+
+### 2. Find the destination group's chat_id
+```bash
+cd ~/TelegramForwarder
+docker exec telegram-forwarder python3 -c "
+import sqlite3
+conn = sqlite3.connect('/app/db/forward.db')
+c = conn.cursor()
+c.execute('SELECT id, telegram_chat_id, name FROM chats')
+for ch in c.fetchall():
+    print(ch)
+conn.close()
+"
+```
+The column is `telegram_chat_id` (NOT `chat_id`).
+
+**⚠️ Supergroup IDs**: If the group was migrated, the old ID is stale. Use `-100` + `channel_id` format. To find the real channel_id:
+```python
+entity = await bot.get_entity(old_id)
+# If entity has migrated_to:
+new_id = -100 * entity.migrated_to.channel_id  # e.g., -1003966251111
+```
+
+### 3. Edit Hermes config (CORRECT KEY!)
+`~/.hermes/config.yaml` — add inside the `telegram:` → `extra:` section:
+```yaml
+telegram:
+  extra:
+    free_response_chats: '-1003966251111'  # ← CORRECT key
+  # free_response_channels: '...'  ← WRONG — code ignores this!
+```
+
+**⚠️ CRITICAL**: The Hermes source code reads `self.config.extra.get("free_response_chats")`. The top-level `free_response_channels` is a DIFFERENT field that is NOT read by the group-message processing logic. Using the wrong key = silently ignored, bot only responds to @mentions.
+
+Multiple groups: comma-separated `'-1003966251111,-1001234567890'`
+
+### 4. Restart Hermes
+```bash
+hermes gateway restart  # from a separate shell, NOT inside the gateway
+```
+If refused ("Refusing to restart from inside the gateway"), ask the user to SSH in and run it.
+
+## Config Field Reference
+| Field | Location | Default | Purpose |
+|-------|----------|---------|---------|
+| `require_mention` | `telegram:` | `true` | If true, bot only responds when @mentioned in groups |
+| `free_response_chats` | `telegram:extra:` | `''` | Comma-separated chat_ids exempt from @mention requirement |
+| `free_response_channels` | `telegram:` | `''` | **NOT used by group message processing** — legacy field, ignore |
+| `allowed_chats` | `telegram:` | `''` | If set, ONLY these chats can interact with the bot (whitelist) |
+
+## Signal Forwarding with channel_prompts
+
+When you want Hermes to process group messages (e.g., trading signals) and forward results to a user DM instead of replying in the group:
+
+```yaml
+telegram:
+  channel_prompts:
+    "-1003966251111": "你是交易信号处理器。收到群里的交易信号后，解析【币种】【方向】【仓位】，然后用send_message工具发送到telegram:656685714（用户DM），格式简洁：信号摘要+当前价格+建议操作+风险提示。不要在群里回复。"
+```
+
+This configures a per-group system prompt. The agent in that group session will follow these instructions instead of the default behavior.
+
+## End-to-End Verification
+
+After setting up the forwarding → Hermes chain, verify each link:
+
+1. **Forwarder listening**: `docker logs telegram-forwarder 2>&1 | grep "收到新消息"` — should show messages from source channel
+2. **Forwarder sending**: `docker logs telegram-forwarder 2>&1 | grep "已发送"` — should show successful sends to target group
+3. **Hermes receiving**: `hermes logs 2>&1 | grep "inbound message"` — should show messages from the group (chat ID should match the group, not the user DM)
+4. **Hermes responding**: Check the group for Hermes's reply
+
+**Common failure chain**:
+- Forwarder logs show "收到新消息" but no "已发送" → keyword filter issue or invalid Peer (check group ID)
+- Forwarder logs show "已发送" but Hermes shows no inbound from group → privacy mode still on, or `free_response_chats` wrong key
+- Hermes shows inbound but no response → check `require_mention` and `free_response_chats`
+
+**⚠️ For testing, use a human account** (not the forwarder bot) to send messages to the source channel or directly to the group. Bot-sent messages may be filtered by Hermes.
+
+## Pitfalls
+- `free_response_chats` takes string values, not integers: `'-1003966251111'` not `-1003966251111`
+- Config changes require Hermes restart to take effect
+- If `allowed_chats` is set (non-empty), the group must also be in that list
+- The forwarder's `handle_mode: FORWARD` uses Telegram's native forward (shows "Forwarded from"). If user wants clean copies, change to `COPY` mode.
+- **Don't test by sending directly to the target group** — this bypasses the forwarding chain and proves nothing. Test by sending to the SOURCE group and verifying the full pipeline: source → forwarder → target → Hermes response.
+- **Bot-to-bot messaging FAILS**: Telegram prohibits bot→bot messaging. Forward to a GROUP where both bots reside, not directly to a bot.
+- **Session database locked**: If you see `sqlite3.OperationalError: database is locked`, stop the container first, edit DB, then restart.