Initial commit: Hermes Agent skills collection
- Trading skills (OKX, dividend, lottery, quantitative) - Creative skills (ASCII art, diagrams, video) - Development skills (GitHub, debugging, TDD) - Research skills (arXiv, blog monitoring) - Productivity skills (email, documents, notes) - MCP integration skills - Custom user skills
This commit is contained in:
@@ -0,0 +1,251 @@
|
||||
# Signal Forwarding Debugging Guide
|
||||
|
||||
Systematic approach to diagnose "信号没收到" (signal not received) complaints.
|
||||
|
||||
## The Chain
|
||||
|
||||
```
|
||||
Source Channel → TelegramForwarder → Target Group → Hermes → User DM
|
||||
(1) (2) (3) (4) (5)
|
||||
```
|
||||
|
||||
Failure at ANY link breaks the chain. Debug from left to right.
|
||||
|
||||
## Step 1: Forwarder Receiving Messages?
|
||||
|
||||
```bash
|
||||
docker logs telegram-forwarder 2>&1 | grep "收到新消息.*实盘监控" | tail -5
|
||||
```
|
||||
|
||||
- **Has entries** → Forwarder is receiving from source. Go to Step 2.
|
||||
- **No entries** → Source channel not bound, or Telethon session expired. Check DB:
|
||||
```bash
|
||||
docker exec telegram-forwarder python3 -c "
|
||||
import sqlite3
|
||||
conn = sqlite3.connect('/app/db/forward.db')
|
||||
for r in conn.execute('SELECT id, telegram_chat_id, name FROM chats'): print(r)
|
||||
conn.close()"
|
||||
```
|
||||
|
||||
## Step 2: Keyword Matching?
|
||||
|
||||
```bash
|
||||
docker logs telegram-forwarder 2>&1 | grep -E "正则匹配成功|未匹配" | tail -5
|
||||
```
|
||||
|
||||
- `正则匹配成功` → Match succeeded. Go to Step 3.
|
||||
- `未匹配到普通白名单关键词` → Signal format changed, or regex doesn't match.
|
||||
- Check the actual signal text: `docker logs telegram-forwarder 2>&1 | grep "内容:" | tail -5`
|
||||
- Common issue: signal uses different brackets (e.g., `[币种]` vs `【币种】`)
|
||||
- Common issue: multi-line signals — `.*` doesn't match `\n`, use `[\s\S]*` instead
|
||||
|
||||
## Step 3: Forwarder Sending to Group?
|
||||
|
||||
```bash
|
||||
docker logs telegram-forwarder 2>&1 | grep -E "已发送|成功获取目标" | tail -5
|
||||
```
|
||||
|
||||
- `消息已发送到: XXX` → Forward succeeded. Go to Step 4.
|
||||
- No send logs after match → Check for errors:
|
||||
```bash
|
||||
docker logs telegram-forwarder 2>&1 | grep -iE "error|invalid Peer|banned|forbidden" | tail -5
|
||||
```
|
||||
- `An invalid Peer was used` → Group migrated to supergroup, ID changed
|
||||
- `Bots can't send messages` → Trying to send to a bot, not a group
|
||||
|
||||
## Step 4: Hermes Receiving from Group?
|
||||
|
||||
**FIRST: Check for competing gateway processes** (most common hidden cause):
|
||||
```bash
|
||||
ps aux | grep "gateway\|openclaw" | grep -v grep
|
||||
```
|
||||
If you see an old `openclaw ... gateway` or duplicate `hermes gateway` process → **KILL IT FIRST**. Two processes sharing one bot token = permanent polling failure. See "Polling Conflict — Scenario A" below.
|
||||
|
||||
Then check polling status:
|
||||
```bash
|
||||
# Check for polling conflict
|
||||
journalctl --user -u hermes-gateway --since "10 min ago" --no-pager | grep -i "conflict"
|
||||
# If found → see "Polling Conflict" section below
|
||||
|
||||
# Check if "✓ telegram connected" appears in logs
|
||||
journalctl --user -u hermes-gateway --since "5 min ago" --no-pager | grep "✓ telegram"
|
||||
# If NOT found → adapter didn't connect (check for competing processes or conflict)
|
||||
|
||||
# Check gateway.log for inbound messages from the group
|
||||
grep "inbound message" ~/.hermes/logs/gateway.log | grep "1003966251111" | tail -5
|
||||
```
|
||||
|
||||
- **Polling conflict found** → Stop gateway, wait 30s, start fresh (see "Polling Conflict" section)
|
||||
- **Has entries** → Hermes receives group messages. Go to Step 5.
|
||||
- **No entries, no conflict** → Hermes can't see group messages. Check:
|
||||
1. Bot privacy mode: `curl -s --proxy http://127.0.0.1:7890 "https://api.telegram.org/bot<TOKEN>/getMe" | grep can_read_all_group_messages`
|
||||
- Must be `true`. If `false`, disable in @BotFather and **kick+re-add** bot to group.
|
||||
2. `free_response_chats` config (NOT `free_response_channels`):
|
||||
```bash
|
||||
grep -A5 "extra:" ~/.hermes/config.yaml | grep free_response
|
||||
```
|
||||
Must be under `telegram.extra.free_response_chats`, not top-level.
|
||||
3. Check if bot is actually in the group.
|
||||
|
||||
## Step 5: Hermes Processing and Responding?
|
||||
|
||||
```bash
|
||||
grep "1003966251111" ~/.hermes/logs/gateway.log | grep -E "response ready|Suppressing" | tail -5
|
||||
```
|
||||
|
||||
- `response ready` → Hermes processed and responded. Check delivery.
|
||||
- `inbound` but no `response` → Agent failed or timed out. Check errors.log:
|
||||
```bash
|
||||
tail -20 ~/.hermes/logs/errors.log
|
||||
```
|
||||
|
||||
## Quick One-Liner Diagnostic
|
||||
|
||||
```bash
|
||||
echo "=== Forwarder ===" && \
|
||||
docker logs telegram-forwarder 2>&1 | grep -cE "正则匹配成功|已发送" && \
|
||||
echo "=== Hermes ===" && \
|
||||
grep -c "inbound.*1003966251111" ~/.hermes/logs/gateway.log 2>/dev/null && \
|
||||
echo "=== Errors ===" && \
|
||||
tail -3 ~/.hermes/logs/errors.log 2>/dev/null
|
||||
```
|
||||
|
||||
## Log Pattern Reference
|
||||
|
||||
| Pattern | Meaning | Where |
|
||||
|---------|---------|-------|
|
||||
| `收到新消息 来自聊天: XXX` | Forwarder received message | forwarder logs |
|
||||
| `正则匹配成功` | Keyword regex matched | forwarder logs |
|
||||
| `未匹配到普通白名单关键词` | Keyword didn't match | forwarder logs |
|
||||
| `所有白名单条件都满足,允许转发` | Forwarding allowed | forwarder logs |
|
||||
| `消息已发送到: XXX` | Message sent to target | forwarder logs |
|
||||
| `过滤器识别到机器人消息,忽略处理` | Bot message filtered (normal) | forwarder logs |
|
||||
| `inbound message: platform=telegram chat=XXX` | Hermes received message | gateway.log |
|
||||
| `response ready: platform=telegram chat=XXX` | Hermes responded | gateway.log |
|
||||
| `Suppressing normal final send` | Response already delivered via stream | gateway.log |
|
||||
|
||||
## Common Scenarios
|
||||
|
||||
### "昨天能收到,今天收不到"
|
||||
- Check if source channel stopped sending (forwarder logs)
|
||||
- Check if Telethon session expired (container restart fixes this)
|
||||
- Check if signal format changed
|
||||
|
||||
### "有信号进来但没收到你的消息"
|
||||
- Could be timing: signal hasn't arrived yet, or arrived but user expected Hermes response
|
||||
- Quick check: `docker logs telegram-forwarder 2>&1 | grep "收到新消息.*实盘监控" | tail -1` — compare timestamp to current time. If >30min stale, the signal may not have reached the forwarder.
|
||||
- If forwarder received and forwarded but no Hermes response: check `tail -20 ~/.hermes/logs/gateway.log | grep inbound` — if no recent group messages, the bot may not be receiving (privacy mode).
|
||||
- **Bot message filtering**: Hermes may ignore messages sent by other bots. Always test with a human account, not the forwarder bot.
|
||||
- **⚠️ Check for competing gateway processes**: `ps aux | grep "gateway\|openclaw" | grep -v grep` — if an old OpenClaw Gateway or duplicate Hermes process is running, it steals the bot token polling session and Hermes NEVER receives any group messages. This was the actual root cause for a user whose group messages had NEVER worked for over a week.
|
||||
|
||||
### "信号在频道里能看到,但没转发到群"
|
||||
- Step 2 failure: keyword mismatch
|
||||
- Check actual signal text vs regex pattern
|
||||
|
||||
### "信号转发到群了,但Hermes没回复"
|
||||
- Step 4 failure: privacy mode or config
|
||||
- Most common: bot privacy mode is ON
|
||||
|
||||
### "重启了 gateway 还是不处理群消息"(Polling Conflict — 两种原因)
|
||||
|
||||
**This is the #1 hidden cause.** There are TWO distinct polling conflict scenarios:
|
||||
|
||||
#### Scenario A: Competing gateway processes (most dangerous)
|
||||
|
||||
Another process is using the same bot token for `getUpdates`. This causes PERMANENT failure — no amount of restarting Hermes will help until the other process is killed.
|
||||
|
||||
**Diagnosis**:
|
||||
```bash
|
||||
# 1. Check for competing processes
|
||||
ps aux | grep "gateway\|openclaw" | grep -v grep
|
||||
# Look for: node .../openclaw/dist/index.js gateway --port 18789
|
||||
# Or: another hermes gateway run
|
||||
|
||||
# 2. Test getUpdates directly (bypass adapter)
|
||||
python3 -c "
|
||||
import json, urllib.request
|
||||
with open('/home/openclaw/.hermes/.env') as f:
|
||||
for line in f:
|
||||
if line.startswith('TELEGRAM_BOT_TOKEN'):
|
||||
token = line.strip().split('=',1)[1]
|
||||
break
|
||||
proxy = urllib.request.ProxyHandler({'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'})
|
||||
opener = urllib.request.build_opener(proxy)
|
||||
try:
|
||||
req = urllib.request.Request(f'https://api.telegram.org/bot{token}/getUpdates?timeout=1')
|
||||
resp = opener.open(req, timeout=5)
|
||||
data = json.loads(resp.read())
|
||||
print(f'getUpdates: {len(data.get(\"result\", []))} updates (no conflict = nobody polling)')
|
||||
except Exception as e:
|
||||
if '409' in str(e):
|
||||
print('409 Conflict — adapter IS polling (normal)')
|
||||
else:
|
||||
print(f'Error: {e}')
|
||||
"
|
||||
```
|
||||
|
||||
**Key diagnostic signal**:
|
||||
- `getUpdates` returns 0 without 409 conflict → **NOBODY is polling** → competing process consumed and dropped updates
|
||||
- `getUpdates` returns 409 → adapter IS polling (conflict is expected, this is normal)
|
||||
- `getUpdates` returns updates → adapter is NOT polling AND competing process dropped off → restart Hermes
|
||||
|
||||
**Fix**: Kill the stale process FIRST, then restart Hermes:
|
||||
```bash
|
||||
kill <stale_pid>
|
||||
systemctl --user restart hermes-gateway
|
||||
```
|
||||
|
||||
**Verify**: `journalctl --user -u hermes-gateway --since "1 min ago" | grep "✓ telegram connected"` — must appear after restart.
|
||||
|
||||
#### Scenario B: `hermes gateway restart` race condition
|
||||
|
||||
The restart command sends SIGTERM and immediately starts a new process. Telegram's servers keep the old polling session alive for ~30 seconds. The new process tries to poll → conflict → retry loop.
|
||||
|
||||
**Diagnosis**:
|
||||
```bash
|
||||
journalctl --user -u hermes-gateway --since "5 min ago" --no-pager | grep -i "conflict"
|
||||
```
|
||||
|
||||
**Fix**:
|
||||
```bash
|
||||
systemctl --user stop hermes-gateway
|
||||
sleep 30 # critical: wait for Telegram session to expire
|
||||
systemctl --user start hermes-gateway
|
||||
```
|
||||
|
||||
#### How to tell which scenario
|
||||
|
||||
```bash
|
||||
# Check how long the gateway has been running
|
||||
systemctl --user status hermes-gateway | grep "Active:"
|
||||
# If running for days/weeks but group messages NEVER worked → Scenario A
|
||||
# If just restarted and immediately getting conflicts → Scenario B
|
||||
|
||||
# Check for stale processes
|
||||
ps aux | grep "gateway" | grep -v grep | grep -v $(pgrep -f "hermes_cli.main gateway")
|
||||
# If you see old processes → Scenario A
|
||||
```
|
||||
|
||||
## Gateway Logs (Binary Format — Use strings!)
|
||||
|
||||
Gateway logs at `~/.hermes/logs/gateway.log` are **binary format**. Use `strings` to extract:
|
||||
|
||||
```bash
|
||||
strings ~/.hermes/logs/gateway.log | tail -30
|
||||
strings ~/.hermes/logs/gateway.log | grep "inbound message" | tail -10
|
||||
strings ~/.hermes/logs/gateway.log | grep "<group_chat_id>" | tail -10
|
||||
strings ~/.hermes/logs/gateway.log | grep "Connected to"
|
||||
```
|
||||
|
||||
## ⚠️ Testing: Use Human Account
|
||||
|
||||
**Never test with bot-sent messages.** Bots cannot see their own messages via getUpdates. Always use a human Telegram account to send test messages to the group or source channel.
|
||||
|
||||
## Agent-Side Filtering Architecture
|
||||
|
||||
When the user says "收所有的消息,在处理消息这边来处理规则过滤":
|
||||
1. Forwarder: `.*` whitelist (pass everything)
|
||||
2. Agent channel_prompts: classify messages as A(signal)/B(confirm)/C(close)/D(ignore)
|
||||
3. Only process A/B/C, silently ignore D
|
||||
|
||||
Benefits: source format changes only require updating the agent prompt, not the forwarder database.
|
||||
Reference in New Issue
Block a user