Hermes-Skills/devops/network-failover/SKILL.md

name, description, origin

name	description	origin
network-failover	Automatic network failover and proxy/VPN switching. Detect connectivity loss → activate backup route (WireGuard VPN, proxy, etc.) → deactivate when primary recovers. Use when user says "自动切换代理", "网络受限", "auto switch VPN", "network failover", "proxy auto switch".	user

Network Failover & Auto-Switch

Automatically detect network restrictions and switch to a backup route (WireGuard VPN, SOCKS/HTTP proxy, etc.), then switch back when the primary network recovers.

When to Use

• User has a WireGuard VPN and wants it activated only when needed
• User wants automatic proxy switching when direct connectivity fails
• Environments where the network is intermittently restricted (GFW, corporate firewalls, etc.)
• Reducing VPN overhead by only using it when necessary

Architecture

┌─────────────────────────────────────────────┐
│  Monitor (systemd timer / cron)             │
│  Every 30s: ping/curl to multiple targets   │
│                                             │
│  ┌─ Success? ──────────────────────────────┐│
│  │  VPN up + N consecutive recoveries      ││
│  │  → disconnect VPN                       ││
│  └─────────────────────────────────────────┘│
│                                             │
│  ┌─ Failure? ──────────────────────────────┐│
│  │  VPN down + M consecutive failures      ││
│  │  → activate VPN                         ││
│  └─────────────────────────────────────────┘│
└─────────────────────────────────────────────┘

Key Design Decisions

1. Hysteresis (anti-flap): Require N consecutive failures before activating, M consecutive successes before deactivating. Never switch on a single check.
2. Multi-target probing: Test multiple IPs (8.8.8.8, 1.1.1.1, 208.67.222.222) with both curl and ping. DNS can break before ICMP, so probe at multiple layers.
3. Logging: Always log switch events to a file for post-mortem debugging.
4. State files: Use /tmp/ files for counters (ephemeral, reset on reboot = safe default).

WireGuard Auto-Switch

See references/wireguard-auto-switch.md for the full implementation script and systemd timer setup.

Pitfalls

• ⚠️ TEST BEFORE EXECUTE — network changes can kill connectivity instantly. WireGuard with AllowedIPs = 0.0.0.0/0 intercepts ALL traffic the moment wg-quick up runs. If the VPN endpoint is unreachable or misconfigured, the machine loses all network access — including SSH. Always: (1) check current state first (ip route, wg show, ping), (2) test VPN endpoint reachability before activating, (3) verify connectivity after each change, (4) have a fallback plan (e.g., a second terminal with a pre-existing SSH session, or physical console access). Never blindly execute wg-quick up/down or route changes.
• Single-ping switching causes flapping — always use hysteresis (3 failures to activate, 2 recoveries to deactivate).
• DNS failure ≠ network failure — if DNS breaks first, curl http://8.8.8.8 still works (IP literal). Test both DNS-dependent and DNS-independent targets.
• Full tunnel kills DNS — when WireGuard AllowedIPs = 0.0.0.0/0, ALL traffic (including DNS) goes through VPN. If VPN has DNS issues, the system is blind. Consider setting DNS = 8.8.8.8 in the WireGuard config.
• sudo without password — the script needs wg-quick up/down which requires root. Either add NOPASSWD sudoers entry or run the systemd service as root.
• Log rotation — the switch log grows unbounded. Add logrotate or truncate periodically.
• PersistentKeepalive = 25 — mandatory on mobile/NAT clients, otherwise idle tunnels get dropped by NAT and the VPN appears "connected" but is actually dead.

Related Skills

• homelab-wireguard-vpn — WireGuard server/client setup fundamentals
• homelab-wireguard-vpn > references/ — DDNS, key generation, split vs full tunnel