diff --git a/AGENTS-PROD.md b/AGENTS-PROD.md deleted file mode 100644 index 073d552..0000000 --- a/AGENTS-PROD.md +++ /dev/null @@ -1,407 +0,0 @@ -# General Bots Cloud — Production Operations Guide - -## Infrastructure Overview -- **Host OS:** Ubuntu 24.04 LTS -- **SSH:** Key auth only -- **Container engine:** LXC (Linux Containers) -- **Tenant:** pragmatismo - ---- - -## PostgreSQL Container (tables) Management - -### Container Info -- **Container Name:** pragmatismo-tables -- **Database:** PostgreSQL 14 -- **Access:** Inside LXC container on port 5432 -- **Data Location:** `/opt/gbo/tables/` inside container - -### Common Operations - -```bash -# Check container status -lxc list | grep pragmatismo-tables - -# Exec into container -lxc exec pragmatismo-tables -- bash - -# Check PostgreSQL status -lxc exec pragmatismo-tables -- pg_isready - -# Query version -lxc exec pragmatismo-tables -- psql -U postgres -c 'SELECT version();' - -# Restart PostgreSQL -lxc exec pragmatismo-tables -- systemctl restart postgresql -``` - -### Backup PostgreSQL - -```bash -# Create database dump -lxc exec pragmatismo-tables -- pg_dump -U postgres -F c -f /tmp/backup.dump pragmatismo - -# Copy backup to host -lxc file pull pragmatismo-tables/tmp/backup.dump ~/backups/postgresql-$(date +%Y%m%d).dump -``` - ---- - -## LXC Container Status (82.29.59.188) - -| Container | Status | Purpose | Notes | -|-----------|--------|---------|-------| -| **pragmatismo-dns** | ✅ RUNNING | CoreDNS | DNS server | -| **pragmatismo-proxy** | ✅ RUNNING | Caddy | Reverse proxy on :80/:443 | -| **pragmatismo-tables** | ✅ RUNNING | PostgreSQL 14 | Database | -| **pragmatismo-system** | ✅ RUNNING | botserver | Bot system | -| **pragmatismo-email** | ✅ RUNNING | Stalwart | Email server | -| **pragmatismo-webmail** | ✅ RUNNING | Roundcube | Webmail interface | -| **pragmatismo-alm** | ✅ RUNNING | Forgejo | Git/Code ALM | -| **pragmatismo-alm-ci** | ✅ RUNNING | Runner | CI/CD runner | -| **pragmatismo-drive** | ✅ RUNNING | MinIO | S3-compatible storage | - ---- - -## LXC Container Management - -### Common Commands - -```bash -# List all containers -lxc list - -# Show container details -lxc info pragmatismo-dns - -# Exec into container -lxc exec pragmatismo-dns -- bash -lxc exec pragmatismo-dns -- /bin/sh - -# Start/Stop/Restart containers -lxc start pragmatismo-dns -lxc stop pragmatismo-dns -lxc restart pragmatismo-dns - -# View logs -lxc log pragmatismo-dns -lxc log pragmatismo-dns --show-log - -# Copy files to/from container -lxc file push localfile pragmatismo-dns/opt/gbo/conf/ -lxc file pull pragmatismo-dns/opt/gbo/logs/output.log . -``` - ---- - -## SSH Setup - -### On Production Server - -```bash -# Add SSH public key for access -mkdir -p /root/.ssh -echo "" >> /root/.ssh/authorized_keys -chmod 700 /root/.ssh -chmod 600 /root/.ssh/authorized_keys -``` - -### From This Machine - -```bash -# SSH key for passwordless access to production -ssh-copy-id root@82.29.59.188 - -# Test connection -ssh root@82.29.59.188 "lxc list" -``` - ---- - -## LXC Container Data Sync (rsync) - -### Prerequisites -- Source: SSH access as `root` to source host -- Network: Same LAN or VPN connection between hosts - -### Copy Individual Container Data - -```bash -# Copy specific container data (e.g., dns container) -sudo rsync -avz --progress -e ssh \ - root@:/opt/gbo/tenants/pragmatismo/dns/ \ - /opt/gbo/dns/ - -# Copy tables (PostgreSQL data) -sudo rsync -avz --progress -e ssh \ - root@:/opt/gbo/tenants/pragmatismo/tables/ \ - /opt/gbo/tables/ - -# Copy drive (MinIO/S3 data) -sudo rsync -avz --progress -e ssh \ - root@:/opt/gbo/tenants/pragmatismo/drive/ \ - /opt/gbo/drive/ -``` - -### Dry Run Before Copy - -```bash -# Preview what will be copied (no changes made) -sudo rsync -avzn --progress -e ssh \ - root@:/opt/gbo/tenants/pragmatismo/dns/ \ - /opt/gbo/dns/ -``` - -### Exclude Patterns - -```bash -# Exclude logs, temp files, and system directories -sudo rsync -avz --progress -e ssh \ - --exclude='*.log' \ - --exclude='*.tmp' \ - --exclude='.git/' \ - --exclude='__pycache__/' \ - root@:/opt/gbo/tenants/pragmatismo/ \ - /opt/gbo/ -``` - ---- - -## DNS Management (pragmatismo-dns) - -### Container Info -- **Container Name:** pragmatismo-dns -- **Service:** CoreDNS -- **Access:** Inside LXC container on port 53 -- **Config:** `/opt/gbo/dns/conf/Corefile` inside container - -### Common Operations - -```bash -# Check container status -lxc list | grep pragmatismo-dns - -# View logs -lxc log pragmatismo-dns -lxc log pragmatismo-dns --show-log - -# Restart DNS -lxc restart pragmatismo-dns - -# Exec into container -lxc exec pragmatismo-dns -- /bin/sh - -# Test DNS -dig @localhost pragmatismo.com.br SOA +short -dig @localhost ddsites.com.br SOA +short -dig @localhost chat.pragmatismo.com.br A +short -``` - -### Update DNS Records - -```bash -# Edit Corefile inside container -lxc exec pragmatismo-dns -- nano /opt/gbo/dns/conf/Corefile - -# Edit zone files inside container -lxc exec pragmatismo-dns -- nano /opt/gbo/dns/data/pragmatismo.com.br.zone - -# Restart CoreDNS after changes -lxc exec pragmatismo-dns -- systemctl restart coredns -# or if running as systemd service: -lxc restart pragmatismo-dns -``` - ---- - -## DNS Cutover - -Update NS/A records at your registrar: -- **NS records:** Point to `ns1.pragmatismo.com.br` / `ns2.pragmatismo.com.br` -- **A records:** Update to `82.29.59.188` - ---- - -## Proxy/Caddy Management (pragmatismo-proxy) - -### Container Info -- **Container Name:** pragmatismo-proxy -- **Service:** Caddy reverse proxy -- **Access:** Inside LXC container on ports 80/443 -- **Config:** `/opt/gbo/proxy/conf/config` inside container - -### Common Operations - -```bash -# Check container status -lxc list | grep pragmatismo-proxy - -# View logs -lxc log pragmatismo-proxy -lxc log pragmatismo-proxy --show-log - -# Restart proxy -lxc restart pragmatismo-proxy - -# Exec into container -lxc exec pragmatismo-proxy -- bash -``` - -### Update Caddy Configuration - -```bash -# Edit Caddyfile inside container -lxc exec pragmatismo-proxy -- nano /opt/gbo/proxy/conf/config - -# Validate configuration -lxc exec pragmatismo-proxy -- caddy validate --config /opt/gbo/proxy/conf/config --adapter caddyfile - -# Reload Caddy after changes -lxc exec pragmatismo-proxy -- caddy reload --config /opt/gbo/proxy/conf/config --adapter caddyfile - -# Or restart the entire container -lxc restart pragmatismo-proxy -``` - ---- - -## ⚠️ Caddy Config — CRITICAL RULES - -**NEVER replace the Caddyfile with a minimal/partial config.** -The full config has ~25 vhosts. - -**Before ANY change:** -1. Backup: `lxc exec pragmatismo-proxy -- cp /opt/gbo/proxy/conf/config /opt/gbo/proxy/conf/config.bak-$(date +%Y%m%d%H%M)` -2. Validate: `lxc exec pragmatismo-proxy -- caddy validate --config /opt/gbo/proxy/conf/config --adapter caddyfile` -3. Reload: `lxc exec pragmatismo-proxy -- caddy reload --config /opt/gbo/proxy/conf/config --adapter caddyfile` - -**Caddy storage must be explicitly set:** -``` -{ - storage file_system { - root /opt/gbo/data/caddy - } -} -``` - ---- - ---- - -## Backup Strategy - -**Location:** Host machine `/opt/gbo/backups/` - -**Format:** `container-name-YYYYMMDD-HHMM.tar.gz` - -**Schedule:** Twice daily (6am and 6pm) via cron - -### Backup Individual Container - -```bash -# Backup container to tar.gz -lxc snapshot pragmatismo-tables backup-$(date +%Y%m%d-%H%M) -lxc publish pragmatismo-tables/backup-$(date +%Y%m%d-%H%M) --alias tables-backup-$(date +%Y%m%d-%H%M) -lxc image export tables-backup-$(date +%Y%m%d-%H%M) /opt/gbo/backups/tables-$(date +%Y%m%d-%H%M).tar.gz - -# List backups -ls -la /opt/gbo/backups/ - -# Restore from backup -lxc image import /opt/gbo/backups/tables-20260326-1227.tar.gz --alias tables-restore -lxc init tables-restore pragmatismo-tables-restored -lxc start pragmatismo-tables-restored -``` - -### Backup PostgreSQL Database - -```bash -# Create database dump -lxc exec pragmatismo-tables -- pg_dump -U postgres -F c -f /tmp/backup.dump pragmatismo - -# Copy backup to host -lxc file pull pragmatismo-tables/tmp/backup.dump /opt/gbo/backups/postgresql-$(date +%Y%m%d).dump -``` - -**Retention:** Last 7 days - ---- - -## Git Workflow - -Push to both remotes: -```bash -cd -git push origin main -git push alm main -cd .. -git add -git commit -m "Update submodule" -git push alm main -``` - ---- - -## Git Workflow - -Push to both remotes: -```bash -cd -git push origin main -git push alm main -cd .. -git add -git commit -m "Update submodule" -git push alm main -``` - ---- - -## Troubleshooting Common Issues - -### Container Not Starting - -```bash -# Check container status -lxc list - -# View logs for startup issues -lxc log pragmatismo- -lxc log pragmatismo- --show-log - -# Check resource usage -lxc info pragmatismo- - -# Try manual start -lxc start pragmatismo- -``` - -### Network Issues - -```bash -# Check container IP -lxc list -c n,4 - -# Test connectivity from host -lxc exec pragmatismo-dns -- ping -c 3 8.8.8.8 - -# Check DNS resolution -lxc exec pragmatismo-system -- dig pragmatismo.com.br -``` - -### Service Issues Inside Container - -```bash -# Check service status inside container -lxc exec pragmatismo- -- systemctl status - -# Restart service inside container -lxc exec pragmatismo- -- systemctl restart - -# View service logs -lxc exec pragmatismo- -- journalctl -u -f -``` - ---- - -(End of file)