# General Bots AI Agent Guidelines

- Use apenas a língua culta ao falar.
- Never save files on root! Use `/tmp` for temp files.
- Never push to ALM without asking first — it is production!
- If in trouble with a tool, go to the official website for install instructions.
- See `botserver/src/drive/local_file_monitor.rs` to load bots from `/opt/gbo/data`.

---

## 📁 Workspace Structure

| Crate | Purpose | Port | Tech Stack |
|-------|---------|------|------------|
| **botserver** | Main API server, business logic | 8080 | Axum, Diesel, Rhai BASIC |
| **botui** | Web UI server (dev) + proxy | 3000 | Axum, HTML/HTMX/CSS |
| **botapp** | Desktop app wrapper | - | Tauri 2 |
| **botlib** | Shared library | - | Core types, errors |
| **botbook** | Documentation | - | mdBook |
| **bottest** | Integration tests | - | tokio-test |
| **botdevice** | IoT/Device support | - | Rust |
| **botplugin** | Browser extension | - | JS |

### Key Paths
- **Binary:** `target/debug/botserver`
- **Run from:** `botserver/` directory
- **Env file:** `botserver/.env`
- **UI Files:** `botui/ui/suite/`
- **Bot data:** `/opt/gbo/data` (primary)
- **Test web:** `http://localhost:3000` — Login: `http://localhost:3000/suite/auth/login.html`

---

## 🧭 LLM Navigation Guide

1. Start with **[Component Dependency Graph](../README.md#-component-dependency-graph)**
2. Review **[Module Responsibility Matrix](../README.md#-module-responsibility-matrix)**
3. Study **[Data Flow Patterns](../README.md#-data-flow-patterns)**
4. Reference **[Common Architectural Patterns](../README.md#-common-architectural-patterns)**
5. Check [Security Rules](#-security-directives---mandatory) — violations are blocking
6. Follow [Code Patterns](#-mandatory-code-patterns) — consistency is mandatory

---

## ❌ Absolute Prohibitions

### Build & Deploy
- ❌ **NEVER** search `/target` folder
- ❌ **NEVER** build in release mode or use `--release`
- ❌ **NEVER** run `cargo build` — use `cargo check` for verification
- ❌ **NEVER** run `cargo clean` — causes 30min rebuilds; use `./reset.sh` for DB issues
- ❌ **NEVER** deploy manually — ALWAYS use CI/CD pipeline (push → ALM → alm-ci builds → deploys)
- ❌ **NEVER** use `scp`, direct SSH binary copy, or manual deployment
- ❌ **NEVER** run the binary directly — use `systemctl` or `./restart.sh`

### Code Quality
- ❌ **NEVER** use `panic!()`, `todo!()`, `unimplemented!()`, `unwrap()`, `expect()`
- ❌ **NEVER** use `Command::new()` directly — use `SafeCommand`
- ❌ **NEVER** return raw error strings to HTTP clients — use `ErrorSanitizer`
- ❌ **NEVER** use `#[allow()]` or lint exceptions in `Cargo.toml` — FIX the code
- ❌ **NEVER** use `_` prefix for unused vars — DELETE or USE them
- ❌ **NEVER** leave unused imports, dead code, or commented-out code
- ❌ **NEVER** use CDN links — all assets must be local
- ❌ **NEVER** create `.md` docs without checking `botbook/` first
- ❌ **NEVER** hardcode credentials — use `generate_random_string()` or env vars

### Security
- ❌ **NEVER** include sensitive data (IPs, tokens, keys) in docs or code
- ❌ **NEVER** write internal IPs to logs — mask them (e.g., "10.x.x.x")
- ❌ **NEVER** create files with secrets in repo root

> **Secret files MUST be placed in `/tmp/` only** (ephemeral, not tracked by git).

---

## 🔐 Security Directives — MANDATORY

### 1. Error Handling — No Panics
```rust
// ❌ FORBIDDEN: unwrap(), expect(), panic!(), todo!()
// ✅ REQUIRED:
value?
value.ok_or_else(|| Error::NotFound)?
value.unwrap_or_default()
if let Some(v) = value { ... }
```

### 2. Command Execution — SafeCommand
```rust
// ❌ FORBIDDEN: Command::new("cmd").arg(user_input).output()
// ✅ REQUIRED:
use crate::security::command_guard::SafeCommand;
SafeCommand::new("allowed_command")?.arg("safe_arg")?.execute()
```

### 3. Error Responses — ErrorSanitizer
```rust
// ❌ FORBIDDEN: Json(json!({ "error": e.to_string() }))
// ✅ REQUIRED:
use crate::security::error_sanitizer::log_and_sanitize;
let sanitized = log_and_sanitize(&e, "context", None);
(StatusCode::INTERNAL_SERVER_ERROR, sanitized)
```

### 4. SQL — sql_guard
```rust
// ❌ FORBIDDEN: format!("SELECT * FROM {}", user_table)
// ✅ REQUIRED:
use crate::security::sql_guard::{sanitize_identifier, validate_table_name};
let safe_table = sanitize_identifier(&user_table);
validate_table_name(&safe_table)?;
```

### 5. Rate Limiting
- General: 100 req/s, Auth: 10 req/s, API: 50 req/s per token, WebSocket: 10 msgs/s
- Use `governor` crate with per-IP and per-User tracking

### 6. CSRF Protection
- ALL state-changing endpoints (POST/PUT/DELETE/PATCH) MUST require CSRF token
- Use `tower_csrf`, bound to user session. Exempt: Bearer Token endpoints

### 7. Security Headers (ALL responses)
`Content-Security-Policy`, `Strict-Transport-Security`, `X-Frame-Options: DENY`, `X-Content-Type-Options: nosniff`, `Referrer-Policy: strict-origin-when-cross-origin`, `Permissions-Policy: geolocation=(), microphone=(), camera=()`

### 8. Dependency Management
- App crates track `Cargo.lock`; lib crates don't
- Critical deps: exact versions (`=1.0.1`); regular: caret (`1.0`)
- Run `cargo audit` weekly; update only via PR with testing

---

## ✅ Mandatory Code Patterns

```rust
impl MyStruct { fn new() -> Self { Self { } } }       // Use Self, not type name
#[derive(PartialEq, Eq)]                               // Always derive both
format!("Hello {name}")                                 // Inline format args
match x { A | B => do_thing(), C => other() }           // Combine identical arms
```

---

## 📏 File Size Limits

- **Max 450 lines per file** — split proactively at 350 lines
- Split by: `types.rs`, `handlers.rs`, `operations.rs`, `utils.rs`, `mod.rs`
- Re-export all public items in `mod.rs`

---

## 🔥 Error Fixing Workflow

### Preferred: Offline Batch Fix
1. Read ENTIRE error list first
2. Group errors by file
3. For each file: view → fix ALL errors → write once
4. Verify with build/diagnostics only AFTER all fixes

### ⚡ Streaming Build Rule
Don't wait for `cargo` to finish — cancel at first errors, fix, re-run.

### 🧠 Memory Issues (process "Killed")
```bash
pkill -9 cargo; pkill -9 rustc; pkill -9 botserver
CARGO_BUILD_JOBS=1 cargo check -p botserver 2>&1 | tail -200
```

---

## 🔄 Reset & Service Management

### reset.sh
- Cleans and restarts dev env (3-5 min bootstrap: Vault, PostgreSQL, Valkey, MinIO, Zitadel, LLM)
- May timeout waiting for Zitadel — check `botserver.log` for "Bootstrap process completed!"

### Verify After Reset
✅ PostgreSQL (5432), ✅ Valkey (6379), ✅ BotServer (8080), ✅ BotUI (3000), ✅ No errors in logs

### Service Commands
```bash
ps aux | grep -E "(botserver|botui)" | grep -v grep
curl http://localhost:8080/health
./restart.sh                        # Restart services
systemctl status|start|stop|restart botserver   # systemd management
journalctl -u botserver -f          # Follow logs
```

---

## 🎭 Playwright Browser Testing

### Browser Setup
If browser fails: `pkill -9 -f brave; pkill -9 -f chrome; pkill -9 -f chromium` → wait 3s → navigate again.

### Bot Testing Flow
1. Navigate to `http://localhost:3000/<botname>`
2. Snapshot → verify welcome message + suggestion buttons + Portuguese accents
3. Click suggestion → wait 3-5s → snapshot → fill data → submit
4. Verify DB records and backend logs

### Desktop UI Note
Chat window may cover other apps — click **middle button** (restore) to minimize, or navigate directly via URL.

### WhatsApp Testing
- Webhook is **global** — bot routing by typing bot name as first message
- Single WhatsApp number serves ALL bots; routing via `whatsapp-id` in `config.csv`

---

## ➕ Adding New Features

### Checklist
- [ ] Which module owns this? (Check Module Responsibility Matrix)
- [ ] Database migrations needed?
- [ ] New API endpoints?
- [ ] Security: input validation, auth, rate limiting, error sanitization?
- [ ] Screens in botui?
- [ ] No `unwrap()`/`expect()`?

### Pattern: types → schema → Diesel model → business logic → API endpoint → BASIC keyword (if applicable) → tests → docs in `botbook/`

### Commit & Deploy
```bash
cd botserver && git push alm main && git push origin main
cd .. && git add botserver && git commit -m "Update botserver: <desc>" && git push alm main && git push origin main
```

---

## 🎨 Frontend Standards

- **HTMX-first** — server returns HTML fragments, not JSON
- **Local assets only** — NO CDN links
- Use `hx-get`, `hx-post`, `hx-target`, `hx-swap`; WebSocket via htmx-ws

---

## 🚀 Performance & Quality

- `cargo clippy --workspace` must pass with **0 warnings**
- `cargo tree --duplicates` / `cargo machete` / `cargo audit` weekly
- Release profile: `opt-level = "z"`, `lto = true`, `codegen-units = 1`, `strip = true`, `panic = "abort"`
- Use `default-features = false` and opt-in to needed features

---

## 🐛 Debugging

### Critical Rule
**STOP on ANY error** — identify → fix root cause → verify → then continue. Never restart to "fix" errors.

### Log Locations
| Component | Log | Prefix |
|-----------|-----|--------|
| botserver | `botserver.log` | — |
| botui | `botui.log` | — |
| drive_monitor | botserver logs | `[drive_monitor]` |
| client errors | botserver logs | `CLIENT:` |

### Bug Fix Flow
1. Reproduce: `grep -E " E | W " botserver.log | tail -20`
2. Trace data flow backwards through call chain
3. Fix minimal change, search for similar occurrences
4. `cargo check -p botserver` → `./restart.sh` → test → check logs
5. Commit with clear root cause description

---

## 🧪 Testing

- **Unit:** per-crate `tests/` or `#[cfg(test)]` modules — `cargo test -p <crate>`
- **Integration:** `bottest/` crate — `cargo test -p bottest`
- **Coverage:** 80%+ on critical paths; ALL error paths and security guards tested

---

## 🚢 Deploy Workflow (CI/CD Only)

1. Push to ALM (triggers CI automatically)
2. CI builds on alm-ci → deploys to system container via SSH
3. Service auto-restarts on binary update
4. Verify: check service status + logs after ~10 min

### Container Architecture

| Container | Service | Port |
|-----------|---------|------|
| system | BotServer + Valkey | 8080/6379 |
| tables | PostgreSQL | 5432 |
| vault | Vault | 8200 |
| directory | Zitadel | 9000 |
| drive | MinIO | 9100 |
| cache | Valkey | 6379 |
| llm | llama.cpp | 8081 |
| vectordb | Qdrant | 6333 |
| meet | LiveKit | 7880 |
| email | Stalwart | 25/587 |
| alm | Forgejo | **4747** (NOT 3000!) |
| alm-ci | Forgejo Runner | — |
| proxy | Caddy | 80/443 |

### Container Management (Incus)
```bash
sudo incus list                              # List all
sudo incus start|stop|restart <container>    # Lifecycle
sudo incus exec <container> -- bash          # Shell access
sudo incus exec <container> -- systemctl restart <service>
sudo incus snapshot create <container> pre-change-$(date +%Y%m%d%H%M%S)
```

---

## 🔑 Core Directives Summary

- **OFFLINE FIRST** — fix all errors from list before compiling
- **BATCH BY FILE** — fix ALL errors in a file at once, write once
- **VERIFY LAST** — only compile after ALL fixes applied
- **DELETE DEAD CODE** — never keep unused code
- **GIT WORKFLOW** — always push to ALL repositories
- **0 warnings, 0 errors** — loop until clean

### 🚨 FUNDAMENTAL: Submodule Push Rule (MANDATORY)

**Every time you push the main repo, you MUST also push ALL submodules!**

```bash
# After ANY main repo push, ALWAYS run:
cd botserver && git push origin main && git push alm main
cd ../botui && git push origin main && git push alm main
cd ../botlib && git push origin main && git push alm main
# ... repeat for ALL submodules
```

**Why:** CI builds based on submodule commits. If submodule isn't pushed, CI deploys old code.

**Checklist before pushing:**
- [ ] botserver pushed?
- [ ] botui pushed?
- [ ] botlib pushed?
- [ ] All other submodules pushed?
- [ ] Main repo points to new submodule commits?

---

## 🔐 Zitadel Setup (Directory Service)

### Container Architecture
- **directory container**: Zitadel running on port **8080** internally
- **tables container**: PostgreSQL database on port 5432
- Use database **PROD-DIRECTORY** for Zitadel data

### Network Access (Container Mode)
- **Internal API**: `http://<directory-ip>:8080`
- **External port 9000** redirected via iptables NAT to directory:8080
- **Health check**: `curl -sf http://localhost:8080/debug/healthz`

### Zitadel Installation Steps

1. **Reset database** (on tables container):
```bash
psql -h localhost -U postgres -d postgres -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = 'PROD-DIRECTORY' AND pid <> pg_backend_pid();"
psql -h localhost -U postgres -d postgres -c "DROP DATABASE IF EXISTS \"PROD-DIRECTORY\";"
psql -h localhost -U postgres -d postgres -c "CREATE DATABASE \"PROD-DIRECTORY\";"
```

2. **Create init config** (on directory container):
```bash
cat > /opt/gbo/conf/directory/zitadel-init-steps.yaml << "EOF"
FirstInstance:
  InstanceName: "BotServer"
  DefaultLanguage: "en"
  PatPath: "/opt/gbo/conf/directory/admin-pat.txt"
  Org:
    Name: "BotServer"
    Machine:
      Machine:
        Username: "admin-sa"
        Name: "Admin Service Account"
      Pat:
        ExpirationDate: "2099-01-01T00:00:00Z"
    Human:
      UserName: "admin"
      FirstName: "Admin"
      LastName: "User"
      Email:
        Address: "admin@localhost"
        Verified: true
      Password: "Admin123!"
      PasswordChangeRequired: false
EOF
```

3. **Start Zitadel** (on directory container):
```bash
pkill -9 zitadel || true
nohup env \
  ZITADEL_DATABASE_POSTGRES_HOST=<tables-ip> \
  ZITADEL_DATABASE_POSTGRES_PORT=5432 \
  ZITADEL_DATABASE_POSTGRES_DATABASE=PROD-DIRECTORY \
  ZITADEL_DATABASE_POSTGRES_USER_USERNAME=postgres \
  ZITADEL_DATABASE_POSTGRES_USER_PASSWORD=postgres \
  ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=disable \
  ZITADEL_DATABASE_POSTGRES_ADMIN_USERNAME=postgres \
  ZITADEL_DATABASE_POSTGRES_ADMIN_PASSWORD=postgres \
  ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE=disable \
  ZITADEL_EXTERNALSECURE=false \
  ZITADEL_EXTERNALDOMAIN=<directory-ip> \
  ZITADEL_EXTERNALPORT=9000 \
  ZITADEL_TLS_ENABLED=false \
  /opt/gbo/bin/zitadel start-from-init \
    --masterkey MasterkeyNeedsToHave32Characters \
    --tlsMode disabled \
    --externalDomain <directory-ip> \
    --externalPort 9000 \
    --steps /opt/gbo/conf/directory/zitadel-init-steps.yaml \
  > /opt/gbo/logs/zitadel.log 2>&1 &
```

4. **Wait for bootstrap** (~90 seconds), then verify:
```bash
curl -sf http://localhost:8080/debug/healthz
cat /opt/gbo/conf/directory/admin-pat.txt
```

5. **Configure iptables** (on system container):
```bash
iptables -t nat -A PREROUTING -p tcp --dport 9000 -j DNAT --to-destination <directory-ip>:8080
iptables -t nat -A OUTPUT -p tcp -d <external-ip> --dport 9000 -j DNAT --to-destination <directory-ip>:8080
```

### Zitadel API Usage

**PAT file location**: `/opt/gbo/conf/directory/admin-pat.txt` (on directory container)

#### Get IAM Info (internal)
```bash
curl -s -H "Authorization: Bearer $PAT" http://<directory-ip>:8080/management/v1/iam
```

#### Get IAM Info (external via port 9000)
```bash
curl -s -H "Authorization: Bearer $PAT" -H "Host: <directory-ip>" http://<external-ip>:9000/management/v1/iam
```

#### Create Human User
```bash
curl -s -X POST \
  -H "Authorization: Bearer $PAT" \
  -H "Host: <directory-ip>" \
  -H "Content-Type: application/json" \
  http://<external-ip>:9000/management/v1/users/human \
  -d '{
    "userName": "janedoe",
    "name": "Jane Doe",
    "profile": {"firstName": "Jane", "lastName": "Doe"},
    "email": {"email": "jane@example.com"}
  }'
```

### Zitadel API Endpoints Reference

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/management/v1/iam` | GET | Get IAM info |
| `/management/v1/orgs/me` | GET | Get current org |
| `/management/v1/users/human` | POST | Create human user |
| `/management/v1/users/machine` | POST | Create machine user |
| `/oauth/v2/token` | POST | Get access token |
| `/debug/healthz` | GET | Health check |

### Important Notes

- **Zitadel listens on port 8080 internally**
- **External port 9000** is forwarded via iptables NAT
- **Use Host header** with directory IP for external API calls
- **PAT file**: `/opt/gbo/conf/directory/admin-pat.txt`
- **Admin credentials**: `admin` / `Admin123!` (human user)
- **Database**: `PROD-DIRECTORY` on tables container
- **Zitadel v4.13.1** is the current version