Commit graph

246 commits

Author SHA1 Message Date
9e799dd6b1 Disable /opt/gbo/data loading, use drive (MinIO) only for bot sources
Some checks failed
BotServer CI/CD / build (push) Failing after 8m28s
- Remove LocalFileMonitor and ConfigWatcher for /opt/gbo/data
- Remove /opt/gbo/data from mount_all_bots() scanning
- Change start.bas, tables.bas, and tool paths to use work directory
- Filter drive buckets to only gbo-* prefix
- Remove unused create_bot_simple method
- Fix all warnings (unused imports, variables, dead code)
2026-04-08 16:55:50 -03:00
9b04af9e7b Fix USE KB and USE WEBSITE default features compilation
Some checks failed
BotServer CI/CD / build (push) Failing after 10m2s
2026-04-07 20:14:12 -03:00
73002b36cc Update botserver: various fixes and improvements
All checks were successful
BotServer CI/CD / build (push) Successful in 9m59s
2026-04-07 13:33:50 -03:00
90c14bcd09 Fix DETECT: use bot-specific DB pool, add anonymous auth when directory disabled
All checks were successful
BotServer CI/CD / build (push) Successful in 12m42s
2026-04-06 13:37:23 -03:00
15d9e3c142 fix(deploy): Ignore error if /opt/gbo/bin/botserver doesn't exist on first deploy
All checks were successful
BotServer CI/CD / build (push) Successful in 5m7s
2026-04-05 19:38:54 -03:00
86bb4cad8e fix(botserver): Handle TOOL_EXEC message type for direct tool execution without KB/LLM
Some checks failed
BotServer CI/CD / build (push) Failing after 5m40s
2026-04-05 19:11:04 -03:00
f6869e6b5c Fix diesel join queries across schemas and FileItem missing fields
Some checks failed
BotServer CI/CD / build (push) Failing after 10m1s
2026-04-05 12:06:35 -03:00
155d465b14 Update botserver: Refactor groups module, add Knowledge Base group association logic, and implement Drive tags for KB access.
Some checks failed
BotServer CI/CD / build (push) Failing after 5m53s
2026-04-05 09:11:54 -03:00
7a5f858d86 fix: remove catch_unwind, add error logging
All checks were successful
BotServer CI/CD / build (push) Successful in 4m22s
2026-04-05 00:26:44 -03:00
b277b032b0 fix: catch panics in send_mail thread
Some checks failed
BotServer CI/CD / build (push) Failing after 1m16s
2026-04-05 00:22:26 -03:00
61ae0cac82 fix: log EmailService::send_email errors instead of silently discarding
All checks were successful
BotServer CI/CD / build (push) Successful in 4m26s
2026-04-04 19:03:19 -03:00
3f3ecc5320 fix: make email tracking and draft saving non-fatal for bots without those tables
All checks were successful
BotServer CI/CD / build (push) Successful in 4m43s
2026-04-04 18:13:46 -03:00
45eb8357cb feat: implement real email sending via lettre + Vault credentials
Some checks failed
BotServer CI/CD / build (push) Failing after 8m6s
- Replace EmailService::send_email stub with full lettre SMTP implementation
- Vault resolution chain: bot-specific → default bot → system fallback
- Seed Vault prod with default email config (contato@pragmatismo.com.br)
- Update all call sites to pass bot_id for Vault lookup
- Support attachments via lettre MultiPart/Attachment API
- Remove unused imports and dead code
2026-04-04 17:16:50 -03:00
0de4565e5a refactor: Generalize WhatsAppAdapter::new to accept &AppState
All checks were successful
BotServer CI/CD / build (push) Successful in 4m55s
- Simplify constructor from (pool, bot_id, cache) to (&state, bot_id)
- Adapter now extracts conn and cache from AppState internally
- Updates 15 call sites across 6 files
- Removes redundant parameter plumbing at every call site
2026-04-04 15:46:10 -03:00
62e9a64340 fix: Remove duplicate code in table_migration.rs
All checks were successful
BotServer CI/CD / build (push) Successful in 4m27s
- Removed duplicate DbColumn struct, PROTECTED_COLUMNS const, and sync_table_schema fn
- File now has single clean implementation with column drop protection
2026-04-04 11:11:22 -03:00
d785d255c6 feat: Add column drop protection in table schema sync
Some checks failed
BotServer CI/CD / build (push) Failing after 2m30s
- Add columns_dropped counter to MigrationResult
- Add PROTECTED_COLUMNS list (id, bot_id, org_id, user_id, created_at, etc.)
- Detect orphaned columns (in DB but not in tables.bas) and drop them
- Protected columns are never dropped automatically
- Uses DROP COLUMN IF EXISTS for safety
- Logs warnings for orphaned columns before dropping
2026-04-04 11:02:47 -03:00
44669c3825 fix: Fix resolve_export_path typo and remove unused PathBuf imports
All checks were successful
BotServer CI/CD / build (push) Successful in 4m28s
2026-04-04 10:23:42 -03:00
be6f0306cc fix: Fix remaining E0716 borrow errors in path refactoring
Some checks failed
BotServer CI/CD / build (push) Has been cancelled
- server.rs: Use PathBuf for cert_dir
- auth_routes.rs: Use PathBuf for pat_path
- qrcode.rs: Bind get_work_path() to local var before unwrap_or
- import_export.rs: Bind get_work_path() to local var in both functions (2 occurrences)
2026-04-04 10:13:40 -03:00
7d8f141fc2 refactor: Replace all hardcoded ./botserver-stack paths with get_stack_path()/get_work_path()
Some checks failed
BotServer CI/CD / build (push) Failing after 1m28s
- Adds get_stack_path() helper: returns /opt/gbo in production (.env without botserver-stack), ./botserver-stack in dev
- Adds get_work_path() helper: returns /opt/gbo/work in production, ./botserver-stack/data/system/work in dev
- Updated 35+ files to use dynamic path resolution
- Production system container no longer needs botserver-stack directory
- Work files go to /opt/gbo/work instead of /opt/gbo/bin/botserver-stack
2026-04-04 09:24:44 -03:00
6ec82c27a6 fix: Replace futures::executor::block_on with thread::spawn in SET USER
All checks were successful
BotServer CI/CD / build (push) Successful in 4m25s
- Fixes panic: Cannot start a runtime from within a runtime
- set_user.rs was using futures::executor::block_on directly in Rhai callback
- Now uses std:🧵:spawn + new_current_thread().block_on() pattern
- This is called during bootstrap and was causing startup crash
2026-04-04 08:01:04 -03:00
2a042d400b fix: Replace Handle::try_current().block_on() with thread::spawn pattern
All checks were successful
BotServer CI/CD / build (push) Successful in 2m38s
- Fixes panic: Cannot start a runtime from within a runtime
- kb_statistics.rs: Wrap all async calls in std:🧵:spawn
- post_to.rs: Replace Handle::try_current with thread::spawn + mpsc
- Removes dead Handle::try_current checks from sync functions
- Follows AGENTS.md pattern for async-from-sync callbacks
2026-04-04 07:35:03 -03:00
4d7297243e Fix clippy warnings: reduce 17 warnings to 0
All checks were successful
BotServer CI/CD / build (push) Successful in 6m58s
- Fix double_ended_iterator_last: use next_back() instead of last()
- Fix manual_clamp: use .clamp() instead of min().max()
- Fix too_many_arguments: create KbInjectionContext struct
- Fix needless_borrow: remove unnecessary & reference
- Fix let_and_return: return value directly
- Fix await_holding_lock: drop guard before await
- Fix collapsible_else_if: collapse nested if-else

All changes verified with cargo clippy (0 warnings, 0 errors)
Note: Local botserver crashes with existing panic during LocalFileMonitor initialization
This panic exists in original code too, not caused by these changes
2026-04-03 22:34:43 -03:00
eb98574c8a fix(runtime): use TransferResult instead of Result in transfer_to_human
All checks were successful
BotServer CI/CD / build (push) Successful in 5m19s
2026-04-03 20:49:17 -03:00
3f94d23e1f fix(runtime): replace Handle::current().block_on() with std:🧵:spawn in transfer_to_human
Some checks failed
BotServer CI/CD / build (push) Failing after 1m18s
- Handle::current().block_on() panics when called from within a runtime
- replaced all 5 occurrences with std:🧵:spawn + mpsc::channel
- matches the pattern already used across other keyword files
2026-04-03 20:43:48 -03:00
8019107ebf fix: remove last remaining block_in_place in TALK TO keyword
Some checks failed
BotServer CI/CD / build (push) Failing after 19m14s
This was the only block_in_place left causing the production panic during
bot compilation. Replaced with std:🧵:spawn + mpsc channel pattern.
2026-04-03 18:35:27 -03:00
6f183c63d2 feat: dual-mode service configs - Vault first, fallback to DB/localhost
Some checks failed
BotServer CI/CD / build (push) Has been cancelled
All services now try Vault first (remote/distributed mode), then fall back
to database config, then localhost defaults (local/dev mode).

Services fixed:
- Qdrant/VectorDB: kb_indexer.rs, kb_statistics.rs, bootstrap_utils.rs, kb_context.rs
- LLM/Embedding: email/vectordb.rs (was hardcoded localhost:8082)
- All services: security/integration.rs (postgres, cache, drive, directory, qdrant, llm)

Pattern: SecretsManager::get_X_config_sync() → DB config → localhost default
2026-04-03 15:01:37 -03:00
61642343a8 fix: replace all block_in_place with std:🧵:spawn to fix nested runtime panic
Some checks are pending
BotServer CI/CD / build (push) Waiting to run
Root cause: block_in_place + new_current_thread().block_on() panics when
called from within tokio runtime (including spawn_blocking). Tokio doesn't
allow nested block_on() calls.

Fix: Replace ALL block_in_place patterns with std:🧵:spawn + mpsc channel.
This creates a completely separate OS thread with its own runtime, avoiding
any nesting issues. Works from any context: async, spawn_blocking, or sync.

Files: 14 files across secrets, utils, state, calendar, analytics, email,
and all keyword handlers (universal_messaging, search, book, create_draft,
create_site, hearing/syntax, use_tool, find, admin_email, goals)
2026-04-03 12:54:36 -03:00
f6a864aa67 fix: replace nested runtime block_on with new_current_thread to prevent panic
All checks were successful
BotServer CI/CD / build (push) Successful in 5m32s
Root cause: Handle::current().block_on() panics inside multi_thread runtime
with 'Cannot start a runtime from within a runtime' error.

Fix: All sync-to-async bridges now use tokio::runtime::Builder::new_current_thread()
instead of Handle::current().block_on(). Also changed SECRETS_MANAGER from
tokio::sync::RwLock to std::sync::RwLock to eliminate unnecessary async overhead.

Files: 14 files across keywords, secrets, utils, state, calendar, analytics, email
Impact: Fixes production crash during bot loading phase
2026-04-03 09:17:23 -03:00
e992ed3b39 Enforce Vault-only secrets: remove env var fallbacks, all secrets from Vault
Some checks are pending
BotServer CI/CD / build (push) Waiting to run
- Remove all std::env::var calls except VAULT_* and PORT
- get_from_env returns hardcoded defaults only (no env var reading)
- Auth config, rate limits, email, analytics, calendar all use Vault
- WORK_PATH replaced with get_work_path() helper reading from Vault
- .env on production cleaned to only VAULT_ADDR, VAULT_TOKEN, VAULT_CACERT, PORT
- All service IPs/credentials stored in Vault secret/gbo/*
2026-04-03 07:11:40 -03:00
fb2e5242da fix: Vault seeding, service health checks, and restart idempotency
All checks were successful
BotServer CI/CD / build (push) Successful in 55m52s
- Replace hardcoded passwords with generate_random_string() for all Vault-seeded services
- Add valkey-cli, nc to SafeCommand allowlist; fix PATH in all 4 execution methods
- Fix empty Vault KV values ('none' placeholder) preventing 'Failed to parse K=V' errors
- Fix special chars in generated passwords triggering shell injection false positives
- Add ALM app.ini creation with absolute paths for Forgejo CLI
- Increase Qdrant timeout 15s→45s, ALM wait 5s→20s
- Persist file_states and kb_states to disk for .bas/KB idempotency across restarts
- Add duplicate check to use_website registration (debug log for existing)
- Remove dead code (SERVER_START_EPOCH, server_epoch)
- Add generate_random_string() to shared mod.rs, remove duplicates
2026-04-01 12:22:57 -03:00
adb26330d2 fix: Simple 50ms timeout for Redis connection 2026-03-21 10:48:47 -03:00
9d6c2686f1 fix: Remove connection caching (no Clone) 2026-03-21 10:37:49 -03:00
b3ce293487 fix: Clean up duplicate Redis code and fix WebSocket log level 2026-03-21 10:30:19 -03:00
cfe6453d1e perf: Add shared Redis connection pool with 50ms timeout 2026-03-21 10:14:10 -03:00
43fd40aed9 fix: Add timeout to Redis get_connection to prevent blocking
- Added get_redis_connection() helper with 2s timeout
- All cache operations now fail fast if Valkey is not ready
- Prevents start.bas from blocking for minutes waiting for cache
- Changes: add_suggestion.rs
2026-03-21 09:34:41 -03:00
e5f3380469 perf: Fix USE TOOL thread contention by removing runtime creation
- Replace thread spawn + tokio runtime creation with block_in_place
- Eliminates 10+ runtime creations per start.bas execution
- Reduces USE TOOL execution from ~2min to milliseconds
- Fixes suggestions not appearing due to start.bas timeout
2026-03-20 22:54:19 -03:00
d19984fa07 feat: Improve KB keywords and package manager installer 2026-03-20 17:38:47 -03:00
3bb115266b feat: Add GUID prefix to Qdrant collection names for KB security isolation 2026-03-19 19:51:28 -03:00
d6ebd0cf6e fix: send suggestions separately from TALK, clear Redis keys for refresh
- Remove suggestions fetching from TALK function
- WebSocket handler now fetches and sends suggestions after start.bas executes
- Clear suggestions and start_bas_executed keys to allow re-run on refresh
- Decouple TALK from suggestions handling
2026-03-19 09:53:39 -03:00
2fcfb05fd6 fix: USE_WEBSITE non-blocking - timeout 3s, never blocks start.bas 2026-03-18 19:41:23 -03:00
1a9208b88e Fix: Use bot_id instead of user_id in TALK suggestions Redis key 2026-03-18 11:05:56 -03:00
ec4fcc094a Fix: Use bot_id instead of user_id in suggestion Redis keys
- Changed all suggestion key formats from suggestions:user_id:session_id to suggestions:bot_id:session_id
- Fixes bug where suggestions were stored under wrong key, preventing frontend from retrieving them
- Affects: CLEAR SUGGESTIONS, ADD SUGGESTION, ADD SUGGESTION TEXT, ADD_SUGGESTION_TOOL
- Impact: Suggestions now correctly associated with bot, not user
2026-03-18 10:39:27 -03:00
af7441ebcb fix: generate mcp.json for tools without PARAM declarations
Tools using only USE KB or other keywords without PARAM were not
getting .mcp.json generated, causing USE TOOL to silently skip them.
2026-03-17 12:20:47 -03:00
ab1f2df476 Read Drive config from Vault at runtime with fallback defaults
Some checks failed
BotServer CI / build (push) Failing after 7m26s
2026-03-17 00:00:36 -03:00
b57c53e2ff Remove WORKFLOW_PLAN.md (moved to gb/prompts)
Some checks failed
BotServer CI / build (push) Failing after 7m22s
2026-03-16 23:40:56 -03:00
7849031ffe Move WORKFLOW_PLAN.md to src/basic/
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-03-16 23:38:35 -03:00
ec1e203859 HEAR: add configurable timeout (hear-timeout-secs, default 1h)
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-03-16 23:12:45 -03:00
dae78479d2 Fix HEAR: block thread on channel instead of throw+re-run from top
Some checks failed
BotServer CI / build (push) Failing after 6m54s
- Add hear_channels: HashMap<Uuid, SyncSender<String>> to AppState
- HEAR now blocks the spawn_blocking thread via sync_channel recv()
- deliver_hear_input() called at top of stream_response() to unblock
- Script continues from exact HEAR position, no side-effect re-execution
- All three HEAR variants (basic, AS TYPE, AS MENU) use same mechanism
2026-03-16 23:00:17 -03:00
6c139fdf63 fix: suggestion and talk keyword improvements
All checks were successful
BotServer CI / build (push) Successful in 13m59s
2026-03-16 13:29:14 -03:00
f062cc4a3e Add THINK KB keyword for explicit knowledge base reasoning
- Implement think_kb.rs with structured search results
- Add keyword registration in mod.rs files
- Include confidence scoring and multi-KB support
2026-03-16 08:15:21 -03:00