Commit graph

263 commits

Author SHA1 Message Date
25d6d2fd57 fix: eliminate LLM keyword deadlock with isolated worker thread
All checks were successful
BotServer CI/CD / build (push) Successful in 3m32s
The previous fix used Handle::current().block_on() which deadlocks when
the Rhai engine runs on a Tokio worker thread — it blocks the very
thread the async task needs to make progress.

New approach: spawn a dedicated background thread with its own
single-threaded Tokio runtime, communicate via mpsc channel with a
45s timeout. This completely isolates the LLM runtime from the
caller's runtime, eliminating any possibility of thread starvation
or nested-runtime deadlock.

Also remove unused 'trace' import from llm/mod.rs.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-04-13 23:20:10 -03:00
b3fd90b056 fix: remove blocking recv_timeout from LLM keyword
All checks were successful
BotServer CI/CD / build (push) Successful in 3m41s
2026-04-13 23:01:54 -03:00
8a65afbfc5 feat: add [BASIC_EXEC] traces for start, tool, scheduler, webhook execution
All checks were successful
BotServer CI/CD / build (push) Successful in 3m18s
2026-04-13 18:16:01 -03:00
6d987c0eea feat: add ADD_SWITCHER keyword with underscore preprocessing
All checks were successful
BotServer CI/CD / build (push) Successful in 3m25s
Implement ADD_SWITCHER keyword following the same pattern as ADD_SUGGESTION_TOOL:
- Created switcher.rs module with add_switcher_keyword() and clear_switchers_keyword()
- Added preprocessing to convert "ADD SWITCHER" to "ADD_SWITCHER"
- Added to keyword patterns and get_all_keywords()
- Stores switcher suggestions in Redis with type "switcher" and action "switch_context"
- Supports both "ADD SWITCHER" and "ADD_SWITCHER" syntax

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-04-13 15:47:21 -03:00
765bd624f4 fix: use Redis SADD instead of RPUSH for suggestions to prevent duplicate buttons
All checks were successful
BotServer CI/CD / build (push) Successful in 3m11s
SADD stores suggestions in a set (deduplicated) instead of a list (accumulates).
get_suggestions now uses SMEMBERS instead of LRANGE. Removed the TODO about
clearing suggestions since SADD inherently prevents duplicates.
2026-04-13 14:09:57 -03:00
f8b47d1ac2 refactor: unify BASIC compilation into BasicCompiler only, runtime uses ScriptService::run() on pre-compiled .ast
Some checks failed
BotServer CI/CD / build (push) Has been cancelled
- Move all preprocessing transforms (convert_multiword_keywords, preprocess_llm_keyword,
  convert_while_wend_syntax, predeclare_variables) into BasicCompiler::preprocess_basic
  so .ast files are fully preprocessed by Drive Monitor
- Replace ScriptService compile/compile_preprocessed/compile_tool_script with
  single run(ast_content) that does engine.compile() + eval_ast_with_scope()
- Remove .bas fallback in tool_executor and start.bas paths - .ast only
- Remove dead code: preprocess_basic_script, normalize_variables_to_lowercase,
  convert_save_for_tools, parse_save_parts, normalize_word
- Fix: USE KB 'cartas' in tool .ast now correctly converted to USE_KB('cartas')
  during compilation, ensuring KB context injection works after tool execution
- Fix: add trace import in llm/mod.rs
2026-04-13 14:05:55 -03:00
782618e265 fix: ADD_SUGGESTION compilation error - AS keyword case mismatch
All checks were successful
BotServer CI/CD / build (push) Successful in 2m50s
Root cause: compiler converts AS -> as (lowercase keywords) but Rhai
custom syntax expected uppercase 'AS'. Rhai syntax is case-sensitive.

Changed:
- ADD_SUGGESTION_TOOL: 'AS' -> 'as'
- ADD_SUGGESTION_TEXT: 'AS' -> 'as'
- ADD_SUGGESTION: 'AS' -> 'as'

This fixes: 'Syntax error: Expecting AS for ADD_SUGGESTION_TOOL expression'

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-04-13 07:40:16 -03:00
c6a47c84ac fix: use ADD_SUGGESTION_TOOL instead of ADD_SUGG_TOOL
All checks were successful
BotServer CI/CD / build (push) Successful in 3m17s
2026-04-12 18:33:34 -03:00
20af25e9e2 fix: use compile_preprocessed for .ast files
All checks were successful
BotServer CI/CD / build (push) Successful in 3m29s
2026-04-12 17:48:41 -03:00
9de9bc983b Fix ADD SUGGESTION preprocessor transform to avoid ADD BOT keyword conflict
All checks were successful
BotServer CI/CD / build (push) Successful in 3m20s
2026-04-12 15:32:31 -03:00
78130caaa1 chore: add clarification comment for closure clones
All checks were successful
BotServer CI/CD / build (push) Successful in 2m3s
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-04-12 15:07:47 -03:00
e34481b7f8 fix: separate clones for each closure to satisfy borrow checker
All checks were successful
BotServer CI/CD / build (push) Successful in 2m54s
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-04-12 14:26:31 -03:00
ff6f2200f0 fix: correct cache3/user_session3 reference in ADD_SUGG handler
Some checks failed
BotServer CI/CD / build (push) Failing after 45s
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-04-12 14:21:29 -03:00
74ac734253 fix: use single-token ADD_SUGG_TOOL to avoid ADD keyword conflicts
Some checks failed
BotServer CI/CD / build (push) Failing after 4m0s
- Replace ADD SUGGESTION TOOL with ADD_SUGG_TOOL (single token)
- Replace ADD SUGGESTION TEXT with ADD_SUGG_TEXT
- Replace ADD SUGGESTION with ADD_SUGG
- Keep ADD_SUGGESTION_TOOL as legacy alias for backward compat
- Preprocessor converts ADD SUGGESTION TOOL -> ADD_SUGG_TOOL automatically
- Eliminates collision with ADD BOT, ADD MEMBER in Rhai parser

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-04-12 14:14:03 -03:00
7a1ec157f1 Fix KB indexing: upsert kb_collections, consistent collection names, preserve indexed flag
All checks were successful
BotServer CI/CD / build (push) Successful in 3m23s
- Bug 1: check_gbkb_changes now preserves indexed=true from previous
  state when etag matches, preventing redundant re-indexing every cycle
- Bug 2: USE KB fallback uses bot_id_short (8 chars) instead of random
  UUID, matching the collection name convention used by DriveMonitor
- Bug 3: handle_gbkb_change now upserts into kb_collections table after
  successful indexing, so USE KB can find the collection at runtime
- Changed ON CONFLICT DO NOTHING to DO UPDATE for kb_collections inserts
- Changed process_gbkb_folder return type to Result<IndexingResult>
2026-04-11 21:26:02 -03:00
f4c99030aa fix: use get_work_path() instead of get_stack_path()+data/system for work dir, add etag check for PROMPT.md downloads
All checks were successful
BotServer CI/CD / build (push) Successful in 3m37s
2026-04-11 18:42:09 -03:00
5576378b3f Update botserver: Multiple improvements across core modules
All checks were successful
BotServer CI/CD / build (push) Successful in 10m41s
2026-04-11 07:33:32 -03:00
9e799dd6b1 Disable /opt/gbo/data loading, use drive (MinIO) only for bot sources
Some checks failed
BotServer CI/CD / build (push) Failing after 8m28s
- Remove LocalFileMonitor and ConfigWatcher for /opt/gbo/data
- Remove /opt/gbo/data from mount_all_bots() scanning
- Change start.bas, tables.bas, and tool paths to use work directory
- Filter drive buckets to only gbo-* prefix
- Remove unused create_bot_simple method
- Fix all warnings (unused imports, variables, dead code)
2026-04-08 16:55:50 -03:00
9b04af9e7b Fix USE KB and USE WEBSITE default features compilation
Some checks failed
BotServer CI/CD / build (push) Failing after 10m2s
2026-04-07 20:14:12 -03:00
73002b36cc Update botserver: various fixes and improvements
All checks were successful
BotServer CI/CD / build (push) Successful in 9m59s
2026-04-07 13:33:50 -03:00
90c14bcd09 Fix DETECT: use bot-specific DB pool, add anonymous auth when directory disabled
All checks were successful
BotServer CI/CD / build (push) Successful in 12m42s
2026-04-06 13:37:23 -03:00
15d9e3c142 fix(deploy): Ignore error if /opt/gbo/bin/botserver doesn't exist on first deploy
All checks were successful
BotServer CI/CD / build (push) Successful in 5m7s
2026-04-05 19:38:54 -03:00
86bb4cad8e fix(botserver): Handle TOOL_EXEC message type for direct tool execution without KB/LLM
Some checks failed
BotServer CI/CD / build (push) Failing after 5m40s
2026-04-05 19:11:04 -03:00
f6869e6b5c Fix diesel join queries across schemas and FileItem missing fields
Some checks failed
BotServer CI/CD / build (push) Failing after 10m1s
2026-04-05 12:06:35 -03:00
155d465b14 Update botserver: Refactor groups module, add Knowledge Base group association logic, and implement Drive tags for KB access.
Some checks failed
BotServer CI/CD / build (push) Failing after 5m53s
2026-04-05 09:11:54 -03:00
7a5f858d86 fix: remove catch_unwind, add error logging
All checks were successful
BotServer CI/CD / build (push) Successful in 4m22s
2026-04-05 00:26:44 -03:00
b277b032b0 fix: catch panics in send_mail thread
Some checks failed
BotServer CI/CD / build (push) Failing after 1m16s
2026-04-05 00:22:26 -03:00
61ae0cac82 fix: log EmailService::send_email errors instead of silently discarding
All checks were successful
BotServer CI/CD / build (push) Successful in 4m26s
2026-04-04 19:03:19 -03:00
3f3ecc5320 fix: make email tracking and draft saving non-fatal for bots without those tables
All checks were successful
BotServer CI/CD / build (push) Successful in 4m43s
2026-04-04 18:13:46 -03:00
45eb8357cb feat: implement real email sending via lettre + Vault credentials
Some checks failed
BotServer CI/CD / build (push) Failing after 8m6s
- Replace EmailService::send_email stub with full lettre SMTP implementation
- Vault resolution chain: bot-specific → default bot → system fallback
- Seed Vault prod with default email config (contato@pragmatismo.com.br)
- Update all call sites to pass bot_id for Vault lookup
- Support attachments via lettre MultiPart/Attachment API
- Remove unused imports and dead code
2026-04-04 17:16:50 -03:00
0de4565e5a refactor: Generalize WhatsAppAdapter::new to accept &AppState
All checks were successful
BotServer CI/CD / build (push) Successful in 4m55s
- Simplify constructor from (pool, bot_id, cache) to (&state, bot_id)
- Adapter now extracts conn and cache from AppState internally
- Updates 15 call sites across 6 files
- Removes redundant parameter plumbing at every call site
2026-04-04 15:46:10 -03:00
62e9a64340 fix: Remove duplicate code in table_migration.rs
All checks were successful
BotServer CI/CD / build (push) Successful in 4m27s
- Removed duplicate DbColumn struct, PROTECTED_COLUMNS const, and sync_table_schema fn
- File now has single clean implementation with column drop protection
2026-04-04 11:11:22 -03:00
d785d255c6 feat: Add column drop protection in table schema sync
Some checks failed
BotServer CI/CD / build (push) Failing after 2m30s
- Add columns_dropped counter to MigrationResult
- Add PROTECTED_COLUMNS list (id, bot_id, org_id, user_id, created_at, etc.)
- Detect orphaned columns (in DB but not in tables.bas) and drop them
- Protected columns are never dropped automatically
- Uses DROP COLUMN IF EXISTS for safety
- Logs warnings for orphaned columns before dropping
2026-04-04 11:02:47 -03:00
44669c3825 fix: Fix resolve_export_path typo and remove unused PathBuf imports
All checks were successful
BotServer CI/CD / build (push) Successful in 4m28s
2026-04-04 10:23:42 -03:00
be6f0306cc fix: Fix remaining E0716 borrow errors in path refactoring
Some checks failed
BotServer CI/CD / build (push) Has been cancelled
- server.rs: Use PathBuf for cert_dir
- auth_routes.rs: Use PathBuf for pat_path
- qrcode.rs: Bind get_work_path() to local var before unwrap_or
- import_export.rs: Bind get_work_path() to local var in both functions (2 occurrences)
2026-04-04 10:13:40 -03:00
7d8f141fc2 refactor: Replace all hardcoded ./botserver-stack paths with get_stack_path()/get_work_path()
Some checks failed
BotServer CI/CD / build (push) Failing after 1m28s
- Adds get_stack_path() helper: returns /opt/gbo in production (.env without botserver-stack), ./botserver-stack in dev
- Adds get_work_path() helper: returns /opt/gbo/work in production, ./botserver-stack/data/system/work in dev
- Updated 35+ files to use dynamic path resolution
- Production system container no longer needs botserver-stack directory
- Work files go to /opt/gbo/work instead of /opt/gbo/bin/botserver-stack
2026-04-04 09:24:44 -03:00
6ec82c27a6 fix: Replace futures::executor::block_on with thread::spawn in SET USER
All checks were successful
BotServer CI/CD / build (push) Successful in 4m25s
- Fixes panic: Cannot start a runtime from within a runtime
- set_user.rs was using futures::executor::block_on directly in Rhai callback
- Now uses std:🧵:spawn + new_current_thread().block_on() pattern
- This is called during bootstrap and was causing startup crash
2026-04-04 08:01:04 -03:00
2a042d400b fix: Replace Handle::try_current().block_on() with thread::spawn pattern
All checks were successful
BotServer CI/CD / build (push) Successful in 2m38s
- Fixes panic: Cannot start a runtime from within a runtime
- kb_statistics.rs: Wrap all async calls in std:🧵:spawn
- post_to.rs: Replace Handle::try_current with thread::spawn + mpsc
- Removes dead Handle::try_current checks from sync functions
- Follows AGENTS.md pattern for async-from-sync callbacks
2026-04-04 07:35:03 -03:00
4d7297243e Fix clippy warnings: reduce 17 warnings to 0
All checks were successful
BotServer CI/CD / build (push) Successful in 6m58s
- Fix double_ended_iterator_last: use next_back() instead of last()
- Fix manual_clamp: use .clamp() instead of min().max()
- Fix too_many_arguments: create KbInjectionContext struct
- Fix needless_borrow: remove unnecessary & reference
- Fix let_and_return: return value directly
- Fix await_holding_lock: drop guard before await
- Fix collapsible_else_if: collapse nested if-else

All changes verified with cargo clippy (0 warnings, 0 errors)
Note: Local botserver crashes with existing panic during LocalFileMonitor initialization
This panic exists in original code too, not caused by these changes
2026-04-03 22:34:43 -03:00
eb98574c8a fix(runtime): use TransferResult instead of Result in transfer_to_human
All checks were successful
BotServer CI/CD / build (push) Successful in 5m19s
2026-04-03 20:49:17 -03:00
3f94d23e1f fix(runtime): replace Handle::current().block_on() with std:🧵:spawn in transfer_to_human
Some checks failed
BotServer CI/CD / build (push) Failing after 1m18s
- Handle::current().block_on() panics when called from within a runtime
- replaced all 5 occurrences with std:🧵:spawn + mpsc::channel
- matches the pattern already used across other keyword files
2026-04-03 20:43:48 -03:00
8019107ebf fix: remove last remaining block_in_place in TALK TO keyword
Some checks failed
BotServer CI/CD / build (push) Failing after 19m14s
This was the only block_in_place left causing the production panic during
bot compilation. Replaced with std:🧵:spawn + mpsc channel pattern.
2026-04-03 18:35:27 -03:00
6f183c63d2 feat: dual-mode service configs - Vault first, fallback to DB/localhost
Some checks failed
BotServer CI/CD / build (push) Has been cancelled
All services now try Vault first (remote/distributed mode), then fall back
to database config, then localhost defaults (local/dev mode).

Services fixed:
- Qdrant/VectorDB: kb_indexer.rs, kb_statistics.rs, bootstrap_utils.rs, kb_context.rs
- LLM/Embedding: email/vectordb.rs (was hardcoded localhost:8082)
- All services: security/integration.rs (postgres, cache, drive, directory, qdrant, llm)

Pattern: SecretsManager::get_X_config_sync() → DB config → localhost default
2026-04-03 15:01:37 -03:00
61642343a8 fix: replace all block_in_place with std:🧵:spawn to fix nested runtime panic
Some checks are pending
BotServer CI/CD / build (push) Waiting to run
Root cause: block_in_place + new_current_thread().block_on() panics when
called from within tokio runtime (including spawn_blocking). Tokio doesn't
allow nested block_on() calls.

Fix: Replace ALL block_in_place patterns with std:🧵:spawn + mpsc channel.
This creates a completely separate OS thread with its own runtime, avoiding
any nesting issues. Works from any context: async, spawn_blocking, or sync.

Files: 14 files across secrets, utils, state, calendar, analytics, email,
and all keyword handlers (universal_messaging, search, book, create_draft,
create_site, hearing/syntax, use_tool, find, admin_email, goals)
2026-04-03 12:54:36 -03:00
f6a864aa67 fix: replace nested runtime block_on with new_current_thread to prevent panic
All checks were successful
BotServer CI/CD / build (push) Successful in 5m32s
Root cause: Handle::current().block_on() panics inside multi_thread runtime
with 'Cannot start a runtime from within a runtime' error.

Fix: All sync-to-async bridges now use tokio::runtime::Builder::new_current_thread()
instead of Handle::current().block_on(). Also changed SECRETS_MANAGER from
tokio::sync::RwLock to std::sync::RwLock to eliminate unnecessary async overhead.

Files: 14 files across keywords, secrets, utils, state, calendar, analytics, email
Impact: Fixes production crash during bot loading phase
2026-04-03 09:17:23 -03:00
e992ed3b39 Enforce Vault-only secrets: remove env var fallbacks, all secrets from Vault
Some checks are pending
BotServer CI/CD / build (push) Waiting to run
- Remove all std::env::var calls except VAULT_* and PORT
- get_from_env returns hardcoded defaults only (no env var reading)
- Auth config, rate limits, email, analytics, calendar all use Vault
- WORK_PATH replaced with get_work_path() helper reading from Vault
- .env on production cleaned to only VAULT_ADDR, VAULT_TOKEN, VAULT_CACERT, PORT
- All service IPs/credentials stored in Vault secret/gbo/*
2026-04-03 07:11:40 -03:00
fb2e5242da fix: Vault seeding, service health checks, and restart idempotency
All checks were successful
BotServer CI/CD / build (push) Successful in 55m52s
- Replace hardcoded passwords with generate_random_string() for all Vault-seeded services
- Add valkey-cli, nc to SafeCommand allowlist; fix PATH in all 4 execution methods
- Fix empty Vault KV values ('none' placeholder) preventing 'Failed to parse K=V' errors
- Fix special chars in generated passwords triggering shell injection false positives
- Add ALM app.ini creation with absolute paths for Forgejo CLI
- Increase Qdrant timeout 15s→45s, ALM wait 5s→20s
- Persist file_states and kb_states to disk for .bas/KB idempotency across restarts
- Add duplicate check to use_website registration (debug log for existing)
- Remove dead code (SERVER_START_EPOCH, server_epoch)
- Add generate_random_string() to shared mod.rs, remove duplicates
2026-04-01 12:22:57 -03:00
adb26330d2 fix: Simple 50ms timeout for Redis connection 2026-03-21 10:48:47 -03:00
9d6c2686f1 fix: Remove connection caching (no Clone) 2026-03-21 10:37:49 -03:00
b3ce293487 fix: Clean up duplicate Redis code and fix WebSocket log level 2026-03-21 10:30:19 -03:00