Commit graph

341 commits

Author SHA1 Message Date
cd0e049e81 Reduce embedding batch_size from 16 to 2 to prevent llama-server crash
All checks were successful
BotServer CI/CD / build (push) Successful in 2m3s
The bge-small-en-v1.5-f32.gguf model has n_ctx_train=512. With batch_size=16
and ~300+ tokens per chunk, total tokens exceed 512 causing GGML_ASSERT crash.

Now with batch_size=2, embeddings are processed safely.
2026-04-12 08:21:39 -03:00
cdab04e999 Fix embedding health check: behavior-based instead of URL whitelist
All checks were successful
BotServer CI/CD / build (push) Successful in 3m32s
- Remove hardcoded URL list for remote API detection
- Try /health first, then probe with HEAD if 404/405
- Re-enable embedding server ready check in drive_monitor
- No more embedding_key hack that skipped health checks entirely
2026-04-12 07:15:54 -03:00
7a1ec157f1 Fix KB indexing: upsert kb_collections, consistent collection names, preserve indexed flag
All checks were successful
BotServer CI/CD / build (push) Successful in 3m23s
- Bug 1: check_gbkb_changes now preserves indexed=true from previous
  state when etag matches, preventing redundant re-indexing every cycle
- Bug 2: USE KB fallback uses bot_id_short (8 chars) instead of random
  UUID, matching the collection name convention used by DriveMonitor
- Bug 3: handle_gbkb_change now upserts into kb_collections table after
  successful indexing, so USE KB can find the collection at runtime
- Changed ON CONFLICT DO NOTHING to DO UPDATE for kb_collections inserts
- Changed process_gbkb_folder return type to Result<IndexingResult>
2026-04-11 21:26:02 -03:00
cf4a00e16e fix: work path uses production /opt/gbo when env exists or path exists; mark .bas files indexed=true after compilation
All checks were successful
BotServer CI/CD / build (push) Successful in 3m20s
- get_work_path_default/get_stack_path no longer rely on CWD-relative botserver-stack check which caused wrong output path in production when CI left that directory
- DriveMonitor now marks .bas file states as indexed=true after list+compile cycle
- Added compile_tool logging for work_dir path
2026-04-11 20:16:22 -03:00
f4c99030aa fix: use get_work_path() instead of get_stack_path()+data/system for work dir, add etag check for PROMPT.md downloads
All checks were successful
BotServer CI/CD / build (push) Successful in 3m37s
2026-04-11 18:42:09 -03:00
a4a3837c4c fix: critical bugs - LLM context truncation, bot creation, S3 endpoint, vectordb seed
All checks were successful
BotServer CI/CD / build (push) Successful in 3m28s
1. Fix model.starts_with('') always true - was limiting ALL models to 768 tokens
   (local llama limit), truncating system prompts and KB context. Now only
   applies when model=='local' or empty string, default is 32k tokens.

2. Fix create_bot_from_drive missing NOT NULL columns (llm_provider,
   context_provider) - bots auto-created from S3 buckets failed to persist.

3. Fix S3 endpoint URL construction missing port 9100.

4. Fix Vault seed: vectordb.url was empty string, now defaults to
   http://localhost:6333.

5. Fix Vault credential regeneration on recovery - added vault_seeds_exist().

6. Fix CA cert path for Vault TLS (botserver-stack vs botserver-stack).

7. Add bot verification after insert in create_bot_from_drive.
2026-04-11 17:56:03 -03:00
a131120638 Fix KB indexing: bot-specific embedding config, PROMPT.md sync, single-file streaming
All checks were successful
BotServer CI/CD / build (push) Successful in 4m1s
2026-04-11 13:27:48 -03:00
12988b637d Fix KB indexing: single file streaming, dedup tracking, .ast cache
All checks were successful
BotServer CI/CD / build (push) Successful in 12m31s
2026-04-11 13:10:09 -03:00
dd4c780c4d Fix Zitadel health check and add ss command to allowed commands
All checks were successful
BotServer CI/CD / build (push) Successful in 3m20s
- Add 'ss' to ALLOWED_COMMANDS for port checking
- Fix Zitadel health check URL to include full address
2026-04-11 07:49:11 -03:00
5576378b3f Update botserver: Multiple improvements across core modules
All checks were successful
BotServer CI/CD / build (push) Successful in 10m41s
2026-04-11 07:33:32 -03:00
cb1998efe8 Fix Zitadel bootstrap: Pass all database env vars to ensure connection
Some checks failed
BotServer CI/CD / build (push) Has been cancelled
- Add ZITADEL_DATABASE_* environment variables to directory component env_vars
- Remove inline env vars from exec_cmd (now applied via spawn_with_envs)
- Use $DB_PASSWORD reference to fetch from Vault at runtime
- This ensures Zitadel gets database credentials on every boot, not just during install

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-04-10 19:55:52 -03:00
ca04e6cecf Fix: change DRIVE URL from https to http
All checks were successful
BotServer CI/CD / build (push) Successful in 58s
2026-04-10 14:34:11 -03:00
0fad62aed9 Fix S3 endpoint: add http:// prefix if missing
All checks were successful
BotServer CI/CD / build (push) Successful in 4m28s
2026-04-10 13:39:07 -03:00
514427c7cc Fix PROMPT.md loading: use get_work_path instead of get_stack_path
All checks were successful
BotServer CI/CD / build (push) Successful in 5m46s
2026-04-10 13:11:45 -03:00
4e60bed7b7 fix: use curl with --cacert for Vault requests
All checks were successful
BotServer CI/CD / build (push) Successful in 1m2s
2026-04-10 08:27:55 -03:00
6fb8063a5b fix: add debug logging for Vault requests
Some checks failed
BotServer CI/CD / build (push) Has been cancelled
2026-04-10 08:11:19 -03:00
2afa8bcfee fix: set SSL_CERT_FILE env before Vault HTTP calls
All checks were successful
BotServer CI/CD / build (push) Successful in 5m24s
2026-04-10 08:04:12 -03:00
452d0ec89b fix: read drive/cache/qdrant config directly from Vault using HTTP
All checks were successful
BotServer CI/CD / build (push) Successful in 12m38s
2026-04-10 07:42:55 -03:00
f58d000ac8 debug: log S3 operator server and access_key
All checks were successful
BotServer CI/CD / build (push) Successful in 5m24s
2026-04-10 00:05:25 -03:00
13fee4d079 Use bot-specific llm-endpoint-path from config
All checks were successful
BotServer CI/CD / build (push) Successful in 5m27s
2026-04-09 22:11:54 -03:00
8dddc916ff fix: use Vault config for Qdrant in KB indexer
- website_crawler_service: use QdrantConfig::from_config instead of default
- local_file_monitor: use QdrantConfig::from_config with DbPool
- kb_indexer: KbFolderMonitor now uses SecretsManager for Qdrant config

This fixes the issue where Qdrant URL was hardcoded to localhost:6333
instead of reading from Vault (gbo/vectordb).
2026-04-09 18:27:10 -03:00
f526fa1daa Fix hardcoded paths for production environment
- Update get_work_path_default() to check for .env in /opt/gbo/bin/.env
- Update get_stack_path() to check for .env in /opt/gbo/bin/.env
- Update DriveMonitor::new() to use get_work_path() instead of hardcoded path
- Update start_config_watcher() to use get_work_path() instead of hardcoded path

This fixes the issue where botserver was using development paths
(/home/rodriguez/src/gb/botserver-stack/data/system/work) in production
instead of production paths (/opt/gbo/work).
2026-04-09 18:21:17 -03:00
61f4353dbb fix: UTF-8 char boundary panics in log truncation
Some checks failed
BotServer CI/CD / build (push) Failing after 6m45s
2026-04-09 01:37:48 -03:00
b4a82b6c06 Disable local file monitoring, use drive (MinIO) as sole bot source
Some checks failed
BotServer CI/CD / build (push) Failing after 13m5s
- Disable LocalFileMonitor and ConfigWatcher - use S3/MinIO only
- Filter S3 buckets to gbo-*.gbai prefix
- Auto-create bots in database when new S3 buckets discovered
- Change file paths to use work directory instead of /opt/gbo/data
- Add RunQueryDsl import for Diesel queries
2026-04-08 17:47:44 -03:00
9e799dd6b1 Disable /opt/gbo/data loading, use drive (MinIO) only for bot sources
Some checks failed
BotServer CI/CD / build (push) Failing after 8m28s
- Remove LocalFileMonitor and ConfigWatcher for /opt/gbo/data
- Remove /opt/gbo/data from mount_all_bots() scanning
- Change start.bas, tables.bas, and tool paths to use work directory
- Filter drive buckets to only gbo-* prefix
- Remove unused create_bot_simple method
- Fix all warnings (unused imports, variables, dead code)
2026-04-08 16:55:50 -03:00
9b04af9e7b Fix USE KB and USE WEBSITE default features compilation
Some checks failed
BotServer CI/CD / build (push) Failing after 10m2s
2026-04-07 20:14:12 -03:00
73002b36cc Update botserver: various fixes and improvements
All checks were successful
BotServer CI/CD / build (push) Successful in 9m59s
2026-04-07 13:33:50 -03:00
90c14bcd09 Fix DETECT: use bot-specific DB pool, add anonymous auth when directory disabled
All checks were successful
BotServer CI/CD / build (push) Successful in 12m42s
2026-04-06 13:37:23 -03:00
86bb4cad8e fix(botserver): Handle TOOL_EXEC message type for direct tool execution without KB/LLM
Some checks failed
BotServer CI/CD / build (push) Failing after 5m40s
2026-04-05 19:11:04 -03:00
155d465b14 Update botserver: Refactor groups module, add Knowledge Base group association logic, and implement Drive tags for KB access.
Some checks failed
BotServer CI/CD / build (push) Failing after 5m53s
2026-04-05 09:11:54 -03:00
552f37a41c fix(secrets): Remove unused sync caching functions to fix CI clippy warnings
All checks were successful
BotServer CI/CD / build (push) Successful in 12m19s
2026-04-05 07:52:41 -03:00
24e0b23030 fix(secrets): Use async Vault client to properly load TLS CA and fix cache expiration bug
Some checks failed
BotServer CI/CD / build (push) Has been cancelled
2026-04-05 07:48:44 -03:00
3502c61faf Update secrets and email types
All checks were successful
BotServer CI/CD / build (push) Successful in 10m52s
- Refactor secrets module structure
- Remove unused type in email types
- Improve error handling in secrets

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-04-05 06:27:34 -03:00
0ac8178baa fix: use std::sync::RwLock for cache to support sync methods
Some checks failed
BotServer CI/CD / build (push) Failing after 6m3s
2026-04-05 01:19:45 -03:00
60dc273681 fix: use CachedSecret struct instead of tuples in sync methods
Some checks failed
BotServer CI/CD / build (push) Failing after 2s
2026-04-05 01:07:42 -03:00
985ba63e38 fix: use env vars for blocking Vault HTTP call
Some checks failed
BotServer CI/CD / build (push) Failing after 5m54s
2026-04-05 00:56:07 -03:00
f1d0a985e4 fix: remove duplicate code block from secrets mod
Some checks failed
BotServer CI/CD / build (push) Failing after 5m52s
2026-04-05 00:48:13 -03:00
697210a37e fix: remove duplicate code block in secrets mod
Some checks failed
BotServer CI/CD / build (push) Failing after 12s
2026-04-05 00:44:20 -03:00
c90c5dc039 fix: use blocking HTTP for Vault email config to avoid runtime nesting
Some checks failed
BotServer CI/CD / build (push) Failing after 2s
2026-04-05 00:37:33 -03:00
eba075bb9d fix: use existing tokio Handle instead of spawning new runtime for email config
All checks were successful
BotServer CI/CD / build (push) Successful in 4m32s
2026-04-05 00:13:10 -03:00
597a962fbf fix: use multi-threaded runtime for email config lookup to avoid blocking
All checks were successful
BotServer CI/CD / build (push) Successful in 4m23s
2026-04-04 23:48:00 -03:00
30bb764876 fix: add 5s timeout to email config lookup to prevent hanging
All checks were successful
BotServer CI/CD / build (push) Successful in 4m26s
2026-04-04 23:35:20 -03:00
1dc11c9b4e refactor: unify email sending to use EmailService + Vault, remove Gmail hardcoded defaults
Some checks failed
BotServer CI/CD / build (push) Failing after 17s
- Replace gmail defaults with Vault-backed get_email_config_for_bot_sync
- send_campaign_email now delegates to EmailService::send_email
- Remove hardcoded smtp.gmail.com, imap.gmail.com, noreply@generalbots.com
- All SMTP config flows through Vault: bot → default bot → system
- Remove unused lettre imports from marketing/email.rs
2026-04-04 22:16:20 -03:00
dde6ac236e fix: check for non-empty smtp_from in email fallback chain
All checks were successful
BotServer CI/CD / build (push) Successful in 4m29s
2026-04-04 18:45:47 -03:00
01db253900 fix: add log::info import, fix ContentType parse, remove unused RedisClient import
All checks were successful
BotServer CI/CD / build (push) Successful in 4m38s
2026-04-04 17:49:40 -03:00
45eb8357cb feat: implement real email sending via lettre + Vault credentials
Some checks failed
BotServer CI/CD / build (push) Failing after 8m6s
- Replace EmailService::send_email stub with full lettre SMTP implementation
- Vault resolution chain: bot-specific → default bot → system fallback
- Seed Vault prod with default email config (contato@pragmatismo.com.br)
- Update all call sites to pass bot_id for Vault lookup
- Support attachments via lettre MultiPart/Attachment API
- Remove unused imports and dead code
2026-04-04 17:16:50 -03:00
0de4565e5a refactor: Generalize WhatsAppAdapter::new to accept &AppState
All checks were successful
BotServer CI/CD / build (push) Successful in 4m55s
- Simplify constructor from (pool, bot_id, cache) to (&state, bot_id)
- Adapter now extracts conn and cache from AppState internally
- Updates 15 call sites across 6 files
- Removes redundant parameter plumbing at every call site
2026-04-04 15:46:10 -03:00
44669c3825 fix: Fix resolve_export_path typo and remove unused PathBuf imports
All checks were successful
BotServer CI/CD / build (push) Successful in 4m28s
2026-04-04 10:23:42 -03:00
552d58376f fix: Fix compilation errors from path refactoring
Some checks failed
BotServer CI/CD / build (push) Failing after 1m27s
- bootstrap_utils.rs: Change Vec<(&'static str,...)> to Vec<(String,...)> to avoid dangling references
- bootstrap_manager.rs: Use name.as_str() for safe_pkill
- setup.rs: Use PathBuf instead of Path::new with format!
- directory/bootstrap.rs: Use PathBuf for pat_dir
- main.rs: Use PathBuf for vault_init_path_early
2026-04-04 10:04:00 -03:00
7d8f141fc2 refactor: Replace all hardcoded ./botserver-stack paths with get_stack_path()/get_work_path()
Some checks failed
BotServer CI/CD / build (push) Failing after 1m28s
- Adds get_stack_path() helper: returns /opt/gbo in production (.env without botserver-stack), ./botserver-stack in dev
- Adds get_work_path() helper: returns /opt/gbo/work in production, ./botserver-stack/data/system/work in dev
- Updated 35+ files to use dynamic path resolution
- Production system container no longer needs botserver-stack directory
- Work files go to /opt/gbo/work instead of /opt/gbo/bin/botserver-stack
2026-04-04 09:24:44 -03:00