Commit graph

272 commits

Author SHA1 Message Date
fb2e5242da fix: Vault seeding, service health checks, and restart idempotency
All checks were successful
BotServer CI/CD / build (push) Successful in 55m52s
- Replace hardcoded passwords with generate_random_string() for all Vault-seeded services
- Add valkey-cli, nc to SafeCommand allowlist; fix PATH in all 4 execution methods
- Fix empty Vault KV values ('none' placeholder) preventing 'Failed to parse K=V' errors
- Fix special chars in generated passwords triggering shell injection false positives
- Add ALM app.ini creation with absolute paths for Forgejo CLI
- Increase Qdrant timeout 15s→45s, ALM wait 5s→20s
- Persist file_states and kb_states to disk for .bas/KB idempotency across restarts
- Add duplicate check to use_website registration (debug log for existing)
- Remove dead code (SERVER_START_EPOCH, server_epoch)
- Add generate_random_string() to shared mod.rs, remove duplicates
2026-04-01 12:22:57 -03:00
3e46a16469 fix: Seed default credentials into Vault after initialization
Some checks failed
BotServer CI/CD / build (push) Failing after 3h13m28s
- Add seed_vault_defaults() to write default creds for all components
  (drive, cache, tables, directory, email, llm, encryption, meet, vectordb, alm)
- Call seed_vault_defaults() after KV2 enable in initialize_vault_local()
- Call seed_vault_defaults() in recover_existing_vault() for recovery path
- Rewrite fetch_vault_credentials() to use SafeCommand directly instead of
  safe_sh_command, avoiding '//' shell injection false positive on URLs
- Components like Drive now get credentials from Vault instead of 403 errors
2026-03-31 22:19:09 -03:00
9919a8321c fix: Use SafeCommand directly for vault health check to avoid shell injection false positive
All checks were successful
BotServer CI/CD / build (push) Successful in 6m46s
- Replace safe_sh_command with SafeCommand::new("curl").args() in vault_health_check()
- The URL contains https:// which triggered '//' pattern detection in shell command
- Direct SafeCommand bypasses shell parsing, URL passed as single argument
- Add vault data directory existence check before recovery attempt
- Prevents 'Dangerous pattern // detected' errors during bootstrap
2026-03-31 21:34:04 -03:00
07a6c1edb3 Merge commit '582ea634'
All checks were successful
BotServer CI/CD / build (push) Successful in 7m38s
2026-03-31 21:10:25 -03:00
582ea634e7 fix: Vault bootstrap recovery for sealed but initialized instances
- Fix vault_health_check() stub that always returned false
- Add recover_existing_vault() to handle Vault with existing data but no init.json
- Add unseal_vault() helper to unseal with existing vault-unseal-keys
- Detect initialized Vault via health endpoint or data directory presence
- Prevents bootstrap failure when reset.sh deletes init.json but Vault data persists

Root cause: vault_health_check() was a stub returning false, causing bootstrap
to always try vault operator init on already-initialized (but sealed) Vault,
which failed with connection refused. This cascaded to all services failing
to fetch credentials from Vault.
2026-03-31 20:49:29 -03:00
2fa59057fa fix: Resolve migration error, Vault 403, cache timeout, and shell injection false positives
Some checks failed
BotServer CI/CD / build (push) Has been cancelled
- Fix migration 6.2.5: Create lost_reason column before VIEW that references it
- Fix Vault 403: Enable KV2 secrets engine after initialization
- Fix cache timeout: Increase Valkey readiness wait from 12s to 30s
- Fix command_guard: Remove () from forbidden chars (safe in std::process::Command)
2026-03-31 19:55:16 -03:00
b83b4ffc4d fix: Remove server_epoch() from start_bas_executed Redis key
The epoch caused a new key to be created every second, bypassing
the 'already executed' check and running start.bas multiple times,
resulting in triplicated suggestions.
2026-03-21 20:40:25 -03:00
1132983064 feat(kb): add with_bot_config to load embedding from bot config
- Adds KnowledgeBaseManager::with_default_config() as alias to new()
- Adds KnowledgeBaseManager::with_bot_config() to load embedding_url,
  embedding_model, and qdrant config from bot's config.csv
- Updates bootstrap to use with_bot_config with default_bot_id
- Enables per-bot embedding configuration instead of global env vars
2026-03-21 18:55:36 -03:00
622f1222dc fix(websocket): force start.bas execution on connection to restore chat on page reload while preventing duplicate execution 2026-03-21 16:38:03 -03:00
d19984fa07 feat: Improve KB keywords and package manager installer 2026-03-20 17:38:47 -03:00
57a8b7f8f0 Fix: use pgrep to check valkey/qdrant running state
- valkey check_cmd: replaced valkey-cli ping (network) with pgrep -x valkey-server
- qdrant check_cmd: replaced curl https check (TLS error 35) with pgrep -x qdrant
- Prevents duplicate instances on each botserver restart
2026-03-20 15:40:22 -03:00
3bb115266b feat: Add GUID prefix to Qdrant collection names for KB security isolation 2026-03-19 19:51:28 -03:00
d6ebd0cf6e fix: send suggestions separately from TALK, clear Redis keys for refresh
- Remove suggestions fetching from TALK function
- WebSocket handler now fetches and sends suggestions after start.bas executes
- Clear suggestions and start_bas_executed keys to allow re-run on refresh
- Decouple TALK from suggestions handling
2026-03-19 09:53:39 -03:00
6e594d68dd Fix: Wait for send_task to be ready before executing start.bas 2026-03-18 14:38:46 -03:00
8f073a15fd Fix: Wait for send_task to be ready before executing start.bas so TALK messages work 2026-03-18 14:18:05 -03:00
346c83871a Fix Vault TLS certificate to include Subject Alternative Name for modern client compatibility 2026-03-18 09:30:27 -03:00
ed2a1d83f0 fix: include server epoch in start_bas_executed key to invalidate after restart 2026-03-17 15:45:02 -03:00
492530ee77 Fix panic: Cannot start a runtime from within a runtime in secrets module
Removed tokio::runtime::Handle::block_on() calls that were causing panics
when called from within async contexts. Now uses direct fallback to
environment variables instead.
2026-03-17 15:04:40 -03:00
c340f95da1 security: bind MinIO and Valkey to 127.0.0.1 only
Some checks failed
BotServer CI / build (push) Failing after 6m44s
2026-03-17 01:32:21 -03:00
ab1f2df476 Read Drive config from Vault at runtime with fallback defaults
Some checks failed
BotServer CI / build (push) Failing after 7m26s
2026-03-17 00:00:36 -03:00
dae78479d2 Fix HEAR: block thread on channel instead of throw+re-run from top
Some checks failed
BotServer CI / build (push) Failing after 6m54s
- Add hear_channels: HashMap<Uuid, SyncSender<String>> to AppState
- HEAR now blocks the spawn_blocking thread via sync_channel recv()
- deliver_hear_input() called at top of stream_response() to unblock
- Script continues from exact HEAR position, no side-effect re-execution
- All three HEAR variants (basic, AS TYPE, AS MENU) use same mechanism
2026-03-16 23:00:17 -03:00
35b793d29c Add interactive vault put - prompt for secrets instead of CLI args
All checks were successful
BotServer CI / build (push) Successful in 10m50s
2026-03-16 22:27:06 -03:00
16394aef01 Fix LXD IPv4: retry DHCP 30s, fallback to static IP for privileged containers
Some checks failed
BotServer CI / build (push) Failing after 6m46s
2026-03-16 21:34:59 -03:00
0db08fbf39 fix: fallback to get_or_create_user_session when session_id not found in start.bas 2026-03-16 13:25:45 -03:00
7ef1efa047 feat: add setup-env command to generate .env from vault container 2026-03-15 21:15:48 -03:00
dfe5162f66 feat(install): add --container-only flag to install command
- Add --container-only flag to create container without completing full installation
- Exit immediately after container creation
- Useful for manual setup or debugging installation issues
2026-03-15 20:49:08 -03:00
d1cb6b758c Fix LXD container mode: PATH, socket proxy, exec
All checks were successful
BotServer CI / build (push) Successful in 10m54s
2026-03-15 20:00:06 -03:00
ba53a0c178 Fix LXD socket handling in container mode
All checks were successful
BotServer CI / build (push) Successful in 11m6s
2026-03-15 18:58:21 -03:00
ef426b7a50 LXD proxy and container improvements
Some checks failed
BotServer CI / build (push) Failing after 7m5s
2026-03-15 15:50:02 -03:00
eb586cf4f9 refactor(secrets): add tenant/org hierarchy with fallback support 2026-03-15 11:38:50 -03:00
7fb73e683f feat: add campaigns, attendance SLA, and marketing modules 2026-03-14 16:35:42 -03:00
13892b3157 Fix tenant-org-bot relationship and CRM lead form 2026-03-12 18:19:18 -03:00
ad4aca21ff feat: implement WhatsApp bursting rate limit and consume activation tokens
All checks were successful
BotServer CI / build (push) Successful in 11m4s
2026-03-10 21:18:14 -03:00
e98de24fe6 chore: update submodules
All checks were successful
BotServer CI / build (push) Successful in 9m56s
2026-03-10 19:39:31 -03:00
260a13e77d refactor: apply various fixes across botserver
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-03-10 15:15:21 -03:00
82bfd0a443 Fix Bedrock config for OpenAI GPT-OSS models
All checks were successful
BotServer CI / build (push) Successful in 12m35s
2026-03-10 12:36:24 -03:00
c523cee177 Use Redis to track last sent time per WhatsApp recipient
All checks were successful
BotServer CI / build (push) Successful in 13m37s
- Store last_sent timestamp in Redis (whatsapp:last_sent:<phone>)
- Always wait 6 seconds between messages to same recipient
- Persists across restarts
2026-03-09 21:00:45 -03:00
47e1013700 Fix Redis BLPOP return type: handle (key, value) tuple
All checks were successful
BotServer CI / build (push) Successful in 13m52s
2026-03-09 19:43:47 -03:00
5d3174b24e Fix compilation: add Debug derive and type annotations
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-03-09 19:32:36 -03:00
d22ce019b6 Implement Meta WhatsApp official rate limits: 1msg/6s, 45 burst, 4^X retry
Some checks failed
BotServer CI / build (push) Failing after 8m45s
- Add Redis-backed message queue with per-recipient tracking
- Enforce 1 message per 6 seconds per recipient (0.17 msg/s)
- Support burst mode: up to 45 messages in 6-second window
- Implement proportional cooldown after burst
- Add exponential backoff retry on error 131056 (4^X seconds)
- Update botbook with official Meta rate limits
- Add unit tests for burst mode and rate limiting
- Fix config inheritance bug: delete all keys before sync
2026-03-09 19:16:41 -03:00
77c35ccde5 feat: add WhatsApp rate limiting and LLM hallucination detection
All checks were successful
BotServer CI / build (push) Successful in 11m51s
2026-03-09 17:22:47 -03:00
c072fb936e fix(llm): load system-prompt from config.csv correctly
All checks were successful
BotServer CI / build (push) Successful in 17m27s
- Move system_prompt retrieval inside spawn_blocking closure
- Include system_prompt in the return tuple to fix scope issue
- Add trace logging for debugging system-prompt loading
- GLM-5 and other LLM providers now correctly receive custom system prompts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-09 11:55:05 -03:00
97661d75e2 feat(whatsapp): isolate lists as single messages and remove code blocks
- Split list detection into numbered and bullet list items
- Add looks_like_list_start() to detect when list is beginning
- Add looks_like_list_end() to detect when list has ended
- Add split_text_before_list() to separate text before list
- Add split_list_from_text() to separate list from text after
- Update streaming logic to send lists as isolated messages
- Add code block removal (triple backticks and inline backticks)
- Add comprehensive unit tests for list detection functions

Resolves: Lists being mixed with other text in WhatsApp messages
Resolves: JavaScript/C# code leaking into WhatsApp messages
2026-03-08 14:52:59 -03:00
859db6b8a0 fix: Lower KB search thresholds and add Cloudflare AI embedding support
All checks were successful
BotServer CI / build (push) Successful in 10m35s
- Lower score_threshold in kb_indexer.rs from 0.5 to 0.3
- Lower website search threshold in kb_context.rs from 0.6 to 0.4
- Lower KB search threshold in kb_context.rs from 0.7 to 0.5
- Add Cloudflare AI (/ai/run/) URL detection in cache.rs
- Add Cloudflare AI request format ({"text": ...}) in cache.rs
- Add Cloudflare AI response parsing (result.data) in cache.rs

This fixes the issue where KB search returned 0 results even with
114 chunks indexed. The high thresholds were filtering out all results.
2026-03-05 00:11:08 -03:00
8500949fcd fix: Lower KB search thresholds and add Cloudflare AI embedding support
- Lower score_threshold in kb_indexer.rs from 0.5 to 0.3
- Lower website search threshold in kb_context.rs from 0.6 to 0.4
- Lower KB search threshold in kb_context.rs from 0.7 to 0.5
- Add Cloudflare AI (/ai/run/) URL detection in cache.rs
- Add Cloudflare AI request format ({"text": ...}) in cache.rs
- Add Cloudflare AI response parsing (result.data) in cache.rs

This fixes the issue where KB search returned 0 results even with
114 chunks indexed. The high thresholds were filtering out all results.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-05 00:06:17 -03:00
5404e3e7ba feat: Enhance KB context, embedding generator, and website crawler
- Improved kb_context with better context management
- Enhanced embedding_generator with extended functionality (+231 lines)
- Updated kb_indexer with improved indexing logic
- Expanded website_crawler_service capabilities (+230 lines)
- Updated use_website keyword implementation
- Refined bootstrap_manager and utils
- Improved drive monitoring and local file monitor
- Added server enhancements
2026-03-04 15:43:37 -03:00
6195062482 fix: add missing PathBuf import 2026-03-03 10:03:49 -03:00
b2d6a51ebd fix: correct tool file path resolution to match architecture
- Updated get_tool_bas_path to look in /opt/gbo/data (source) first
- Then check botserver-stack/data/system/work (compiled) second
- Removed incorrect $HOME/data and $HOME/gb/work paths
- Fixes 'Tool file not found' error when executing inscricao tool

According to AGENTS.md architecture:
- Bots are in /opt/gbo/data primary
- They are compiled into work directory by local_file_monitor
- tool_executor was looking in wrong directories
2026-03-03 10:01:33 -03:00
1e71c9be09 feat: Add comprehensive stage progress logging
Some checks failed
BotServer CI / build (push) Failing after 2m39s
- Add detailed logging for all 5 pipeline stages (PLAN, BUILD, REVIEW, DEPLOY, MONITOR)
- Log stage start/complete events with agent IDs and progress details
- Add resource creation/deletion logging in drive_handlers
- Improve pipeline summary logging with task ID, nodes, resources, and URL

This addresses the requirement for textual progress in console logs.
2026-03-01 22:36:14 -03:00
c326581a9e fix(zitadel): resolve OAuth client initialization timing issue
- Fix PAT extraction timing with retry loop (waits up to 60s for PAT in logs)
- Add sync command to flush filesystem buffers before extraction
- Improve logging with progress messages and PAT verification
- Refactor setup code into consolidated setup.rs module
- Fix YAML indentation for PatPath and MachineKeyPath
- Change Zitadel init parameter from --config to --steps

The timing issue occurred because:
1. Zitadel writes PAT to logs at startup (~18:08:59)
2. Post-install extraction ran too early (~18:09:35)
3. PAT file wasn't created until ~18:10:38 (63s after installation)
4. OAuth client creation failed because PAT file didn't exist yet

With the retry loop:
- Waits for PAT to appear in logs with sync+grep check
- Extracts PAT immediately when found
- OAuth client creation succeeds
- directory_config.json saved with valid credentials
- Login flow works end-to-end

Tested: Full reset.sh and login verification successful
2026-03-01 19:06:09 -03:00