Commit graph

57 commits

Author SHA1 Message Date
86bb4cad8e fix(botserver): Handle TOOL_EXEC message type for direct tool execution without KB/LLM
Some checks failed
BotServer CI/CD / build (push) Failing after 5m40s
2026-04-05 19:11:04 -03:00
7d8f141fc2 refactor: Replace all hardcoded ./botserver-stack paths with get_stack_path()/get_work_path()
Some checks failed
BotServer CI/CD / build (push) Failing after 1m28s
- Adds get_stack_path() helper: returns /opt/gbo in production (.env without botserver-stack), ./botserver-stack in dev
- Adds get_work_path() helper: returns /opt/gbo/work in production, ./botserver-stack/data/system/work in dev
- Updated 35+ files to use dynamic path resolution
- Production system container no longer needs botserver-stack directory
- Work files go to /opt/gbo/work instead of /opt/gbo/bin/botserver-stack
2026-04-04 09:24:44 -03:00
4d7297243e Fix clippy warnings: reduce 17 warnings to 0
All checks were successful
BotServer CI/CD / build (push) Successful in 6m58s
- Fix double_ended_iterator_last: use next_back() instead of last()
- Fix manual_clamp: use .clamp() instead of min().max()
- Fix too_many_arguments: create KbInjectionContext struct
- Fix needless_borrow: remove unnecessary & reference
- Fix let_and_return: return value directly
- Fix await_holding_lock: drop guard before await
- Fix collapsible_else_if: collapse nested if-else

All changes verified with cargo clippy (0 warnings, 0 errors)
Note: Local botserver crashes with existing panic during LocalFileMonitor initialization
This panic exists in original code too, not caused by these changes
2026-04-03 22:34:43 -03:00
fb2e5242da fix: Vault seeding, service health checks, and restart idempotency
All checks were successful
BotServer CI/CD / build (push) Successful in 55m52s
- Replace hardcoded passwords with generate_random_string() for all Vault-seeded services
- Add valkey-cli, nc to SafeCommand allowlist; fix PATH in all 4 execution methods
- Fix empty Vault KV values ('none' placeholder) preventing 'Failed to parse K=V' errors
- Fix special chars in generated passwords triggering shell injection false positives
- Add ALM app.ini creation with absolute paths for Forgejo CLI
- Increase Qdrant timeout 15s→45s, ALM wait 5s→20s
- Persist file_states and kb_states to disk for .bas/KB idempotency across restarts
- Add duplicate check to use_website registration (debug log for existing)
- Remove dead code (SERVER_START_EPOCH, server_epoch)
- Add generate_random_string() to shared mod.rs, remove duplicates
2026-04-01 12:22:57 -03:00
b83b4ffc4d fix: Remove server_epoch() from start_bas_executed Redis key
The epoch caused a new key to be created every second, bypassing
the 'already executed' check and running start.bas multiple times,
resulting in triplicated suggestions.
2026-03-21 20:40:25 -03:00
622f1222dc fix(websocket): force start.bas execution on connection to restore chat on page reload while preventing duplicate execution 2026-03-21 16:38:03 -03:00
3bb115266b feat: Add GUID prefix to Qdrant collection names for KB security isolation 2026-03-19 19:51:28 -03:00
d6ebd0cf6e fix: send suggestions separately from TALK, clear Redis keys for refresh
- Remove suggestions fetching from TALK function
- WebSocket handler now fetches and sends suggestions after start.bas executes
- Clear suggestions and start_bas_executed keys to allow re-run on refresh
- Decouple TALK from suggestions handling
2026-03-19 09:53:39 -03:00
6e594d68dd Fix: Wait for send_task to be ready before executing start.bas 2026-03-18 14:38:46 -03:00
8f073a15fd Fix: Wait for send_task to be ready before executing start.bas so TALK messages work 2026-03-18 14:18:05 -03:00
ed2a1d83f0 fix: include server epoch in start_bas_executed key to invalidate after restart 2026-03-17 15:45:02 -03:00
ab1f2df476 Read Drive config from Vault at runtime with fallback defaults
Some checks failed
BotServer CI / build (push) Failing after 7m26s
2026-03-17 00:00:36 -03:00
dae78479d2 Fix HEAR: block thread on channel instead of throw+re-run from top
Some checks failed
BotServer CI / build (push) Failing after 6m54s
- Add hear_channels: HashMap<Uuid, SyncSender<String>> to AppState
- HEAR now blocks the spawn_blocking thread via sync_channel recv()
- deliver_hear_input() called at top of stream_response() to unblock
- Script continues from exact HEAR position, no side-effect re-execution
- All three HEAR variants (basic, AS TYPE, AS MENU) use same mechanism
2026-03-16 23:00:17 -03:00
0db08fbf39 fix: fallback to get_or_create_user_session when session_id not found in start.bas 2026-03-16 13:25:45 -03:00
7fb73e683f feat: add campaigns, attendance SLA, and marketing modules 2026-03-14 16:35:42 -03:00
13892b3157 Fix tenant-org-bot relationship and CRM lead form 2026-03-12 18:19:18 -03:00
e98de24fe6 chore: update submodules
All checks were successful
BotServer CI / build (push) Successful in 9m56s
2026-03-10 19:39:31 -03:00
260a13e77d refactor: apply various fixes across botserver
Some checks failed
BotServer CI / build (push) Has been cancelled
2026-03-10 15:15:21 -03:00
82bfd0a443 Fix Bedrock config for OpenAI GPT-OSS models
All checks were successful
BotServer CI / build (push) Successful in 12m35s
2026-03-10 12:36:24 -03:00
77c35ccde5 feat: add WhatsApp rate limiting and LLM hallucination detection
All checks were successful
BotServer CI / build (push) Successful in 11m51s
2026-03-09 17:22:47 -03:00
c072fb936e fix(llm): load system-prompt from config.csv correctly
All checks were successful
BotServer CI / build (push) Successful in 17m27s
- Move system_prompt retrieval inside spawn_blocking closure
- Include system_prompt in the return tuple to fix scope issue
- Add trace logging for debugging system-prompt loading
- GLM-5 and other LLM providers now correctly receive custom system prompts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-09 11:55:05 -03:00
1e71c9be09 feat: Add comprehensive stage progress logging
Some checks failed
BotServer CI / build (push) Failing after 2m39s
- Add detailed logging for all 5 pipeline stages (PLAN, BUILD, REVIEW, DEPLOY, MONITOR)
- Log stage start/complete events with agent IDs and progress details
- Add resource creation/deletion logging in drive_handlers
- Improve pipeline summary logging with task ID, nodes, resources, and URL

This addresses the requirement for textual progress in console logs.
2026-03-01 22:36:14 -03:00
0b1b17406d chore(core): various email and security updates 2026-02-24 19:02:48 -03:00
1856215d05 chore: update dependencies and formatting
All checks were successful
BotServer CI / build (push) Successful in 7m30s
2026-02-22 15:55:39 -03:00
de017241f2 fix: Complete security remediation - RCE and SSRF fixes
All checks were successful
BotServer CI / build (push) Successful in 7m34s
- Fixed RCE vulnerability in trusted_shell_script_arg execution
- Fixed SSRF vulnerability in GET command with internal IP blocking
- Updated SafeCommand to use explicit positional arguments

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 01:14:14 +00:00
3b21ab5ef9 fix: prevent duplicate message in chat when tool is executed
- Track tool_was_executed flag in stream_response
- Send empty content in final is_complete message when tool already sent results
- Prevents the LLM's pre-tool text from appearing twice in the chat UI
- DB message saving is unaffected (uses full_response_clone before the check)
2026-02-18 20:31:34 +00:00
b1118f977d fix: Correct parameter names in tool .bas files to match database schema
- Tool 06: Change tipoExibicao to tipoDescricao (matches pedidos_uso_imagem table)
- Tool 07: Change tipoExibicao to categoriaDescricao (matches licenciamentos table)
- Both tools now compile and execute successfully with database inserts

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-18 17:51:47 +00:00
c264ad1294 fix: Improve tool call handling and BASIC SELECT/CASE conversion
All checks were successful
BotServer CI / build (push) Successful in 9m34s
- Add tool_call_buffer to accumulate JSON chunks across multiple LLM responses
- Handle incomplete tool call JSON that spans multiple chunks
- Convert SELECT...CASE/END SELECT to Rhai match expressions
- Fix NOT IN operator conversion to !in for IF conditions
2026-02-16 09:30:19 +00:00
b92ef7c034 feat: Add detailed logging for session tools loading in WebSocket
All checks were successful
BotServer CI / build (push) Successful in 10m35s
Add error and warning logs to help diagnose why session tools are not
working in production. Logs now show:
- Number of tools loaded successfully
- Detailed error messages when tool loading fails
- Bot name lookup failures

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 23:57:17 +00:00
8257d49967 feat: Load session tools in WebSocket connection
Some checks failed
BotServer CI / build (push) Has been cancelled
This loads and sends the available tools to the client when establishing
a WebSocket connection. Tools are loaded based on the bot configuration
and sent in the initial welcome message.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 23:19:38 +00:00
a9cbbbffa0 fix: Use correct default LLM model name for local DeepSeek server
Changed default model from 'gpt-3.5-turbo' to 'DeepSeek-R1-Distill-Qwen-1.5B-Q3_K_M.gguf'
in bot message handler. This ensures the local llama-server receives the correct model
name and can process requests properly.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 20:23:04 +00:00
8a8008a72c fix: Resolve compilation errors
All checks were successful
BotServer CI / build (push) Successful in 13m3s
- Escape format placeholders in designer_ai.rs ({{botname}})
- Remove undefined 'prefix' filter in drive_monitor
- Fix type mismatch in use_tool.rs (str vs &String)
- Remove unused TextExpressionMethods import

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 22:15:35 +00:00
e34848507d fix: Update multiple modules for i18n and drive monitoring
Some checks failed
BotServer CI / build (push) Failing after 6m22s
- Update auto_task modules (app_generator, designer_ai, intent_classifier)
- Refactor use_tool.rs for better structure
- Update bot core and website crawler
- Improve drive_monitor and local_file_monitor
- Update bootstrap module

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 22:06:57 +00:00
4b44602d39 Update bootstrap and core modules
All checks were successful
BotServer CI / build (push) Successful in 10m27s
2026-02-14 09:54:14 +00:00
5ea171d126 Refactor: Split large files into modular subdirectories
Some checks failed
BotServer CI / build (push) Failing after 1m34s
Split 20+ files over 1000 lines into focused subdirectories for better
maintainability and code organization. All changes maintain backward
compatibility through re-export wrappers.

Major splits:
- attendance/llm_assist.rs (2074→7 modules)
- basic/keywords/face_api.rs → face_api/ (7 modules)
- basic/keywords/file_operations.rs → file_ops/ (8 modules)
- basic/keywords/hear_talk.rs → hearing/ (6 modules)
- channels/wechat.rs → wechat/ (10 modules)
- channels/youtube.rs → youtube/ (5 modules)
- contacts/mod.rs → contacts_api/ (6 modules)
- core/bootstrap/mod.rs → bootstrap/ (5 modules)
- core/shared/admin.rs → admin_*.rs (5 modules)
- designer/canvas.rs → canvas_api/ (6 modules)
- designer/mod.rs → designer_api/ (6 modules)
- docs/handlers.rs → handlers_api/ (11 modules)
- drive/mod.rs → drive_handlers.rs, drive_types.rs
- learn/mod.rs → types.rs
- main.rs → main_module/ (7 modules)
- meet/webinar.rs → webinar_api/ (8 modules)
- paper/mod.rs → (10 modules)
- security/auth.rs → auth_api/ (7 modules)
- security/passkey.rs → (4 modules)
- sources/mod.rs → sources_api/ (5 modules)
- tasks/mod.rs → task_api/ (5 modules)

Stats: 38,040 deletions, 1,315 additions across 318 files

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 21:09:30 +00:00
76abcea5e9 Fix start.bas repeated execution and add tool calling system
- Add Redis-based tracking to prevent start.bas from running repeatedly
  when clicking suggestion buttons. start.bas now executes only once per
  session with a 24-hour expiration on the tracking key.

- Add generic tool executor (ToolExecutor) for parsing and executing
  tool calls from any LLM provider. Works with Claude, OpenAI, and
  other providers that use standard tool calling formats.

- Update both start.bas execution paths (WebSocket handler and LLM
  message handler) to check Redis before executing.

- Fix suggestion duplication by clearing suggestions from Redis after
  fetching them.

- Add rate limiter for LLM API calls.

- Improve error handling and logging throughout.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 13:49:54 +00:00
6215908536 Add ConfigWatcher and fix model routing
New features:
- Add ConfigWatcher for hot-reloading config.csv from ~/data
- Add LocalFileMonitor for watching ~/data/*.gbai directories
- Add GLM LLM provider implementation
- Add tool context for LLM tool calling

Bug fixes:
- Fix model routing to respect session → bot → default hierarchy
- Fix ConfigWatcher to handle local embedded (llm-server=true)
- Skip DriveMonitor for default bot (managed via ConfigWatcher)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:10:27 +00:00
355215c2a2 Update: refactor migrations, update source files, and add new features 2026-02-04 13:29:29 -03:00
5fb4c889b7 fix(llm-config): Fix ConfigManager fallback logic for LLM configuration
Some checks failed
GBCI / build (push) Failing after 12m26s
- Fix ConfigManager to treat 'none', 'null', 'n/a', and empty values as placeholders
  and fall back to default bot's configuration instead of using these as literal values

- Fix ConfigManager to detect local file paths (e.g., .gguf, .bin, ../) and fall back
  to default bot's model when using remote API, allowing bots to keep local model
  config for local LLM server while automatically using remote model for API calls

- Fix get_default_bot() to return the bot actually named 'default' instead of
  the first active bot by ID, ensuring consistent fallback behavior

- Add comprehensive debug logging to trace LLM configuration from database to API call

This fixes the issue where bots with incomplete or local LLM configuration would
fail with 401/400 errors when trying to use remote API, instead of automatically
falling back to the default bot's configuration from config.csv.

Closes: #llm-config-fallback
2026-02-02 19:20:37 -03:00
26963f2caf Fix bot_id: Use bot_id from URL path instead of client message
- Extract bot_name from WebSocket query parameters
- Look up bot_id from bot_name using database
- Pass bot_id to WebSocket message handler
- Use session's bot_id for LLM configuration instead of client-provided bot_id
- Fixes issue where client sends 'default' bot_id when accessing /edu
2026-01-28 17:18:22 -03:00
0a24cd4b50 Fix build errors and unused imports in core, security and package_manager modules 2026-01-24 22:04:47 -03:00
6fa52e1dd8 feat: implement feature bundling architecture and fix conditional compilation
- Restructured Cargo.toml with Bundle Pattern for easy feature selection
- Added feature bundles: tasks → automation + drive + monitoring
- Applied conditional compilation guards throughout codebase:
  * AppState fields (drive, cache, task_engine, task_scheduler)
  * main.rs initialization (S3, Redis, Tasks)
  * SessionManager Redis usage
  * bootstrap S3/Drive operations
  * compiler task scheduling
  * shared module Task/NewTask exports
- Eliminated all botserver compilation warnings
- Minimal build now compiles successfully
- Accepted core dependencies: automation (Rhai), drive (S3), cache (Redis)
- Created DEPENDENCY_FIX_PLAN.md with complete documentation

Minimal feature set: chat + automation + drive + cache
Verified: cargo check -p botserver --no-default-features --features minimal 
2026-01-23 13:14:20 -03:00
66abce913f Feature gating refactor: modular compilation with minimal feature set 2026-01-22 19:45:18 -03:00
5919aa6bf0 Add video module, RBAC, security features, billing, contacts, dashboards, learn, social, and multiple new modules
Major additions:
- Video editing engine with AI features (transcription, captions, TTS, scene detection)
- RBAC middleware and organization management
- Security enhancements (MFA, passkey, DLP, encryption, audit)
- Billing and subscription management
- Contacts management
- Dashboards module
- Learn/LMS module
- Social features
- Compliance (SOC2, SOP middleware, vulnerability scanner)
- New migrations for RBAC, learn, and video tables
2026-01-08 13:16:17 -03:00
a5dee11002 Security audit: Remove all production .unwrap()/.expect(), add SafeCommand, ErrorSanitizer
- Phase 1 Critical: All 115 .unwrap() verified in test code only
- Phase 1 Critical: All runtime .expect() converted to proper error handling
- Phase 2 H1: Antivirus commands now use SafeCommand (added which/where to whitelist)
- Phase 2 H2: db_api.rs error responses use log_and_sanitize()
- Phase 2 H5: Removed duplicate sanitize_identifier (re-exports from sql_guard)

32 files modified for security hardening.
Moon deployment criteria: 10/10 met
2025-12-28 21:26:08 -03:00
c67aaa677a feat(security): Complete security infrastructure implementation
SECURITY MODULES ADDED:
- security/auth.rs: Full RBAC with roles (Anonymous, User, Moderator, Admin, SuperAdmin, Service, Bot, BotOwner, BotOperator, BotViewer) and permissions
- security/cors.rs: Hardened CORS (no wildcard in production, env-based config)
- security/panic_handler.rs: Panic catching middleware with safe 500 responses
- security/path_guard.rs: Path traversal protection, null byte prevention
- security/request_id.rs: UUID request tracking with correlation IDs
- security/error_sanitizer.rs: Sensitive data redaction from responses
- security/zitadel_auth.rs: Zitadel token introspection and role mapping
- security/sql_guard.rs: SQL injection prevention with table whitelist
- security/command_guard.rs: Command injection prevention
- security/secrets.rs: Zeroizing secret management
- security/validation.rs: Input validation utilities
- security/rate_limiter.rs: Rate limiting with governor crate
- security/headers.rs: Security headers (CSP, HSTS, X-Frame-Options)

MAIN.RS UPDATES:
- Replaced tower_http::cors::Any with hardened create_cors_layer()
- Added panic handler middleware
- Added request ID tracking middleware
- Set global panic hook

SECURITY STATUS:
- 0 unwrap() in production code
- 0 panic! in production code
- 0 unsafe blocks
- cargo audit: PASS (no vulnerabilities)
- Estimated completion: ~98%

Remaining: Wire auth middleware to handlers, audit logs for sensitive data
2025-12-28 19:29:18 -03:00
14b7cf70af feat(autotask): Implement AutoTask system with intent classification and app generation
- Add IntentClassifier with 7 intent types (APP_CREATE, TODO, MONITOR, ACTION, SCHEDULE, GOAL, TOOL)
- Add AppGenerator with LLM-powered app structure analysis
- Add DesignerAI for modifying apps through conversation
- Add app_server for serving generated apps with clean URLs
- Add db_api for CRUD operations on bot database tables
- Add ask_later keyword for pending info collection
- Add migration 6.1.1 with tables: pending_info, auto_tasks, execution_plans, task_approvals, task_decisions, safety_audit_log, generated_apps, intent_classifications, designer_changes
- Write apps to S3 drive and sync to SITE_ROOT for serving
- Clean URL structure: /apps/{app_name}/
- Integrate with DriveMonitor for file sync

Based on Chapter 17 - Autonomous Tasks specification
2025-12-27 21:10:09 -03:00
5da86bbef2 Fix clippy warnings: match arms, async/await, Debug impls, formatting
- Fix match arms with identical bodies by consolidating patterns
- Fix case-insensitive file extension comparisons using eq_ignore_ascii_case
- Fix unnecessary Debug formatting in log/format macros
- Fix clone_from usage instead of clone assignment
- Fix let...else patterns where appropriate
- Fix format! append to String using write! macro
- Fix unwrap_or with function calls to use unwrap_or_else
- Add missing fields to manual Debug implementations
- Fix duplicate code in if blocks
- Add type aliases for complex types
- Rename struct fields to avoid common prefixes
- Various other clippy warning fixes

Note: Some 'unused async' warnings remain for functions that are
called with .await but don't contain await internally - these are
kept async for API compatibility.
2025-12-26 08:59:25 -03:00
883c6d07e1 Remove all code comments and fix ratatui version 2025-12-23 18:40:58 -03:00
512ddc5f75 Fix LLM 400 error - remove invalid bot_id from messages array
The bot_id object was being pushed into the messages array sent to the LLM,
but LLM APIs expect messages to only contain role/content objects.
2025-12-10 18:29:34 -03:00