Commit graph

272 commits

Author SHA1 Message Date
1bbb94d500 fix(directory): improve error message for invalid credentials
Some checks failed
BotServer CI / build (push) Failing after 2m34s
- Updated error message to explain when credentials are found but invalid
- Clarified that credentials might be from a previous installation
- Added step-by-step solution for resetting credentials
- Removed misleading 'No admin credentials found' message

The error now accurately reflects the actual problem: authentication
failure rather than missing credentials.
2026-03-01 10:17:30 -03:00
8016caa790 fix(bootstrap): create OAuth client when Zitadel is already running
Some checks failed
BotServer CI / build (push) Failing after 3m20s
- Added OAuth client creation to the 'already running' branch
- Previously, OAuth client was only created when Zitadel was started
- This fixes the issue where OAuth client wasn't created on subsequent runs

Fixes the OAuth client creation issue discovered during testing.
2026-03-01 10:11:42 -03:00
2a6c599c75 feat(directory): read admin credentials from ~/.gb-setup-credentials
Some checks failed
BotServer CI / build (push) Failing after 5m3s
- Updated setup_directory() to read credentials from saved file
- Added read_saved_credentials() to parse ~/.gb-setup-credentials
- Added get_admin_credentials() to try multiple sources
- Removed default credentials approach (doesn't work)
- Improved error messages with solution steps

This matches the working approach from commit 86cfccc2 where
credentials were saved during first bootstrap and reused for
OAuth client creation on subsequent runs.
2026-03-01 10:06:30 -03:00
eb5c12c466 fix(directory): add .await to ensure_admin_token() calls
Some checks failed
BotServer CI / build (push) Failing after 5m28s
Fixed compilation errors by adding .await to all ensure_admin_token() calls:
- create_organization()
- create_user()
- save_config()

The method was made async but the calls weren't updated.
2026-03-01 09:52:31 -03:00
2c92a81302 merge: Unify master into main - all commits unified
Some checks failed
BotServer CI / build (push) Failing after 6m9s
2026-03-01 07:43:07 -03:00
8f495c75ec WIP: Local changes before merging master into main 2026-03-01 07:40:11 -03:00
bbdf243c86 fix: OAuth client creation during bootstrap
Some checks failed
BotServer CI / build (push) Failing after 6m2s
- Add password grant authentication support in DirectorySetup
- Extract initial admin credentials from Zitadel log file
- Fix race condition in Zitadel startup (wait for health check before starting)
- Create parent directories before saving config
- Add retry logic for OAuth client creation
- Improve error handling with detailed messages

Fixes authentication service not configured error after bootstrap.
2026-03-01 07:31:17 -03:00
0b1b17406d chore(core): various email and security updates 2026-02-24 19:02:48 -03:00
764f058653 fix: update work directory paths to use botserver-stack/data/system/work
All checks were successful
BotServer CI / build (push) Successful in 7m4s
Updated all hardcoded work/ directory references to use the correct
relative path from the current working directory:
- botserver-stack/data/system/work

This ensures consistent file location resolution regardless of where
botserver is run from (/home/rodriguez/src/gb/ or /opt/gbo/bin/).

Changes:
- local_file_monitor.rs: Use std::env::current_dir() for work_root
- drive_monitor/mod.rs: Use work_root PathBuf for tool compilation
- website_crawler_service.rs: Use std::env::current_dir() for work_path

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-22 16:20:07 -03:00
1856215d05 chore: update dependencies and formatting
All checks were successful
BotServer CI / build (push) Successful in 7m30s
2026-02-22 15:55:39 -03:00
affb702e90 fix(tools): auto-create work directory if missing in production
All checks were successful
BotServer CI / build (push) Successful in 8m12s
Added logic to create botserver-stack/data/system/work directory
if it doesn't exist. This ensures production deployments work
without manual directory setup.

Changes:
- Added fs::create_dir_all() in use_tool.rs
- Added fs::create_dir_all() in tool_context.rs
- Logs when directory is created
- Fixes production deployment where /system/work may not exist

This ensures the tool loading works in fresh production environments
where the work folder hasn't been populated yet.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-22 12:58:58 -03:00
febfa2e708 fix(tools): use relative path for tool .mcp.json files
All checks were successful
BotServer CI / build (push) Successful in 8m37s
Changed tool loading to use relative path from current directory instead
of hardcoded HOME/gb path. This makes the code portable across different
deployment environments.

- Updated use_tool.rs to use std::env::current_dir()
- Updated tool_context.rs to use std::env::current_dir()
- Added PathBuf import to both files
- Tools now load from botserver-stack/data/system/work/

Fixes issue where tools weren't being loaded because .mcp.json files
were in a different location than expected.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-02-22 11:21:07 -03:00
de017241f2 fix: Complete security remediation - RCE and SSRF fixes
All checks were successful
BotServer CI / build (push) Successful in 7m34s
- Fixed RCE vulnerability in trusted_shell_script_arg execution
- Fixed SSRF vulnerability in GET command with internal IP blocking
- Updated SafeCommand to use explicit positional arguments

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 01:14:14 +00:00
e143968179 feat: Add JWT secret rotation and health verification
SEC-02: Implement credential rotation security improvements

- Add JWT secret rotation to rotate-secret command
- Generate 64-character HS512-compatible secrets
- Automatic .env backup with timestamp
- Atomic file updates via temp+rename pattern
- Add health verification for rotated credentials
- Route rotate-secret, rotate-secrets, vault commands in CLI
- Add verification attempts for database and JWT endpoints

Security improvements:
- JWT_SECRET now rotatable (previously impossible)
- Automatic rollback via backup files
- Health checks catch configuration errors
- Clear warnings about token invalidation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-19 19:42:41 +00:00
ac5b814536 fix(security): Fix unsafe code, CORS logic, and expect usage 2026-02-19 12:06:05 +00:00
d7211a6c19 fix: Resolve unused import and variable warnings 2026-02-19 11:48:17 +00:00
3b21ab5ef9 fix: prevent duplicate message in chat when tool is executed
- Track tool_was_executed flag in stream_response
- Send empty content in final is_complete message when tool already sent results
- Prevents the LLM's pre-tool text from appearing twice in the chat UI
- DB message saving is unaffected (uses full_response_clone before the check)
2026-02-18 20:31:34 +00:00
b1118f977d fix: Correct parameter names in tool .bas files to match database schema
- Tool 06: Change tipoExibicao to tipoDescricao (matches pedidos_uso_imagem table)
- Tool 07: Change tipoExibicao to categoriaDescricao (matches licenciamentos table)
- Both tools now compile and execute successfully with database inserts

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-18 17:51:47 +00:00
848b875698 fix: Use PORT env var instead of BOTSERVER_PORT, default to port 9000
- Change BOTSERVER_PORT to PORT for consistency with .env.embedded
- Update default port from 8080 to 9000 in config
- Fix service port references in security integration
- Update directory setup ExternalPort to 9000

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 14:43:16 +00:00
c264ad1294 fix: Improve tool call handling and BASIC SELECT/CASE conversion
All checks were successful
BotServer CI / build (push) Successful in 9m34s
- Add tool_call_buffer to accumulate JSON chunks across multiple LLM responses
- Handle incomplete tool call JSON that spans multiple chunks
- Convert SELECT...CASE/END SELECT to Rhai match expressions
- Fix NOT IN operator conversion to !in for IF conditions
2026-02-16 09:30:19 +00:00
4ca7e5da40 fix: Handle tool calls in JSON array format
All checks were successful
BotServer CI / build (push) Successful in 9m25s
Fixed issue where LLM tool calls returned as JSON arrays were not being
detected and were displayed as raw JSON in the chat instead of being executed.

The parse_tool_call method now handles:
- Single tool call objects
- Arrays of tool calls (OpenAI standard format)

This prevents tool call JSON from appearing in the chat window and ensures
tools are executed properly.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-16 00:19:03 +00:00
b92ef7c034 feat: Add detailed logging for session tools loading in WebSocket
All checks were successful
BotServer CI / build (push) Successful in 10m35s
Add error and warning logs to help diagnose why session tools are not
working in production. Logs now show:
- Number of tools loaded successfully
- Detailed error messages when tool loading fails
- Bot name lookup failures

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 23:57:17 +00:00
8257d49967 feat: Load session tools in WebSocket connection
Some checks failed
BotServer CI / build (push) Has been cancelled
This loads and sends the available tools to the client when establishing
a WebSocket connection. Tools are loaded based on the bot configuration
and sent in the initial welcome message.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 23:19:38 +00:00
a9cbbbffa0 fix: Use correct default LLM model name for local DeepSeek server
Changed default model from 'gpt-3.5-turbo' to 'DeepSeek-R1-Distill-Qwen-1.5B-Q3_K_M.gguf'
in bot message handler. This ensures the local llama-server receives the correct model
name and can process requests properly.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 20:23:04 +00:00
e9a428ab1c fix: Auto-create bot database when not configured
Modified get_bot_pool() to automatically create the database for a bot
if it doesn't exist, instead of failing with "No database configured" error.

This fixes the issue where bots created after the initial sync don't have
a database_name set in the bots table, causing table creation to fail.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 14:57:22 +00:00
0c9665dd8b fix: Enable vector_db by default with health check and fallback to local LLM
- Add vector_db_health_check() function to verify Qdrant availability
- Add wait loop for vector_db startup in bootstrap (15 seconds)
- Fallback to local LLM when external URL configured but no API key provided
- Prevent external LLM (api.z.ai) usage without authentication key

This fixes the production issues:
- Qdrant vector database not available at https://localhost:6333
- External LLM being used instead of local when no key is configured
- Ensures vector_db is properly started and ready before use

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 14:54:17 +00:00
fc34461b2f fix: Improve cache_health_check TCP test to eliminate false positives
All checks were successful
BotServer CI / build (push) Successful in 8m28s
The previous /dev/tcp test was giving false positives, reporting that
Valkey was running when it was actually down. This caused bootstrap to
skip starting Valkey, leading to botserver hanging on cache connection.

Changes:
- Use nc (netcat) with -z flag for reliable port checking
- Final fallback: /dev/tcp with actual PING/PONG verification
- Only returns true if port is open AND responds correctly

This ensures cache_health_check() accurately reports Valkey status.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 12:02:23 +00:00
d243666cc2 fix: Support valkey-cli in cache_health_check for Valkey-only setups
All checks were successful
BotServer CI / build (push) Successful in 8m19s
- Try valkey-cli first (preferred for Valkey installations)
- Fall back to redis-cli (for Redis installations)
- Fall back to TCP connection test (works for both)

This fixes environments that only have Valkey installed without
Redis symlinks or redis-cli.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 11:48:02 +00:00
ba70bd97bc fix: Bootstrap should check Valkey is running and use correct component name
All checks were successful
BotServer CI / build (push) Successful in 8m56s
- Fix component name mismatch: "redis" -> "cache" in bootstrap_manager
- Add cache_health_check() function to verify Valkey is responding
- Add health check loop after starting cache (12s wait with PING test)
- Ensures cache is ready before proceeding with bootstrap

This fixes the issue where botserver would hang waiting for cache
connection because the cache component was never started.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 11:26:27 +00:00
dec3a086ab fix: Extract Valkey to cache/bin/ directory
All checks were successful
BotServer CI / build (push) Successful in 8m1s
- Binaries at cache/bin/valkey-server (correct production path)
- Use --strip-components=1 for extraction
- Matches /opt/gbo/bin/botserver-stack/bin/cache/bin/

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:22:59 +00:00
bfbdf8396f fix: Extract Valkey binaries directly to cache/ directory
Some checks failed
BotServer CI / build (push) Has been cancelled
- Use --strip-components=2 to flatten tarball structure
- Binaries go to cache/valkey-server (not cache/bin/valkey-server)
- Matches production path: /opt/gbo/bin/botserver-stack/bin/cache/

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:21:20 +00:00
0af972eda6 fix: Use correct valkey-server binary path
Some checks failed
BotServer CI / build (push) Has been cancelled
- Binary is at cache/bin/bin/valkey-server after extraction
- Update exec_cmd and check_cmd to use bin/ subdirectory
- Create symlinks at parent level for convenience

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:20:56 +00:00
14b3a0e6c0 fix: Remove redis symlinks from Valkey installation
Some checks failed
BotServer CI / build (push) Has been cancelled
- Use valkey-server and valkey-cli directly
- No redis compatibility symlinks needed
- Simplifies installation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:20:08 +00:00
4e4aca0986 fix: Use precompiled Valkey binary from download.valkey.io
All checks were successful
BotServer CI / build (push) Successful in 7m53s
- Update 3rdparty.toml to download valkey-9.0.2-jammy-x86_64.tar.gz
- Remove source compilation approach (no gcc/make required)
- Update installer to handle precompiled binaries directly
- Fix symlinks for redis compatibility

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 23:51:33 +00:00
8a8008a72c fix: Resolve compilation errors
All checks were successful
BotServer CI / build (push) Successful in 13m3s
- Escape format placeholders in designer_ai.rs ({{botname}})
- Remove undefined 'prefix' filter in drive_monitor
- Fix type mismatch in use_tool.rs (str vs &String)
- Remove unused TextExpressionMethods import

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 22:15:35 +00:00
e34848507d fix: Update multiple modules for i18n and drive monitoring
Some checks failed
BotServer CI / build (push) Failing after 6m22s
- Update auto_task modules (app_generator, designer_ai, intent_classifier)
- Refactor use_tool.rs for better structure
- Update bot core and website crawler
- Improve drive_monitor and local_file_monitor
- Update bootstrap module

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 22:06:57 +00:00
23c93cbb3d feat: Restore .env file creation for local Vault bootstrap
All checks were successful
BotServer CI / build (push) Successful in 9m6s
- Add TLS certificate generation to Vault post-install commands (CA, server, client, PostgreSQL)
- Add initialize_vault_local() function to handle Vault initialization for local installs
- Add ensure_env_file_exists() function to create .env when Vault already initialized
- Modify start() method to call Vault initialization after successful start (local mode)
- Fix Vault CLI flags: use -tls-skip-verify (not -skip-verify or -skip-tls-verify)

This restores the behavior where .env is automatically created with VAULT_ADDR,
VAULT_TOKEN, and VAULT_CACERT during local bootstrap, matching the LXC container
deployment behavior in facade.rs.

Fixes issue where .env file was only created for LXC deployments but not for
local installations.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 16:35:24 +00:00
0c44f7784d Debug: Add logging for missing translation keys
All checks were successful
BotServer CI / build (push) Successful in 10m8s
2026-02-14 12:25:46 +00:00
4b44602d39 Update bootstrap and core modules
All checks were successful
BotServer CI / build (push) Successful in 10m27s
2026-02-14 09:54:14 +00:00
17cb4ef147 Fix: Update source files and fix notify dependency reference
All checks were successful
BotServer CI / build (push) Successful in 11m35s
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-13 22:30:57 +00:00
5ea171d126 Refactor: Split large files into modular subdirectories
Some checks failed
BotServer CI / build (push) Failing after 1m34s
Split 20+ files over 1000 lines into focused subdirectories for better
maintainability and code organization. All changes maintain backward
compatibility through re-export wrappers.

Major splits:
- attendance/llm_assist.rs (2074→7 modules)
- basic/keywords/face_api.rs → face_api/ (7 modules)
- basic/keywords/file_operations.rs → file_ops/ (8 modules)
- basic/keywords/hear_talk.rs → hearing/ (6 modules)
- channels/wechat.rs → wechat/ (10 modules)
- channels/youtube.rs → youtube/ (5 modules)
- contacts/mod.rs → contacts_api/ (6 modules)
- core/bootstrap/mod.rs → bootstrap/ (5 modules)
- core/shared/admin.rs → admin_*.rs (5 modules)
- designer/canvas.rs → canvas_api/ (6 modules)
- designer/mod.rs → designer_api/ (6 modules)
- docs/handlers.rs → handlers_api/ (11 modules)
- drive/mod.rs → drive_handlers.rs, drive_types.rs
- learn/mod.rs → types.rs
- main.rs → main_module/ (7 modules)
- meet/webinar.rs → webinar_api/ (8 modules)
- paper/mod.rs → (10 modules)
- security/auth.rs → auth_api/ (7 modules)
- security/passkey.rs → (4 modules)
- sources/mod.rs → sources_api/ (5 modules)
- tasks/mod.rs → task_api/ (5 modules)

Stats: 38,040 deletions, 1,315 additions across 318 files

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 21:09:30 +00:00
3566a8c87f Fix batismo tool natural language processing
- Enhanced conversational input handling for batismo tool
- Improved keyword extraction and format recognition
- Fixed field extraction from informal user messages
- Better natural language understanding

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 20:17:49 +00:00
76abcea5e9 Fix start.bas repeated execution and add tool calling system
- Add Redis-based tracking to prevent start.bas from running repeatedly
  when clicking suggestion buttons. start.bas now executes only once per
  session with a 24-hour expiration on the tracking key.

- Add generic tool executor (ToolExecutor) for parsing and executing
  tool calls from any LLM provider. Works with Claude, OpenAI, and
  other providers that use standard tool calling formats.

- Update both start.bas execution paths (WebSocket handler and LLM
  message handler) to check Redis before executing.

- Fix suggestion duplication by clearing suggestions from Redis after
  fetching them.

- Add rate limiter for LLM API calls.

- Improve error handling and logging throughout.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 13:49:54 +00:00
6215908536 Add ConfigWatcher and fix model routing
New features:
- Add ConfigWatcher for hot-reloading config.csv from ~/data
- Add LocalFileMonitor for watching ~/data/*.gbai directories
- Add GLM LLM provider implementation
- Add tool context for LLM tool calling

Bug fixes:
- Fix model routing to respect session → bot → default hierarchy
- Fix ConfigWatcher to handle local embedded (llm-server=true)
- Skip DriveMonitor for default bot (managed via ConfigWatcher)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 15:10:27 +00:00
Rodrigo Rodriguez
fc0926ffff WIP: Multiple code improvements from previous session
- Fix various compiler warnings
- Update analytics, auto_task, and basic keywords
- Improve security, channels, and core modules
- Update designer, directory, and drive modules
- Fix embedded UI and LLM modules

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 12:25:37 +00:00
Claude Sonnet 4.5
e8ce642b81 Update default port from 8088 to 9000
- Update .env.embedded PORT variable
- Update README.md documentation
- Update deploy/README.md port references
- Update scripts/deploy-embedded.sh kiosk URLs
- Update src/directory/bootstrap.rs login URLs
- Update src/core/package_manager/setup/directory_setup.rs auth callback URLs

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 12:13:42 +00:00
84458b2a69 feat: Add BOTSERVER_PORT environment variable override
All checks were successful
BotServer CI / build (push) Successful in 12m37s
- Check BOTSERVER_PORT env var before database config
- Override server_port from database if BOTSERVER_PORT is set
- Apply to both from_database() and from_env() config paths
- Allows easy port configuration via environment variable
2026-02-06 12:56:52 -03:00
7da1442c91 Fix PostgreSQL readiness check timeout in bootstrap
All checks were successful
BotServer CI / build (push) Successful in 13m13s
- Use local pg_isready path when available (./botserver-stack/bin/tables/bin/pg_isready)
- Fall back to system pg_isready if local binary not found
- Prevents 30-second timeout during bootstrap when PostgreSQL is actually running
- Applied to both readiness checks in start_all() method
2026-02-05 22:23:04 -03:00
07ff7a6297 Fix PostgreSQL health check - remove database-specific check
All checks were successful
BotServer CI / build (push) Successful in 12m18s
- Removed -d 'postgres' parameter from pg_isready health checks
- Health check now only verifies server connection on port 5432
- Fixes false positive failures when PostgreSQL is running but specific database has issues
- PostgreSQL logs showed 'database system is ready' but health check was failing
2026-02-05 21:55:47 -03:00
de3e5c4fec Fix PostgreSQL ready check for already-running instances
All checks were successful
BotServer CI / build (push) Successful in 14m23s
Add pg_isready health check to the 'already running' branch to ensure
PostgreSQL is properly detected as ready, even when running as a
non-interactive user (sudo -u gbuser).

This complements the previous fix for fresh PostgreSQL starts.
2026-02-05 21:09:23 -03:00