Commit graph

122 commits

Author SHA1 Message Date
68ef554132 fix: Vault as single source of truth - credentials + location for all services
All checks were successful
BotServer CI/CD / build (push) Successful in 4m53s
- Qdrant health check: recognize 'healthz check passed' response (fixes 45s timeout)
- seed_vault_defaults: add host/port/url/grpc_port for ALL 10 services
- fetch_vault_credentials: fetch ALL services via generic loop (drive, cache, tables, vectordb, directory, llm, meet, alm, encryption)
- vectordb URL: fix https://localhost:6334 -> http://localhost:6333 in all config getters
- get_from_env: add host/port/grpc_port for vectordb fallback
- Tested: .reset (fresh install) + .restart (idempotent) - zero errors
2026-04-01 16:46:16 -03:00
fb2e5242da fix: Vault seeding, service health checks, and restart idempotency
All checks were successful
BotServer CI/CD / build (push) Successful in 55m52s
- Replace hardcoded passwords with generate_random_string() for all Vault-seeded services
- Add valkey-cli, nc to SafeCommand allowlist; fix PATH in all 4 execution methods
- Fix empty Vault KV values ('none' placeholder) preventing 'Failed to parse K=V' errors
- Fix special chars in generated passwords triggering shell injection false positives
- Add ALM app.ini creation with absolute paths for Forgejo CLI
- Increase Qdrant timeout 15s→45s, ALM wait 5s→20s
- Persist file_states and kb_states to disk for .bas/KB idempotency across restarts
- Add duplicate check to use_website registration (debug log for existing)
- Remove dead code (SERVER_START_EPOCH, server_epoch)
- Add generate_random_string() to shared mod.rs, remove duplicates
2026-04-01 12:22:57 -03:00
3e46a16469 fix: Seed default credentials into Vault after initialization
Some checks failed
BotServer CI/CD / build (push) Failing after 3h13m28s
- Add seed_vault_defaults() to write default creds for all components
  (drive, cache, tables, directory, email, llm, encryption, meet, vectordb, alm)
- Call seed_vault_defaults() after KV2 enable in initialize_vault_local()
- Call seed_vault_defaults() in recover_existing_vault() for recovery path
- Rewrite fetch_vault_credentials() to use SafeCommand directly instead of
  safe_sh_command, avoiding '//' shell injection false positive on URLs
- Components like Drive now get credentials from Vault instead of 403 errors
2026-03-31 22:19:09 -03:00
9919a8321c fix: Use SafeCommand directly for vault health check to avoid shell injection false positive
All checks were successful
BotServer CI/CD / build (push) Successful in 6m46s
- Replace safe_sh_command with SafeCommand::new("curl").args() in vault_health_check()
- The URL contains https:// which triggered '//' pattern detection in shell command
- Direct SafeCommand bypasses shell parsing, URL passed as single argument
- Add vault data directory existence check before recovery attempt
- Prevents 'Dangerous pattern // detected' errors during bootstrap
2026-03-31 21:34:04 -03:00
07a6c1edb3 Merge commit '582ea634'
All checks were successful
BotServer CI/CD / build (push) Successful in 7m38s
2026-03-31 21:10:25 -03:00
582ea634e7 fix: Vault bootstrap recovery for sealed but initialized instances
- Fix vault_health_check() stub that always returned false
- Add recover_existing_vault() to handle Vault with existing data but no init.json
- Add unseal_vault() helper to unseal with existing vault-unseal-keys
- Detect initialized Vault via health endpoint or data directory presence
- Prevents bootstrap failure when reset.sh deletes init.json but Vault data persists

Root cause: vault_health_check() was a stub returning false, causing bootstrap
to always try vault operator init on already-initialized (but sealed) Vault,
which failed with connection refused. This cascaded to all services failing
to fetch credentials from Vault.
2026-03-31 20:49:29 -03:00
2fa59057fa fix: Resolve migration error, Vault 403, cache timeout, and shell injection false positives
Some checks failed
BotServer CI/CD / build (push) Has been cancelled
- Fix migration 6.2.5: Create lost_reason column before VIEW that references it
- Fix Vault 403: Enable KV2 secrets engine after initialization
- Fix cache timeout: Increase Valkey readiness wait from 12s to 30s
- Fix command_guard: Remove () from forbidden chars (safe in std::process::Command)
2026-03-31 19:55:16 -03:00
d19984fa07 feat: Improve KB keywords and package manager installer 2026-03-20 17:38:47 -03:00
57a8b7f8f0 Fix: use pgrep to check valkey/qdrant running state
- valkey check_cmd: replaced valkey-cli ping (network) with pgrep -x valkey-server
- qdrant check_cmd: replaced curl https check (TLS error 35) with pgrep -x qdrant
- Prevents duplicate instances on each botserver restart
2026-03-20 15:40:22 -03:00
346c83871a Fix Vault TLS certificate to include Subject Alternative Name for modern client compatibility 2026-03-18 09:30:27 -03:00
c340f95da1 security: bind MinIO and Valkey to 127.0.0.1 only
Some checks failed
BotServer CI / build (push) Failing after 6m44s
2026-03-17 01:32:21 -03:00
35b793d29c Add interactive vault put - prompt for secrets instead of CLI args
All checks were successful
BotServer CI / build (push) Successful in 10m50s
2026-03-16 22:27:06 -03:00
16394aef01 Fix LXD IPv4: retry DHCP 30s, fallback to static IP for privileged containers
Some checks failed
BotServer CI / build (push) Failing after 6m46s
2026-03-16 21:34:59 -03:00
7ef1efa047 feat: add setup-env command to generate .env from vault container 2026-03-15 21:15:48 -03:00
dfe5162f66 feat(install): add --container-only flag to install command
- Add --container-only flag to create container without completing full installation
- Exit immediately after container creation
- Useful for manual setup or debugging installation issues
2026-03-15 20:49:08 -03:00
d1cb6b758c Fix LXD container mode: PATH, socket proxy, exec
All checks were successful
BotServer CI / build (push) Successful in 10m54s
2026-03-15 20:00:06 -03:00
ba53a0c178 Fix LXD socket handling in container mode
All checks were successful
BotServer CI / build (push) Successful in 11m6s
2026-03-15 18:58:21 -03:00
ef426b7a50 LXD proxy and container improvements
Some checks failed
BotServer CI / build (push) Failing after 7m5s
2026-03-15 15:50:02 -03:00
7fb73e683f feat: add campaigns, attendance SLA, and marketing modules 2026-03-14 16:35:42 -03:00
13892b3157 Fix tenant-org-bot relationship and CRM lead form 2026-03-12 18:19:18 -03:00
c326581a9e fix(zitadel): resolve OAuth client initialization timing issue
- Fix PAT extraction timing with retry loop (waits up to 60s for PAT in logs)
- Add sync command to flush filesystem buffers before extraction
- Improve logging with progress messages and PAT verification
- Refactor setup code into consolidated setup.rs module
- Fix YAML indentation for PatPath and MachineKeyPath
- Change Zitadel init parameter from --config to --steps

The timing issue occurred because:
1. Zitadel writes PAT to logs at startup (~18:08:59)
2. Post-install extraction ran too early (~18:09:35)
3. PAT file wasn't created until ~18:10:38 (63s after installation)
4. OAuth client creation failed because PAT file didn't exist yet

With the retry loop:
- Waits for PAT to appear in logs with sync+grep check
- Extracts PAT immediately when found
- OAuth client creation succeeds
- directory_config.json saved with valid credentials
- Login flow works end-to-end

Tested: Full reset.sh and login verification successful
2026-03-01 19:06:09 -03:00
1bbb94d500 fix(directory): improve error message for invalid credentials
Some checks failed
BotServer CI / build (push) Failing after 2m34s
- Updated error message to explain when credentials are found but invalid
- Clarified that credentials might be from a previous installation
- Added step-by-step solution for resetting credentials
- Removed misleading 'No admin credentials found' message

The error now accurately reflects the actual problem: authentication
failure rather than missing credentials.
2026-03-01 10:17:30 -03:00
2a6c599c75 feat(directory): read admin credentials from ~/.gb-setup-credentials
Some checks failed
BotServer CI / build (push) Failing after 5m3s
- Updated setup_directory() to read credentials from saved file
- Added read_saved_credentials() to parse ~/.gb-setup-credentials
- Added get_admin_credentials() to try multiple sources
- Removed default credentials approach (doesn't work)
- Improved error messages with solution steps

This matches the working approach from commit 86cfccc2 where
credentials were saved during first bootstrap and reused for
OAuth client creation on subsequent runs.
2026-03-01 10:06:30 -03:00
eb5c12c466 fix(directory): add .await to ensure_admin_token() calls
Some checks failed
BotServer CI / build (push) Failing after 5m28s
Fixed compilation errors by adding .await to all ensure_admin_token() calls:
- create_organization()
- create_user()
- save_config()

The method was made async but the calls weren't updated.
2026-03-01 09:52:31 -03:00
2c92a81302 merge: Unify master into main - all commits unified
Some checks failed
BotServer CI / build (push) Failing after 6m9s
2026-03-01 07:43:07 -03:00
8f495c75ec WIP: Local changes before merging master into main 2026-03-01 07:40:11 -03:00
bbdf243c86 fix: OAuth client creation during bootstrap
Some checks failed
BotServer CI / build (push) Failing after 6m2s
- Add password grant authentication support in DirectorySetup
- Extract initial admin credentials from Zitadel log file
- Fix race condition in Zitadel startup (wait for health check before starting)
- Create parent directories before saving config
- Add retry logic for OAuth client creation
- Improve error handling with detailed messages

Fixes authentication service not configured error after bootstrap.
2026-03-01 07:31:17 -03:00
1856215d05 chore: update dependencies and formatting
All checks were successful
BotServer CI / build (push) Successful in 7m30s
2026-02-22 15:55:39 -03:00
e143968179 feat: Add JWT secret rotation and health verification
SEC-02: Implement credential rotation security improvements

- Add JWT secret rotation to rotate-secret command
- Generate 64-character HS512-compatible secrets
- Automatic .env backup with timestamp
- Atomic file updates via temp+rename pattern
- Add health verification for rotated credentials
- Route rotate-secret, rotate-secrets, vault commands in CLI
- Add verification attempts for database and JWT endpoints

Security improvements:
- JWT_SECRET now rotatable (previously impossible)
- Automatic rollback via backup files
- Health checks catch configuration errors
- Clear warnings about token invalidation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-19 19:42:41 +00:00
b1118f977d fix: Correct parameter names in tool .bas files to match database schema
- Tool 06: Change tipoExibicao to tipoDescricao (matches pedidos_uso_imagem table)
- Tool 07: Change tipoExibicao to categoriaDescricao (matches licenciamentos table)
- Both tools now compile and execute successfully with database inserts

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-18 17:51:47 +00:00
848b875698 fix: Use PORT env var instead of BOTSERVER_PORT, default to port 9000
- Change BOTSERVER_PORT to PORT for consistency with .env.embedded
- Update default port from 8080 to 9000 in config
- Fix service port references in security integration
- Update directory setup ExternalPort to 9000

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 14:43:16 +00:00
dec3a086ab fix: Extract Valkey to cache/bin/ directory
All checks were successful
BotServer CI / build (push) Successful in 8m1s
- Binaries at cache/bin/valkey-server (correct production path)
- Use --strip-components=1 for extraction
- Matches /opt/gbo/bin/botserver-stack/bin/cache/bin/

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:22:59 +00:00
bfbdf8396f fix: Extract Valkey binaries directly to cache/ directory
Some checks failed
BotServer CI / build (push) Has been cancelled
- Use --strip-components=2 to flatten tarball structure
- Binaries go to cache/valkey-server (not cache/bin/valkey-server)
- Matches production path: /opt/gbo/bin/botserver-stack/bin/cache/

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:21:20 +00:00
0af972eda6 fix: Use correct valkey-server binary path
Some checks failed
BotServer CI / build (push) Has been cancelled
- Binary is at cache/bin/bin/valkey-server after extraction
- Update exec_cmd and check_cmd to use bin/ subdirectory
- Create symlinks at parent level for convenience

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:20:56 +00:00
14b3a0e6c0 fix: Remove redis symlinks from Valkey installation
Some checks failed
BotServer CI / build (push) Has been cancelled
- Use valkey-server and valkey-cli directly
- No redis compatibility symlinks needed
- Simplifies installation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 01:20:08 +00:00
4e4aca0986 fix: Use precompiled Valkey binary from download.valkey.io
All checks were successful
BotServer CI / build (push) Successful in 7m53s
- Update 3rdparty.toml to download valkey-9.0.2-jammy-x86_64.tar.gz
- Remove source compilation approach (no gcc/make required)
- Update installer to handle precompiled binaries directly
- Fix symlinks for redis compatibility

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 23:51:33 +00:00
23c93cbb3d feat: Restore .env file creation for local Vault bootstrap
All checks were successful
BotServer CI / build (push) Successful in 9m6s
- Add TLS certificate generation to Vault post-install commands (CA, server, client, PostgreSQL)
- Add initialize_vault_local() function to handle Vault initialization for local installs
- Add ensure_env_file_exists() function to create .env when Vault already initialized
- Modify start() method to call Vault initialization after successful start (local mode)
- Fix Vault CLI flags: use -tls-skip-verify (not -skip-verify or -skip-tls-verify)

This restores the behavior where .env is automatically created with VAULT_ADDR,
VAULT_TOKEN, and VAULT_CACERT during local bootstrap, matching the LXC container
deployment behavior in facade.rs.

Fixes issue where .env file was only created for LXC deployments but not for
local installations.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-14 16:35:24 +00:00
5ea171d126 Refactor: Split large files into modular subdirectories
Some checks failed
BotServer CI / build (push) Failing after 1m34s
Split 20+ files over 1000 lines into focused subdirectories for better
maintainability and code organization. All changes maintain backward
compatibility through re-export wrappers.

Major splits:
- attendance/llm_assist.rs (2074→7 modules)
- basic/keywords/face_api.rs → face_api/ (7 modules)
- basic/keywords/file_operations.rs → file_ops/ (8 modules)
- basic/keywords/hear_talk.rs → hearing/ (6 modules)
- channels/wechat.rs → wechat/ (10 modules)
- channels/youtube.rs → youtube/ (5 modules)
- contacts/mod.rs → contacts_api/ (6 modules)
- core/bootstrap/mod.rs → bootstrap/ (5 modules)
- core/shared/admin.rs → admin_*.rs (5 modules)
- designer/canvas.rs → canvas_api/ (6 modules)
- designer/mod.rs → designer_api/ (6 modules)
- docs/handlers.rs → handlers_api/ (11 modules)
- drive/mod.rs → drive_handlers.rs, drive_types.rs
- learn/mod.rs → types.rs
- main.rs → main_module/ (7 modules)
- meet/webinar.rs → webinar_api/ (8 modules)
- paper/mod.rs → (10 modules)
- security/auth.rs → auth_api/ (7 modules)
- security/passkey.rs → (4 modules)
- sources/mod.rs → sources_api/ (5 modules)
- tasks/mod.rs → task_api/ (5 modules)

Stats: 38,040 deletions, 1,315 additions across 318 files

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 21:09:30 +00:00
3566a8c87f Fix batismo tool natural language processing
- Enhanced conversational input handling for batismo tool
- Improved keyword extraction and format recognition
- Fixed field extraction from informal user messages
- Better natural language understanding

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 20:17:49 +00:00
Rodrigo Rodriguez
fc0926ffff WIP: Multiple code improvements from previous session
- Fix various compiler warnings
- Update analytics, auto_task, and basic keywords
- Improve security, channels, and core modules
- Update designer, directory, and drive modules
- Fix embedded UI and LLM modules

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 12:25:37 +00:00
Claude Sonnet 4.5
e8ce642b81 Update default port from 8088 to 9000
- Update .env.embedded PORT variable
- Update README.md documentation
- Update deploy/README.md port references
- Update scripts/deploy-embedded.sh kiosk URLs
- Update src/directory/bootstrap.rs login URLs
- Update src/core/package_manager/setup/directory_setup.rs auth callback URLs

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-08 12:13:42 +00:00
3258f3ef85 Fix PostgreSQL startup failure for non-interactive users
All checks were successful
BotServer CI / build (push) Successful in 11m58s
Changed pg_isready checks from '-U gbuser' to '-d postgres' to properly
detect PostgreSQL readiness during bootstrap. The gbuser database doesn't
exist yet during startup, causing pg_isready to fail and bootstrap to timeout.

This fixes the issue when running botserver as a non-interactive user
(e.g., sudo -u gbuser).
2026-02-05 20:40:52 -03:00
35af28a041 Fix compilation errors: duplicate imports, feature-gated functions, type mismatch
Some checks failed
BotServer CI / build (push) Failing after 7m28s
2026-02-05 18:31:14 -03:00
dfb18f75e6 Fix bootstrap database creation - use BOOTSTRAP_DB_PASSWORD env var 2026-02-05 14:42:13 -03:00
355215c2a2 Update: refactor migrations, update source files, and add new features 2026-02-04 13:29:29 -03:00
b103c07248 Fix migration errors and reorganize migration files
- Fixed 'relation session_kb_associations does not exist' error in core consolidated migration.
- Renamed migration directories from timestamp-based to version-based (6.0.x, 6.1.x, 6.2.x).
- Reorganized migrations into dedicated feature folders (products, dashboards, learn, video).
- Updated migration execution order in core/shared/utils.rs.
- Moves legacy migrations to 6.0.x/6.1.x and workflow to 6.2.0.
2026-01-27 13:45:54 -03:00
3c279f43e5 refactor: embed 3rdparty.toml and remove hardcoded defaults from installer
All checks were successful
GBCI / build (push) Successful in 10m36s
2026-01-26 20:48:57 -03:00
933d42a200 Fix: Replace brace expansion with explicit paths in mkdir command and improve LXC error logging
All checks were successful
GBCI / build (push) Successful in 14m12s
2026-01-26 17:38:34 -03:00
b113267aef Refactor: Genericize default organization to 'system' and update tenant paths 2026-01-26 17:00:21 -03:00
ff4e6c4fe8 fix(lxc): configure lxd-sock proxy for brother mode and update installers
Some checks failed
GBCI / build (push) Failing after 47s
2026-01-26 11:44:18 -03:00