Commit graph

272 commits

Author SHA1 Message Date
3258f3ef85 Fix PostgreSQL startup failure for non-interactive users
All checks were successful
BotServer CI / build (push) Successful in 11m58s
Changed pg_isready checks from '-U gbuser' to '-d postgres' to properly
detect PostgreSQL readiness during bootstrap. The gbuser database doesn't
exist yet during startup, causing pg_isready to fail and bootstrap to timeout.

This fixes the issue when running botserver as a non-interactive user
(e.g., sudo -u gbuser).
2026-02-05 20:40:52 -03:00
717ca8c35b Fix: Remove undefined db_password variable in database creation
Some checks failed
BotServer CI / build (push) Failing after 11m39s
2026-02-05 19:06:59 -03:00
309c5bc55a Fix: Ensure botserver database exists before running migrations
Some checks are pending
BotServer CI / build (push) Waiting to run
2026-02-05 19:02:01 -03:00
35af28a041 Fix compilation errors: duplicate imports, feature-gated functions, type mismatch
Some checks failed
BotServer CI / build (push) Failing after 7m28s
2026-02-05 18:31:14 -03:00
24b516406a Add automatic log dump when component fails to start 2026-02-05 14:53:37 -03:00
dfb18f75e6 Fix bootstrap database creation - use BOOTSTRAP_DB_PASSWORD env var 2026-02-05 14:42:13 -03:00
025e6fe823 Improve error messages: show PostgreSQL log contents on startup failure 2026-02-05 14:13:31 -03:00
355215c2a2 Update: refactor migrations, update source files, and add new features 2026-02-04 13:29:29 -03:00
d26f0652e5 fix(website-crawler): Add missing crawl_status column to SELECT query
Some checks failed
GBCI / build (push) Failing after 14m19s
Fix the SQL query in check_and_crawl_websites() to also include crawl_status
in the SELECT clause. The WebsiteCrawlRecord struct expects this field but it was
only present in the WHERE clause, causing Diesel to fail with
'Column crawl_status was not present in query'.

This resolves the second website crawler service error.
2026-02-02 20:10:20 -03:00
841b59affd fix(website-crawler): Add missing next_crawl column to SELECT query
Some checks failed
GBCI / build (push) Failing after 12m2s
Fix the SQL query in check_and_crawl_websites() to include next_crawl in the
SELECT clause. The WebsiteCrawlRecord struct expects this field but it was
missing, causing Diesel to fail with 'Column next_crawl was not present in query'.

This resolves the website crawler service error that was preventing websites
from being properly queried and recrawled.
2026-02-02 19:58:11 -03:00
5fb4c889b7 fix(llm-config): Fix ConfigManager fallback logic for LLM configuration
Some checks failed
GBCI / build (push) Failing after 12m26s
- Fix ConfigManager to treat 'none', 'null', 'n/a', and empty values as placeholders
  and fall back to default bot's configuration instead of using these as literal values

- Fix ConfigManager to detect local file paths (e.g., .gguf, .bin, ../) and fall back
  to default bot's model when using remote API, allowing bots to keep local model
  config for local LLM server while automatically using remote model for API calls

- Fix get_default_bot() to return the bot actually named 'default' instead of
  the first active bot by ID, ensuring consistent fallback behavior

- Add comprehensive debug logging to trace LLM configuration from database to API call

This fixes the issue where bots with incomplete or local LLM configuration would
fail with 401/400 errors when trying to use remote API, instead of automatically
falling back to the default bot's configuration from config.csv.

Closes: #llm-config-fallback
2026-02-02 19:20:37 -03:00
39c4dba838 feat: Add template validation system with .valid file
- Modify bootstrap to read .valid file and validate templates before loading
- Templates not in .valid file are skipped during bootstrap
- Backward compatible: if .valid file missing, all templates are loaded
- Enables controlled template loading during bootstrap
2026-02-01 14:20:35 -03:00
748fceff5d Fix issues: remove unused import, fix ownership error, reduce crawler interval 2026-01-30 12:21:30 -03:00
94fede7cc4 feat: Add search_enabled and menu_launcher_enabled directives to .product file
- Add search_enabled field to ProductConfig to control omnibox visibility (defaults to false)
- Add menu_launcher_enabled field to ProductConfig to control apps menu button visibility (defaults to false)
- Update .product file to set both directives to false by default
- Update get_product_config_json to include new fields in API response
- Parse search_enabled and menu_launcher_enabled from .product file with support for true/false, 1/0, yes/no values

This allows disabling the suite search mechanism and hiding the menu launcher when empty,
providing a cleaner UI for deployments that don't need these features.
2026-01-29 23:55:50 -03:00
26963f2caf Fix bot_id: Use bot_id from URL path instead of client message
- Extract bot_name from WebSocket query parameters
- Look up bot_id from bot_name using database
- Pass bot_id to WebSocket message handler
- Use session's bot_id for LLM configuration instead of client-provided bot_id
- Fixes issue where client sends 'default' bot_id when accessing /edu
2026-01-28 17:18:22 -03:00
b103c07248 Fix migration errors and reorganize migration files
- Fixed 'relation session_kb_associations does not exist' error in core consolidated migration.
- Renamed migration directories from timestamp-based to version-based (6.0.x, 6.1.x, 6.2.x).
- Reorganized migrations into dedicated feature folders (products, dashboards, learn, video).
- Updated migration execution order in core/shared/utils.rs.
- Moves legacy migrations to 6.0.x/6.1.x and workflow to 6.2.0.
2026-01-27 13:45:54 -03:00
3c279f43e5 refactor: embed 3rdparty.toml and remove hardcoded defaults from installer
All checks were successful
GBCI / build (push) Successful in 10m36s
2026-01-26 20:48:57 -03:00
933d42a200 Fix: Replace brace expansion with explicit paths in mkdir command and improve LXC error logging
All checks were successful
GBCI / build (push) Successful in 14m12s
2026-01-26 17:38:34 -03:00
b113267aef Refactor: Genericize default organization to 'system' and update tenant paths 2026-01-26 17:00:21 -03:00
ff4e6c4fe8 fix(lxc): configure lxd-sock proxy for brother mode and update installers
Some checks failed
GBCI / build (push) Failing after 47s
2026-01-26 11:44:18 -03:00
a3babbcaa7 revert: removing production URL default logic (requested by user)
Some checks failed
GBCI / build (push) Has been cancelled
2026-01-25 22:48:47 -03:00
c643a59e29 config: use production URL as default when BOTSERVER_ENV=production
Some checks failed
GBCI / build (push) Failing after 5m28s
2026-01-25 22:42:15 -03:00
9087bb17cd feat: complete General Bots 7.0 (v6.2.0) 2026-01-25 10:29:54 -03:00
0a24cd4b50 Fix build errors and unused imports in core, security and package_manager modules 2026-01-24 22:04:47 -03:00
6fa52e1dd8 feat: implement feature bundling architecture and fix conditional compilation
- Restructured Cargo.toml with Bundle Pattern for easy feature selection
- Added feature bundles: tasks → automation + drive + monitoring
- Applied conditional compilation guards throughout codebase:
  * AppState fields (drive, cache, task_engine, task_scheduler)
  * main.rs initialization (S3, Redis, Tasks)
  * SessionManager Redis usage
  * bootstrap S3/Drive operations
  * compiler task scheduling
  * shared module Task/NewTask exports
- Eliminated all botserver compilation warnings
- Minimal build now compiles successfully
- Accepted core dependencies: automation (Rhai), drive (S3), cache (Redis)
- Created DEPENDENCY_FIX_PLAN.md with complete documentation

Minimal feature set: chat + automation + drive + cache
Verified: cargo check -p botserver --no-default-features --features minimal 
2026-01-23 13:14:20 -03:00
f8a907bd81 Update PROMPT.md and feature gating 2026-01-22 20:24:05 -03:00
66abce913f Feature gating refactor: modular compilation with minimal feature set 2026-01-22 19:45:18 -03:00
3db87c029d Modularize learn schema and clean up channels/project schema stubs 2026-01-22 14:01:50 -03:00
fc9c07d204 Refactor schema.rs into modular files 2026-01-22 13:57:40 -03:00
18b8afd54c O 2026-01-19 21:19:10 -03:00
4f63065da2 Update botserver submodule 2026-01-19 15:43:45 -03:00
5126c648ff Auto-commit: 20260118_195334 2026-01-18 19:53:34 -03:00
033bb504b9 Various updates: dependencies, features, and bug fixes 2026-01-16 11:29:22 -03:00
446dee26b6 Add extended product fields: dimensions, weight, tax codes, brand, inventory, pricing, SEO 2026-01-14 10:02:55 -03:00
95a7591c39 Add Stone Pagamentos fields: SKU, dimensões, peso, NCM, GTIN, tributos, etc 2026-01-14 09:50:39 -03:00
a2783f9b32 Fix 5 errors and 32 warnings: calendar, compliance, billing_alert_broadcast, unused vars 2026-01-13 22:21:25 -03:00
31777432b4 Implement TODO items: session auth, face API, task logs, intent storage
Learn Module:
- All 9 handlers now use AuthenticatedUser extractor

Security:
- validate_session_sync reads roles from SESSION_CACHE

AutoTask:
- get_task_logs reads from manifest with status logs
- store_compiled_intent saves to cache and database

Face API:
- AWS Rekognition, OpenCV, InsightFace implementations
- Detection, verification, analysis methods

Other fixes:
- Calendar/task integration database queries
- Recording database methods
- Analytics insights trends
- Email/folder monitoring mock data
2026-01-13 14:48:49 -03:00
a886478548 Implement database persistence for dashboards, legal, and compliance modules
- Add PostgreSQL persistence for dashboards module (was returning empty vec![])
  - Tables: dashboards, dashboard_widgets, dashboard_data_sources, dashboard_filters,
    dashboard_widget_data_sources, conversational_queries
  - Full CRUD operations with spawn_blocking pattern

- Add PostgreSQL persistence for legal module (was using in-memory HashMap)
  - Tables: legal_documents, legal_document_versions, cookie_consents, consent_history,
    legal_acceptances, data_deletion_requests, data_export_requests
  - GDPR-compliant consent tracking and document management

- Add PostgreSQL persistence for compliance module (was returning empty results)
  - Tables: compliance_checks, compliance_issues, compliance_audit_log, compliance_evidence,
    compliance_risk_assessments, compliance_risks, compliance_training_records,
    compliance_access_reviews
  - Support for GDPR, SOC2, ISO27001, HIPAA, PCI-DSS frameworks

- Add migration files for all new tables
- Update schema.rs with new table definitions and joinables
- Register new routes in main.rs
- Add recursion_limit = 512 for macro expansion
2026-01-13 00:07:22 -03:00
4ed05f3f19 feat(i18n): add missing translation keys to TRANSLATION_KEYS array
- Add people-* keys (title, subtitle, search, tabs, form fields)
- Add crm-* keys (stages, stats, metrics)
- Add billing-* keys (subtitle, new-payment, revenue metrics)
- Add products-* keys (subtitle, items, stats)
2026-01-12 14:13:35 -03:00
1686bfb454 feat(i18n): add missing navigation keys to TRANSLATION_KEYS
- Add nav-docs, nav-sheet, nav-slides, nav-social, nav-all-apps
- Add nav-people, nav-editor, nav-dashboards, nav-security
- Add nav-designer, nav-project, nav-canvas, nav-goals
- Add nav-player, nav-workspace, nav-video, nav-learn
- Add nav-crm, nav-billing, nav-products, nav-tickets
2026-01-10 10:53:56 -03:00
e3b3f04206 Normalize API paths: remove unnecessary /v1/ prefix
- Update all internal API routes from /api/v1/* to /api/*
- Protection API: /api/security/protection/*
- Botmodels calls: /api/vision/*, /api/audio/*, /api/speech/*
- Remove /api/v1/health from anonymous paths (keep /api/health)

External APIs (Reddit, Facebook, etc.) keep their original versioned paths
2026-01-10 09:48:43 -03:00
faeae250bc Add security protection module with sudo-based privilege escalation
- Create installer.rs for 'botserver install protection' command
- Requires root to install packages and create sudoers config
- Sudoers uses exact commands (no wildcards) for security
- Update all tool files (lynis, rkhunter, chkrootkit, suricata, lmd) to use sudo
- Update manager.rs service management to use sudo
- Add 'sudo' and 'visudo' to command_guard.rs whitelist
- Update CLI with install/remove/status protection commands

Security model:
- Installation requires root (sudo botserver install protection)
- Runtime uses sudoers NOPASSWD for specific commands only
- No wildcards in sudoers - exact command specifications
- Tools run on host system, not in containers
2026-01-10 09:41:12 -03:00
115b2770cb fix: Add VAULT_CACERT to vault CLI commands in ensure_vault_unsealed
- vault status and unseal commands now use CA cert for TLS verification
- Fixes x509 certificate signed by unknown authority error
2026-01-09 12:29:45 -03:00
00acf1c76e fix: Add trusted_shell_script_arg for internal scripts
- shell_script_arg blocks $( and backticks for user input safety
- trusted_shell_script_arg allows these for internal installer scripts
- Internal scripts need shell features like command substitution
- Updated bootstrap, installer, facade, and llm modules
2026-01-09 12:13:35 -03:00
db267714ca fix: Allow URL-safe characters in SafeCommand arguments
- Allow &, ?, = in URL arguments (http:// or https://)
- Allow // pattern in URLs (needed for protocol)
- These are safe since Command::new().args() doesn't use shell
- Fixes Vault health check with query parameters
- Add debug logging to safe_curl and vault_health_check
2026-01-09 11:56:11 -03:00
cb59ceb60f fix: Smart mTLS for Vault - use client cert if exists, plain TLS during bootstrap
- Add vault_health_check() function that checks if client certs exist
- If certs exist: use mTLS (secure, post-installation)
- If certs don't exist yet: use plain TLS (during initial bootstrap)
- This allows bootstrap to complete while maintaining mTLS security after setup
- No security hole: mTLS is enforced once certs are generated
2026-01-09 11:23:49 -03:00
63aee6f6bc Revert "fix: Disable mTLS for Vault (temporary until new binary compiled)"
This reverts commit efb458ba66588de090fe6df76cdf04515eacce52.
2026-01-09 11:19:36 -03:00
460d39f399 fix: Disable mTLS for Vault (temporary until new binary compiled)
- Remove tls_client_ca_file from vault config templates
- Remove --cert/--key from health checks
- TLS still enabled for encryption, just no client cert required
- TODO: Re-enable mTLS when binary with cert health checks is compiled
2026-01-09 11:18:30 -03:00
703497b00c fix: Remove dead code and unused fields - proper fixes per PROMPT.md
- large_org_optimizer.rs: Remove unused fields (cached_at, roles, organization_id from structs)
  Add partition_manager() getter to use the field
- middleware.rs: Implement Display for AuthError to use InvalidToken message
- organization_rbac.rs: Remove unused user_groups and user_direct_permissions fields
- passkey.rs: Remove unused first_attempt_at field
- opencv.rs: Remove unused format field and ImageFormat enum
- rekognition.rs: Remove unused liveness_sessions field

No #[allow(dead_code)] - code is properly fixed or deleted
2026-01-09 11:00:14 -03:00
c59d456e25 fix: Add mTLS client certificate to all Vault health checks
- Keep mTLS enabled for security (even in dev)
- Add --cert and --key to all curl commands for Vault health checks
- Fix fetch_vault_credentials to use https and mTLS
- Fix Zitadel commands to use https with VAULT_CACERT
- All Vault communications now use proper mutual TLS
2026-01-09 10:48:13 -03:00