Commit graph

163 commits

Author SHA1 Message Date
841b59affd fix(website-crawler): Add missing next_crawl column to SELECT query
Some checks failed
GBCI / build (push) Failing after 12m2s
Fix the SQL query in check_and_crawl_websites() to include next_crawl in the
SELECT clause. The WebsiteCrawlRecord struct expects this field but it was
missing, causing Diesel to fail with 'Column next_crawl was not present in query'.

This resolves the website crawler service error that was preventing websites
from being properly queried and recrawled.
2026-02-02 19:58:11 -03:00
5fb4c889b7 fix(llm-config): Fix ConfigManager fallback logic for LLM configuration
Some checks failed
GBCI / build (push) Failing after 12m26s
- Fix ConfigManager to treat 'none', 'null', 'n/a', and empty values as placeholders
  and fall back to default bot's configuration instead of using these as literal values

- Fix ConfigManager to detect local file paths (e.g., .gguf, .bin, ../) and fall back
  to default bot's model when using remote API, allowing bots to keep local model
  config for local LLM server while automatically using remote model for API calls

- Fix get_default_bot() to return the bot actually named 'default' instead of
  the first active bot by ID, ensuring consistent fallback behavior

- Add comprehensive debug logging to trace LLM configuration from database to API call

This fixes the issue where bots with incomplete or local LLM configuration would
fail with 401/400 errors when trying to use remote API, instead of automatically
falling back to the default bot's configuration from config.csv.

Closes: #llm-config-fallback
2026-02-02 19:20:37 -03:00
39c4dba838 feat: Add template validation system with .valid file
- Modify bootstrap to read .valid file and validate templates before loading
- Templates not in .valid file are skipped during bootstrap
- Backward compatible: if .valid file missing, all templates are loaded
- Enables controlled template loading during bootstrap
2026-02-01 14:20:35 -03:00
748fceff5d Fix issues: remove unused import, fix ownership error, reduce crawler interval 2026-01-30 12:21:30 -03:00
94fede7cc4 feat: Add search_enabled and menu_launcher_enabled directives to .product file
- Add search_enabled field to ProductConfig to control omnibox visibility (defaults to false)
- Add menu_launcher_enabled field to ProductConfig to control apps menu button visibility (defaults to false)
- Update .product file to set both directives to false by default
- Update get_product_config_json to include new fields in API response
- Parse search_enabled and menu_launcher_enabled from .product file with support for true/false, 1/0, yes/no values

This allows disabling the suite search mechanism and hiding the menu launcher when empty,
providing a cleaner UI for deployments that don't need these features.
2026-01-29 23:55:50 -03:00
26963f2caf Fix bot_id: Use bot_id from URL path instead of client message
- Extract bot_name from WebSocket query parameters
- Look up bot_id from bot_name using database
- Pass bot_id to WebSocket message handler
- Use session's bot_id for LLM configuration instead of client-provided bot_id
- Fixes issue where client sends 'default' bot_id when accessing /edu
2026-01-28 17:18:22 -03:00
b103c07248 Fix migration errors and reorganize migration files
- Fixed 'relation session_kb_associations does not exist' error in core consolidated migration.
- Renamed migration directories from timestamp-based to version-based (6.0.x, 6.1.x, 6.2.x).
- Reorganized migrations into dedicated feature folders (products, dashboards, learn, video).
- Updated migration execution order in core/shared/utils.rs.
- Moves legacy migrations to 6.0.x/6.1.x and workflow to 6.2.0.
2026-01-27 13:45:54 -03:00
3c279f43e5 refactor: embed 3rdparty.toml and remove hardcoded defaults from installer
All checks were successful
GBCI / build (push) Successful in 10m36s
2026-01-26 20:48:57 -03:00
933d42a200 Fix: Replace brace expansion with explicit paths in mkdir command and improve LXC error logging
All checks were successful
GBCI / build (push) Successful in 14m12s
2026-01-26 17:38:34 -03:00
b113267aef Refactor: Genericize default organization to 'system' and update tenant paths 2026-01-26 17:00:21 -03:00
ff4e6c4fe8 fix(lxc): configure lxd-sock proxy for brother mode and update installers
Some checks failed
GBCI / build (push) Failing after 47s
2026-01-26 11:44:18 -03:00
a3babbcaa7 revert: removing production URL default logic (requested by user)
Some checks failed
GBCI / build (push) Has been cancelled
2026-01-25 22:48:47 -03:00
c643a59e29 config: use production URL as default when BOTSERVER_ENV=production
Some checks failed
GBCI / build (push) Failing after 5m28s
2026-01-25 22:42:15 -03:00
9087bb17cd feat: complete General Bots 7.0 (v6.2.0) 2026-01-25 10:29:54 -03:00
0a24cd4b50 Fix build errors and unused imports in core, security and package_manager modules 2026-01-24 22:04:47 -03:00
6fa52e1dd8 feat: implement feature bundling architecture and fix conditional compilation
- Restructured Cargo.toml with Bundle Pattern for easy feature selection
- Added feature bundles: tasks → automation + drive + monitoring
- Applied conditional compilation guards throughout codebase:
  * AppState fields (drive, cache, task_engine, task_scheduler)
  * main.rs initialization (S3, Redis, Tasks)
  * SessionManager Redis usage
  * bootstrap S3/Drive operations
  * compiler task scheduling
  * shared module Task/NewTask exports
- Eliminated all botserver compilation warnings
- Minimal build now compiles successfully
- Accepted core dependencies: automation (Rhai), drive (S3), cache (Redis)
- Created DEPENDENCY_FIX_PLAN.md with complete documentation

Minimal feature set: chat + automation + drive + cache
Verified: cargo check -p botserver --no-default-features --features minimal 
2026-01-23 13:14:20 -03:00
f8a907bd81 Update PROMPT.md and feature gating 2026-01-22 20:24:05 -03:00
66abce913f Feature gating refactor: modular compilation with minimal feature set 2026-01-22 19:45:18 -03:00
3db87c029d Modularize learn schema and clean up channels/project schema stubs 2026-01-22 14:01:50 -03:00
fc9c07d204 Refactor schema.rs into modular files 2026-01-22 13:57:40 -03:00
18b8afd54c O 2026-01-19 21:19:10 -03:00
4f63065da2 Update botserver submodule 2026-01-19 15:43:45 -03:00
5126c648ff Auto-commit: 20260118_195334 2026-01-18 19:53:34 -03:00
033bb504b9 Various updates: dependencies, features, and bug fixes 2026-01-16 11:29:22 -03:00
446dee26b6 Add extended product fields: dimensions, weight, tax codes, brand, inventory, pricing, SEO 2026-01-14 10:02:55 -03:00
95a7591c39 Add Stone Pagamentos fields: SKU, dimensões, peso, NCM, GTIN, tributos, etc 2026-01-14 09:50:39 -03:00
a2783f9b32 Fix 5 errors and 32 warnings: calendar, compliance, billing_alert_broadcast, unused vars 2026-01-13 22:21:25 -03:00
31777432b4 Implement TODO items: session auth, face API, task logs, intent storage
Learn Module:
- All 9 handlers now use AuthenticatedUser extractor

Security:
- validate_session_sync reads roles from SESSION_CACHE

AutoTask:
- get_task_logs reads from manifest with status logs
- store_compiled_intent saves to cache and database

Face API:
- AWS Rekognition, OpenCV, InsightFace implementations
- Detection, verification, analysis methods

Other fixes:
- Calendar/task integration database queries
- Recording database methods
- Analytics insights trends
- Email/folder monitoring mock data
2026-01-13 14:48:49 -03:00
a886478548 Implement database persistence for dashboards, legal, and compliance modules
- Add PostgreSQL persistence for dashboards module (was returning empty vec![])
  - Tables: dashboards, dashboard_widgets, dashboard_data_sources, dashboard_filters,
    dashboard_widget_data_sources, conversational_queries
  - Full CRUD operations with spawn_blocking pattern

- Add PostgreSQL persistence for legal module (was using in-memory HashMap)
  - Tables: legal_documents, legal_document_versions, cookie_consents, consent_history,
    legal_acceptances, data_deletion_requests, data_export_requests
  - GDPR-compliant consent tracking and document management

- Add PostgreSQL persistence for compliance module (was returning empty results)
  - Tables: compliance_checks, compliance_issues, compliance_audit_log, compliance_evidence,
    compliance_risk_assessments, compliance_risks, compliance_training_records,
    compliance_access_reviews
  - Support for GDPR, SOC2, ISO27001, HIPAA, PCI-DSS frameworks

- Add migration files for all new tables
- Update schema.rs with new table definitions and joinables
- Register new routes in main.rs
- Add recursion_limit = 512 for macro expansion
2026-01-13 00:07:22 -03:00
4ed05f3f19 feat(i18n): add missing translation keys to TRANSLATION_KEYS array
- Add people-* keys (title, subtitle, search, tabs, form fields)
- Add crm-* keys (stages, stats, metrics)
- Add billing-* keys (subtitle, new-payment, revenue metrics)
- Add products-* keys (subtitle, items, stats)
2026-01-12 14:13:35 -03:00
1686bfb454 feat(i18n): add missing navigation keys to TRANSLATION_KEYS
- Add nav-docs, nav-sheet, nav-slides, nav-social, nav-all-apps
- Add nav-people, nav-editor, nav-dashboards, nav-security
- Add nav-designer, nav-project, nav-canvas, nav-goals
- Add nav-player, nav-workspace, nav-video, nav-learn
- Add nav-crm, nav-billing, nav-products, nav-tickets
2026-01-10 10:53:56 -03:00
e3b3f04206 Normalize API paths: remove unnecessary /v1/ prefix
- Update all internal API routes from /api/v1/* to /api/*
- Protection API: /api/security/protection/*
- Botmodels calls: /api/vision/*, /api/audio/*, /api/speech/*
- Remove /api/v1/health from anonymous paths (keep /api/health)

External APIs (Reddit, Facebook, etc.) keep their original versioned paths
2026-01-10 09:48:43 -03:00
faeae250bc Add security protection module with sudo-based privilege escalation
- Create installer.rs for 'botserver install protection' command
- Requires root to install packages and create sudoers config
- Sudoers uses exact commands (no wildcards) for security
- Update all tool files (lynis, rkhunter, chkrootkit, suricata, lmd) to use sudo
- Update manager.rs service management to use sudo
- Add 'sudo' and 'visudo' to command_guard.rs whitelist
- Update CLI with install/remove/status protection commands

Security model:
- Installation requires root (sudo botserver install protection)
- Runtime uses sudoers NOPASSWD for specific commands only
- No wildcards in sudoers - exact command specifications
- Tools run on host system, not in containers
2026-01-10 09:41:12 -03:00
115b2770cb fix: Add VAULT_CACERT to vault CLI commands in ensure_vault_unsealed
- vault status and unseal commands now use CA cert for TLS verification
- Fixes x509 certificate signed by unknown authority error
2026-01-09 12:29:45 -03:00
00acf1c76e fix: Add trusted_shell_script_arg for internal scripts
- shell_script_arg blocks $( and backticks for user input safety
- trusted_shell_script_arg allows these for internal installer scripts
- Internal scripts need shell features like command substitution
- Updated bootstrap, installer, facade, and llm modules
2026-01-09 12:13:35 -03:00
db267714ca fix: Allow URL-safe characters in SafeCommand arguments
- Allow &, ?, = in URL arguments (http:// or https://)
- Allow // pattern in URLs (needed for protocol)
- These are safe since Command::new().args() doesn't use shell
- Fixes Vault health check with query parameters
- Add debug logging to safe_curl and vault_health_check
2026-01-09 11:56:11 -03:00
cb59ceb60f fix: Smart mTLS for Vault - use client cert if exists, plain TLS during bootstrap
- Add vault_health_check() function that checks if client certs exist
- If certs exist: use mTLS (secure, post-installation)
- If certs don't exist yet: use plain TLS (during initial bootstrap)
- This allows bootstrap to complete while maintaining mTLS security after setup
- No security hole: mTLS is enforced once certs are generated
2026-01-09 11:23:49 -03:00
63aee6f6bc Revert "fix: Disable mTLS for Vault (temporary until new binary compiled)"
This reverts commit efb458ba66588de090fe6df76cdf04515eacce52.
2026-01-09 11:19:36 -03:00
460d39f399 fix: Disable mTLS for Vault (temporary until new binary compiled)
- Remove tls_client_ca_file from vault config templates
- Remove --cert/--key from health checks
- TLS still enabled for encryption, just no client cert required
- TODO: Re-enable mTLS when binary with cert health checks is compiled
2026-01-09 11:18:30 -03:00
703497b00c fix: Remove dead code and unused fields - proper fixes per PROMPT.md
- large_org_optimizer.rs: Remove unused fields (cached_at, roles, organization_id from structs)
  Add partition_manager() getter to use the field
- middleware.rs: Implement Display for AuthError to use InvalidToken message
- organization_rbac.rs: Remove unused user_groups and user_direct_permissions fields
- passkey.rs: Remove unused first_attempt_at field
- opencv.rs: Remove unused format field and ImageFormat enum
- rekognition.rs: Remove unused liveness_sessions field

No #[allow(dead_code)] - code is properly fixed or deleted
2026-01-09 11:00:14 -03:00
c59d456e25 fix: Add mTLS client certificate to all Vault health checks
- Keep mTLS enabled for security (even in dev)
- Add --cert and --key to all curl commands for Vault health checks
- Fix fetch_vault_credentials to use https and mTLS
- Fix Zitadel commands to use https with VAULT_CACERT
- All Vault communications now use proper mutual TLS
2026-01-09 10:48:13 -03:00
95fbb4f129 Revert "fix: Remove mTLS requirement from Vault config to fix health check failures"
This reverts commit 510fadf24b60c61bb39938f59d88add9b4c08168.
2026-01-09 10:45:41 -03:00
6c904f7dc9 fix: Remove mTLS requirement from Vault config to fix health check failures
- Remove tls_client_ca_file from vault config in installer.rs (Linux and macOS)
- Remove tls_client_ca_file from vault config in bootstrap/mod.rs
- TLS encryption still enabled, just no client certificate required
- Health checks now work with simple -sk curl flags
2026-01-09 10:33:11 -03:00
b674d85583 Fix SafeCommand to allow shell scripts with redirects and command chaining
- Add shell_script_arg() method for bash/sh/cmd -c scripts
- Allow > < redirects in shell scripts (blocked in regular args)
- Allow && || command chaining in shell scripts
- Update safe_sh_command functions to use shell_script_arg
- Update run_commands, start, and LLM server commands
- Block dangerous patterns: backticks, path traversal
- Fix struct field mismatches and type errors
2026-01-08 23:50:38 -03:00
a4cbf145d2 fix: Multiple compilation fixes per PROMPT.md
Video module:
- Fix state.db -> state.conn field name
- Fix analytics.rs imports placement
- Remove AppState dependency from websocket.rs (use global broadcaster)
- Simplify render.rs broadcaster usage

Other modules:
- Add sha1 crate dependency
- Fix AppState import paths (project, legal)
- Fix db_pool -> conn throughout codebase
- Add missing types: RefundResult, ExternalSyncError, TasksIntegrationError, RecordingError, FallbackAttemptTracker
- Add stub implementations for GoogleContactsClient, MicrosoftPeopleClient
- Fix social/mod.rs format string
- Fix designer/canvas.rs SVG path
- Remove doc comments per PROMPT.md
- Add missing handler implementations in calendar_integration.rs
2026-01-08 15:35:03 -03:00
5919aa6bf0 Add video module, RBAC, security features, billing, contacts, dashboards, learn, social, and multiple new modules
Major additions:
- Video editing engine with AI features (transcription, captions, TTS, scene detection)
- RBAC middleware and organization management
- Security enhancements (MFA, passkey, DLP, encryption, audit)
- Billing and subscription management
- Contacts management
- Dashboards module
- Learn/LMS module
- Social features
- Compliance (SOC2, SOP middleware, vulnerability scanner)
- New migrations for RBAC, learn, and video tables
2026-01-08 13:16:17 -03:00
479950945b feat(auth): Add OTP password display on bootstrap and fix Zitadel login flow
- Add generate_secure_password() for OTP generation during admin bootstrap
- Display admin credentials (username/password) in console on first run
- Save credentials to ~/.gb-setup-credentials file
- Fix Zitadel client to support PAT token authentication
- Replace OAuth2 password grant with Zitadel Session API for login
- Fix get_current_user to fetch user data from Zitadel session
- Return session_id as access_token for proper authentication
- Set email as verified on user creation to skip verification
- Add password grant type to OAuth application config
- Update directory_setup to include proper redirect URIs
2026-01-06 22:56:35 -03:00
29b80f597c Fix email_accounts -> user_email_accounts table name typo in list_emails_htmx 2026-01-04 08:48:27 -03:00
c743754c6c Add per-bot database creation
- Added database_name field to bots schema
- Bot creation now creates a dedicated PostgreSQL database (bot_{name})
- Updated add_bot.rs to create database and store database_name
- Added create_bot_database() function with safe name validation
- Added dynamic table check to all db_api handlers
2026-01-02 19:34:59 -03:00
0385047c5c Fix task progress: real-time updates, MIME types, WebSocket event types
- Fix MIME type for app files by preserving directory structure in sanitize_file_path()
- Add with_event_type() to TaskProgressEvent for correct WebSocket event types
- broadcast_manifest_update() now sends 'manifest_update' type correctly
- update_item_status() broadcasts automatically for real-time file progress
2025-12-31 23:45:29 -03:00