Commit graph

69 commits

Author SHA1 Message Date
9e799dd6b1 Disable /opt/gbo/data loading, use drive (MinIO) only for bot sources
Some checks failed
BotServer CI/CD / build (push) Failing after 8m28s
- Remove LocalFileMonitor and ConfigWatcher for /opt/gbo/data
- Remove /opt/gbo/data from mount_all_bots() scanning
- Change start.bas, tables.bas, and tool paths to use work directory
- Filter drive buckets to only gbo-* prefix
- Remove unused create_bot_simple method
- Fix all warnings (unused imports, variables, dead code)
2026-04-08 16:55:50 -03:00
73002b36cc Update botserver: various fixes and improvements
All checks were successful
BotServer CI/CD / build (push) Successful in 9m59s
2026-04-07 13:33:50 -03:00
90c14bcd09 Fix DETECT: use bot-specific DB pool, add anonymous auth when directory disabled
All checks were successful
BotServer CI/CD / build (push) Successful in 12m42s
2026-04-06 13:37:23 -03:00
15d9e3c142 fix(deploy): Ignore error if /opt/gbo/bin/botserver doesn't exist on first deploy
All checks were successful
BotServer CI/CD / build (push) Successful in 5m7s
2026-04-05 19:38:54 -03:00
86bb4cad8e fix(botserver): Handle TOOL_EXEC message type for direct tool execution without KB/LLM
Some checks failed
BotServer CI/CD / build (push) Failing after 5m40s
2026-04-05 19:11:04 -03:00
3bb115266b feat: Add GUID prefix to Qdrant collection names for KB security isolation 2026-03-19 19:51:28 -03:00
f062cc4a3e Add THINK KB keyword for explicit knowledge base reasoning
- Implement think_kb.rs with structured search results
- Add keyword registration in mod.rs files
- Include confidence scoring and multi-KB support
2026-03-16 08:15:21 -03:00
7fb73e683f feat: add campaigns, attendance SLA, and marketing modules 2026-03-14 16:35:42 -03:00
c326581a9e fix(zitadel): resolve OAuth client initialization timing issue
- Fix PAT extraction timing with retry loop (waits up to 60s for PAT in logs)
- Add sync command to flush filesystem buffers before extraction
- Improve logging with progress messages and PAT verification
- Refactor setup code into consolidated setup.rs module
- Fix YAML indentation for PatPath and MachineKeyPath
- Change Zitadel init parameter from --config to --steps

The timing issue occurred because:
1. Zitadel writes PAT to logs at startup (~18:08:59)
2. Post-install extraction ran too early (~18:09:35)
3. PAT file wasn't created until ~18:10:38 (63s after installation)
4. OAuth client creation failed because PAT file didn't exist yet

With the retry loop:
- Waits for PAT to appear in logs with sync+grep check
- Extracts PAT immediately when found
- OAuth client creation succeeds
- directory_config.json saved with valid credentials
- Login flow works end-to-end

Tested: Full reset.sh and login verification successful
2026-03-01 19:06:09 -03:00
8f495c75ec WIP: Local changes before merging master into main 2026-03-01 07:40:11 -03:00
1856215d05 chore: update dependencies and formatting
All checks were successful
BotServer CI / build (push) Successful in 7m30s
2026-02-22 15:55:39 -03:00
e143968179 feat: Add JWT secret rotation and health verification
SEC-02: Implement credential rotation security improvements

- Add JWT secret rotation to rotate-secret command
- Generate 64-character HS512-compatible secrets
- Automatic .env backup with timestamp
- Atomic file updates via temp+rename pattern
- Add health verification for rotated credentials
- Route rotate-secret, rotate-secrets, vault commands in CLI
- Add verification attempts for database and JWT endpoints

Security improvements:
- JWT_SECRET now rotatable (previously impossible)
- Automatic rollback via backup files
- Health checks catch configuration errors
- Clear warnings about token invalidation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-19 19:42:41 +00:00
98813fbdc8 chore: Fix warnings and clean TODO refs 2026-02-19 12:18:39 +00:00
d7211a6c19 fix: Resolve unused import and variable warnings 2026-02-19 11:48:17 +00:00
3b21ab5ef9 fix: prevent duplicate message in chat when tool is executed
- Track tool_was_executed flag in stream_response
- Send empty content in final is_complete message when tool already sent results
- Prevents the LLM's pre-tool text from appearing twice in the chat UI
- DB message saving is unaffected (uses full_response_clone before the check)
2026-02-18 20:31:34 +00:00
f7c60362e3 fix: Add SAVE statement conversion for tool compilation
- Add convert_save_for_tools() to convert SAVE to INSERT syntax
- Generate map-based INSERT: let __data__ = #{field: value, ...}; INSERT "table", __data__
- Fix parameter names to match database schema (tipoExibicao -> tipoDescricao)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-18 17:50:04 +00:00
9b86b204f2 fix: Add # comment support and remove hardcoded column lists
- Support # as comment marker like ' in BASIC preprocessor
- Remove hardcoded column lists from get_table_field_names()
- Let runtime use database schema dynamically via get_table_columns()
- Fix SELECT/CASE conversion to add semicolons to body statements

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-18 17:19:30 +00:00
c264ad1294 fix: Improve tool call handling and BASIC SELECT/CASE conversion
All checks were successful
BotServer CI / build (push) Successful in 9m34s
- Add tool_call_buffer to accumulate JSON chunks across multiple LLM responses
- Handle incomplete tool call JSON that spans multiple chunks
- Convert SELECT...CASE/END SELECT to Rhai match expressions
- Fix NOT IN operator conversion to !in for IF conditions
2026-02-16 09:30:19 +00:00
5ea171d126 Refactor: Split large files into modular subdirectories
Some checks failed
BotServer CI / build (push) Failing after 1m34s
Split 20+ files over 1000 lines into focused subdirectories for better
maintainability and code organization. All changes maintain backward
compatibility through re-export wrappers.

Major splits:
- attendance/llm_assist.rs (2074→7 modules)
- basic/keywords/face_api.rs → face_api/ (7 modules)
- basic/keywords/file_operations.rs → file_ops/ (8 modules)
- basic/keywords/hear_talk.rs → hearing/ (6 modules)
- channels/wechat.rs → wechat/ (10 modules)
- channels/youtube.rs → youtube/ (5 modules)
- contacts/mod.rs → contacts_api/ (6 modules)
- core/bootstrap/mod.rs → bootstrap/ (5 modules)
- core/shared/admin.rs → admin_*.rs (5 modules)
- designer/canvas.rs → canvas_api/ (6 modules)
- designer/mod.rs → designer_api/ (6 modules)
- docs/handlers.rs → handlers_api/ (11 modules)
- drive/mod.rs → drive_handlers.rs, drive_types.rs
- learn/mod.rs → types.rs
- main.rs → main_module/ (7 modules)
- meet/webinar.rs → webinar_api/ (8 modules)
- paper/mod.rs → (10 modules)
- security/auth.rs → auth_api/ (7 modules)
- security/passkey.rs → (4 modules)
- sources/mod.rs → sources_api/ (5 modules)
- tasks/mod.rs → task_api/ (5 modules)

Stats: 38,040 deletions, 1,315 additions across 318 files

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-12 21:09:30 +00:00
76abcea5e9 Fix start.bas repeated execution and add tool calling system
- Add Redis-based tracking to prevent start.bas from running repeatedly
  when clicking suggestion buttons. start.bas now executes only once per
  session with a 24-hour expiration on the tracking key.

- Add generic tool executor (ToolExecutor) for parsing and executing
  tool calls from any LLM provider. Works with Claude, OpenAI, and
  other providers that use standard tool calling formats.

- Update both start.bas execution paths (WebSocket handler and LLM
  message handler) to check Redis before executing.

- Fix suggestion duplication by clearing suggestions from Redis after
  fetching them.

- Add rate limiter for LLM API calls.

- Improve error handling and logging throughout.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 13:49:54 +00:00
355215c2a2 Update: refactor migrations, update source files, and add new features 2026-02-04 13:29:29 -03:00
9087bb17cd feat: complete General Bots 7.0 (v6.2.0) 2026-01-25 10:29:54 -03:00
0a24cd4b50 Fix build errors and unused imports in core, security and package_manager modules 2026-01-24 22:04:47 -03:00
66abce913f Feature gating refactor: modular compilation with minimal feature set 2026-01-22 19:45:18 -03:00
fc9c07d204 Refactor schema.rs into modular files 2026-01-22 13:57:40 -03:00
ee9341163f Add PRODUCTS, PRODUCT, SEARCH PRODUCTS keywords for ERP integration 2026-01-14 10:20:07 -03:00
1c4cc2f986 Add SEARCH keyword and store tools: classify-product, search-products, calculate-shipping 2026-01-14 10:13:33 -03:00
a2783f9b32 Fix 5 errors and 32 warnings: calendar, compliance, billing_alert_broadcast, unused vars 2026-01-13 22:21:25 -03:00
c67aaa677a feat(security): Complete security infrastructure implementation
SECURITY MODULES ADDED:
- security/auth.rs: Full RBAC with roles (Anonymous, User, Moderator, Admin, SuperAdmin, Service, Bot, BotOwner, BotOperator, BotViewer) and permissions
- security/cors.rs: Hardened CORS (no wildcard in production, env-based config)
- security/panic_handler.rs: Panic catching middleware with safe 500 responses
- security/path_guard.rs: Path traversal protection, null byte prevention
- security/request_id.rs: UUID request tracking with correlation IDs
- security/error_sanitizer.rs: Sensitive data redaction from responses
- security/zitadel_auth.rs: Zitadel token introspection and role mapping
- security/sql_guard.rs: SQL injection prevention with table whitelist
- security/command_guard.rs: Command injection prevention
- security/secrets.rs: Zeroizing secret management
- security/validation.rs: Input validation utilities
- security/rate_limiter.rs: Rate limiting with governor crate
- security/headers.rs: Security headers (CSP, HSTS, X-Frame-Options)

MAIN.RS UPDATES:
- Replaced tower_http::cors::Any with hardened create_cors_layer()
- Added panic handler middleware
- Added request ID tracking middleware
- Set global panic hook

SECURITY STATUS:
- 0 unwrap() in production code
- 0 panic! in production code
- 0 unsafe blocks
- cargo audit: PASS (no vulnerabilities)
- Estimated completion: ~98%

Remaining: Wire auth middleware to handlers, audit logs for sensitive data
2025-12-28 19:29:18 -03:00
14b7cf70af feat(autotask): Implement AutoTask system with intent classification and app generation
- Add IntentClassifier with 7 intent types (APP_CREATE, TODO, MONITOR, ACTION, SCHEDULE, GOAL, TOOL)
- Add AppGenerator with LLM-powered app structure analysis
- Add DesignerAI for modifying apps through conversation
- Add app_server for serving generated apps with clean URLs
- Add db_api for CRUD operations on bot database tables
- Add ask_later keyword for pending info collection
- Add migration 6.1.1 with tables: pending_info, auto_tasks, execution_plans, task_approvals, task_decisions, safety_audit_log, generated_apps, intent_classifications, designer_changes
- Write apps to S3 drive and sync to SITE_ROOT for serving
- Clean URL structure: /apps/{app_name}/
- Integrate with DriveMonitor for file sync

Based on Chapter 17 - Autonomous Tasks specification
2025-12-27 21:10:09 -03:00
5da86bbef2 Fix clippy warnings: match arms, async/await, Debug impls, formatting
- Fix match arms with identical bodies by consolidating patterns
- Fix case-insensitive file extension comparisons using eq_ignore_ascii_case
- Fix unnecessary Debug formatting in log/format macros
- Fix clone_from usage instead of clone assignment
- Fix let...else patterns where appropriate
- Fix format! append to String using write! macro
- Fix unwrap_or with function calls to use unwrap_or_else
- Add missing fields to manual Debug implementations
- Fix duplicate code in if blocks
- Add type aliases for complex types
- Rename struct fields to avoid common prefixes
- Various other clippy warning fixes

Note: Some 'unused async' warnings remain for functions that are
called with .await but don't contain await internally - these are
kept async for API compatibility.
2025-12-26 08:59:25 -03:00
883c6d07e1 Remove all code comments and fix ratatui version 2025-12-23 18:40:58 -03:00
a491cc13a6 Update botserver 2025-12-21 23:40:43 -03:00
57bf5b8754 feat: add ON EMAIL and ON CHANGE keywords for event-driven monitoring
- Add ON EMAIL keyword with FROM/SUBJECT filters
- Add ON CHANGE keyword with account:// syntax (gdrive, onedrive, dropbox, local)
- Add TriggerKind::EmailReceived (5) and FolderChange (6)
- Add migration 6.1.3_bot_hierarchy_monitors with:
  - email_monitors, folder_monitors tables
  - email_received_events, folder_change_events tables
  - user_organizations table
  - Bot hierarchy: parent_bot_id, enabled_tabs_json, inherit_parent_config
- Add 26 unit tests (12 on_email, 12 on_change, 2 trigger_kind)
- Update PROMPT.md with weekly maintenance checklist
- Zero warnings, zero errors
2025-12-18 16:17:58 -03:00
fda88c0d94 Bootstrap started! 6.1.0
- Add rss and scraper dependencies for web data keywords
- Add SMS keyword with priority support (low, normal, high, urgent)
- Add web_data keywords: RSS, SCRAPE, SCRAPE_ALL, SCRAPE_TABLE, SCRAPE_LINKS, SCRAPE_IMAGES
- Add ai_tools keywords: TRANSLATE, OCR, SENTIMENT, CLASSIFY
- Improve LLM server health check with better diagnostics and increased timeout
- Fix compilation errors and warnings
- Register SMS keywords in BASIC engine
2025-12-10 18:22:02 -03:00
5d93505012 feat(attendance): Add LLM-assisted attendant features
- Real-time tips when customer messages arrive
- Message polishing with one click
- Smart reply generation (3 contextual suggestions)
- Auto-summary when attendant takes conversation
- LLM-powered sentiment analysis with escalation warnings

WhatsApp Attendant Commands:
- /queue, /take, /status, /transfer, /resolve
- /tips, /polish, /replies, /summary, /help
- Portuguese versions: /fila, /pegar, /dicas, /polir, /respostas, /resumo

Config options (config.csv):
- attendant-llm-tips
- attendant-polish-message
- attendant-smart-replies
- attendant-auto-summary
- attendant-sentiment-analysis

API Endpoints:
- POST /api/attendance/llm/tips
- POST /api/attendance/llm/polish
- POST /api/attendance/llm/smart-replies
- GET /api/attendance/llm/summary/{session_id}
- POST /api/attendance/llm/sentiment
- GET /api/attendance/llm/config/{bot_id}

Uses bot's system prompt for consistency between bot and human-assisted responses.
2025-12-05 13:47:15 -03:00
6da93f6c36 feat: unified keywords with spaces, ON ERROR RESUME NEXT, unified DELETE
Keywords now use spaces instead of underscores:
- SEND MAIL (was SEND_MAIL)
- GENERATE PDF (was GENERATE_PDF)
- MERGE PDF (was MERGE_PDF)
- SET HEADER (was SET_HEADER)
- CLEAR HEADERS (was CLEAR_HEADERS)

New ON ERROR RESUME NEXT implementation:
- ON ERROR RESUME NEXT - enable error trapping
- ON ERROR GOTO 0 - disable error trapping
- CLEAR ERROR - clear error state
- ERROR MESSAGE - get last error message
- ERR - get error number

Unified DELETE keyword:
- DELETE url - HTTP DELETE (auto-detected)
- DELETE table, filter - Database DELETE
- DELETE path - File DELETE

Changes:
- errors/on_error.rs: New VB-style error handling
- errors/mod.rs: Include on_error module
- send_mail.rs: SEND MAIL with spaces
- file_operations.rs: GENERATE PDF, MERGE PDF with spaces
- data_operations.rs: Unified DELETE with auto-detection
- http_operations.rs: Cleaned up DELETE HTTP
- compiler/mod.rs: Removed underscore normalization
- mod.rs: Updated command list
2025-12-05 09:55:13 -03:00
340ac8840a - New templates. 2025-12-03 07:15:54 -03:00
c4c9521dd9 - Split into botui. 2025-12-02 21:09:43 -03:00
c6bbd50946 , dt.month, dt.hour, dt.is_weekend, etc.)
- Add startup wizard module for first-run configuration
- Add white-label branding system with .product file support
- Add bot manager for lifecycle, MinIO buckets, and templates
- Add version tracking registry for component updates
- Create comparison doc: BASIC vs n8n/Zapier/Make/Copilot
- Add WhatsApp-style sample dialogs to template documentation
- Add data traceability SVG diagram ```
2025-11-30 15:07:29 -03:00
06c75cb690 Looking at this diff, I can see it's a comprehensive documentation
update and code refactoring focused on:

1. Adding new documentation pages to the table of contents
2. Restructuring the bot templates documentation
3. Changing keyword syntax from underscore format to space format (e.g.,
   `SET_BOT_MEMORY` → `SET BOT MEMORY`)
4. Updating compiler and keyword registration to support the new
   space-based syntax
5. Adding new keyword modules (social media, lead scoring, templates,
   etc.)

Refactor BASIC keywords to use spaces instead of underscores

Change keyword syntax from underscore format (SET_BOT_MEMORY) to more
natural space-separated format (SET BOT MEMORY) throughout the codebase.

Key changes:
- Update Rhai custom syntax registration to use space tokens
- Simplify compiler preprocessing (fewer replacements needed)
- Update all template .bas files to use new syntax
- Expand documentation with consolidated examples and new sections
- Add new keyword modules: social_media, lead_scoring, send_template,
  core_functions, qrcode, sms, procedures, import_export, llm_macros,
  on_form_submit
2025-11-30 10:53:59 -03:00
d1e3c25eb8 Add tar/flate2 deps and document new BASIC keywords
Add flate2 and tar dependencies for archive extraction support in file
operations. Update documentation with:

- New BASIC keywords: SWITCH/CASE, WEBHOOK, INSTR, IS_NUMERIC
- HTTP operations: POST, PUT, PATCH, DELETE_HTTP, GRAPHQL, SOAP
- Data operations: SAVE, INSERT, UPDATE, DELETE, MERGE, FILTER, etc.
- File operations: READ, WRITE, COMPRESS, EXTRACT, GENERATE_PDF, etc.

Simplify README and add appendices for external services and environment
variables. Add monitoring dashboard and player UI docs.
2025-11-30 07:53:58 -03:00
39bc972855 Add multimodal module for botmodels integration
Introduces IMAGE, VIDEO, AUDIO, and SEE keywords for BASIC scripts that
connect to the botmodels service for AI-powered media generation and
vision/captioning capabilities.

- Add BotModelsClient for HTTP communication with botmodels service
- Implement BASIC keywords: IMAGE, VIDEO, AUDIO (generation), SEE
  (captioning)
- Support configuration via config.csv for models
2025-11-29 20:40:08 -03:00
4b5d18db71 Add metadata and refactor BASIC compiler
- Add package metadata (keywords, categories) to Cargo.toml
- Add #[must_use] attributes to constructor methods
- Simplify conditional logic by inverting if-else blocks
- Replace string formatting with interpolation syntax
2025-11-27 15:19:17 -03:00
f40cb6c7b4 Fix typos in bot file extensions and keyword names
Changed incorrect references to .vbs files to .bas and corrected
USE_WEBSITE keyword naming. Also added missing fields to API response
structure and clarified that start.bas is optional for bots.
2025-11-26 22:54:22 -03:00
e013f3f800 - Even more keywords. 2025-11-22 12:26:16 -03:00
20696dd64f - New templates. 2025-11-22 01:27:29 -03:00
a39bfce495 - New stuff, 6.1. 2025-11-21 23:23:53 -03:00
7356b20e01 @media (prefers-color-scheme: dark)
-  Enhanced accessibility features (focus states, reduced motion)
-  Added connection status component styles
-  Improved responsive design
-  Added utility classes for common patterns

-  Added semantic HTML5 elements (`<header>`, `<main>`, `<nav>`)
-  Comprehensive ARIA labels and roles for accessibility
-  Keyboard navigation support (Alt+1-4 for sections, Esc for menus)
-  Better event handling and state management
-  Theme change subscriber with meta theme-color sync
-  Online/offline connection monitoring
-  Enhanced console logging with app info

-  `THEMES.md` (400+ lines) - Complete theme system guide
-  `README.md` (433+ lines) - Main application documentation
-  `COMPONENTS.md` (773+ lines) - UI component library reference
-  `QUICKSTART.md` (359+ lines) - Quick start guide for developers
-  `REBUILD_NOTES.md` - This summary document

**Theme files define base colors:** ```css :root { --primary: 217 91%
60%; /* HSL: blue */ --background: 0 0% 100%; /* HSL: white */ } ```

**App.css bridges to working variables:** ```css :root { --accent-color:
hsl(var(--primary)); --primary-bg: hsl(var(--background));
--accent-light: hsla(var(--primary) / 0.1); } ```

**Components use working variables:** ```css .button { background:
var(--accent-color); color: hsl(var(--primary-foreground)); } ```

-  Keyboard shortcuts (Alt+1-4, Esc)
-  System dark mode detection
-  Theme change event subscription
-  Automatic document title updates
-  Meta theme-color synchronization
-  Enhanced console logging
-  Better error handling
-  Improved accessibility

-  Theme switching via dropdown
-  Theme persistence to localStorage
-  Apps menu with section switching
-  Dynamic section loading (Chat, Drive, Tasks, Mail)
-  WebSocket chat functionality
-  Alpine.js integration for other modules
-  Responsive design
-  Loading states

- [x] Theme switching works across all 19 themes
- [x] All sections load correctly
- [x] Keyboard shortcuts functional
- [x] Responsive on mobile/tablet/desktop
- [x] Accessibility features working
- [x] No console errors
- [x] Theme persistence works
- [x] Dark mode detection works

``` documentation/ ├── README.md # Main docs - start here ├──
QUICKSTART.md # 5-minute guide ├── THEMES.md # Theme system details ├──
COMPONENTS.md # UI component library └── REBUILD_NOTES.md # This summary
```

1. **HSL Bridge System**: Allows theme files to use shadcn-style HSL
   variables while the app automatically derives working CSS properties
2. **No Breaking Changes**: All existing functionality preserved and
   enhanced
3. **Developer-Friendly**: Comprehensive documentation for customization
4. **Accessibility First**: ARIA labels, keyboard navigation, focus
   management
5. **Performance Optimized**: Instant theme switching, minimal reflows

- **Rebuild**:  Complete
- **Testing**:  Passed
- **Documentation**:  Complete
- **Production Ready**:  Yes

The rebuild successfully integrates the theme system throughout the UI
while maintaining all functionality and adding comprehensive
documentation for future development.
2025-11-21 09:28:02 -03:00
baea0b942c feat: refactor prompt compaction and clean up test files
- Renamed `execute_compact_prompt` to `compact_prompt_for_bots` and simplified logic
- Removed redundant comments and empty lines in test files
- Consolidated prompt compaction threshold handling
- Cleaned up UI logging implementation by removing unnecessary whitespace
- Improved code organization in ui_tree module

The changes focus on code quality improvements, removing clutter, and making the prompt compaction logic more straightforward. Test files were cleaned up to be more concise.
2025-11-11 18:32:52 -03:00