Commit Graph

16 Commits

Author SHA1 Message Date
916f86725d feat: RSO observation system, child safety, Discord adapter, Telegram watchdog, email attachments
Core agent improvements:
- RSO (Relevance Scoring & Observation) system: interaction_logger, memory_scorer, signal_detector
- Memory access logging (memory_access_log table) for relevance scoring; high-signal turn detection
- Rich conversation storage for notable turns; compact_conversation truncates long user messages
- Task-type classifier (query/action/analysis/creative) for observation tagging
- Nested sub-agent visibility: deep delegations now register against the main agent's manager

Child safety (Gabriel profile):
- child_safety.py: filtering, audit logging, prompt constants for restricted sessions
- .kiro/specs/child-safety-profile: requirements, design, tasks specs
- GABRIEL_BOT_PROPOSAL.md: initial proposal doc
- Reduced context window (10 msgs) and tutor-mode identity for restricted users

Telegram adapter:
- Polling watchdog: auto-restarts updater if polling drops unexpectedly
- get_me() with exponential-backoff retry on NetworkError at startup
- Correct stop() ordering: signal watchdog before cancelling tasks

Email / Gmail:
- send_email: supports file attachments (attachments list param)
- get_email: surfaces attachment metadata in response

Scheduled tasks / weather:
- Remove OpenWeatherMap API calls from morning-weather task; use wttr.in exclusively
- New scheduled tasks and scheduler state persistence

Discord:
- adapters/discord/__init__.py scaffold
- discord-plugin: MCP plugin for Claude Code Discord integration (server.ts, skills, config)

Infrastructure:
- n8n workflow exports (garvis_webhook, content_pipeline variants)
- memory_workspace: context, homelab-repo-updates, weekly observation summaries, error logs
- UCS C240 migration plan doc
- requirements.txt: new deps
- .claude/settings.json, fix_hooks.py: hook/permission tuning
2026-04-23 07:54:01 -06:00
1232490c3b Increase main agent timeout to 30 minutes for long delegate tasks
Changed from 600s (10 min) to 1800s (30 min) to prevent main agent
from timing out before delegate tasks can complete.

Timeout hierarchy:
- SubAgent idle timeout: 300s (5 min) - no progress
- SubAgent total timeout: 900s (15 min) - hard cap
- Delegate task timeout: 900s (15 min) - thread timeout
- Main agent timeout: 1800s (30 min) - allows long operations

This ensures main agent waits long enough for:
- Single delegate tasks up to 15 min
- Multiple sequential delegate tasks
- Complex tasks with retries

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-04 18:01:02 -07:00
a8f3ed40a8 Fix critical performance issues: thread pool exhaustion and tool tracking
Root Cause Analysis:
- delegate_task used run_in_executor with default ThreadPoolExecutor (8-12 threads)
- Each delegation blocked one thread for 2-8 minutes (full sub-agent conversation)
- After 6-8 parallel delegations, pool exhausted → all work hung
- Tool tracking used hasattr(block, 'type') but ToolUseBlock has no .type attribute

Changes:

1. mcp_tools.py: Replace thread pool with dedicated threads
   - Each delegate_task creates dedicated daemon thread with isolated event loop
   - Uses asyncio.Future + loop.call_soon_threadsafe for result communication
   - Added semaphore to limit concurrent delegations (4 max)
   - Eliminates pool exhaustion, enables unlimited parallel delegations

2. llm_interface.py: Fix tool tracking
   - Added TextBlock/ToolUseBlock imports from claude_agent_sdk
   - Replaced hasattr(block, 'type') checks with isinstance() checks
   - Fixes tool_calls=0 bug (now correctly tracks tools used)

3. agent.py: Event loop isolation and thread safety
   - Added defensive sub_agent.llm._event_loop = None in spawn_sub_agent
   - Ensures sub-agents use asyncio.run() fallback with isolated loops
   - Generate unique agent IDs with timestamps to prevent caching race conditions

Impact:
- Fixes 6-8 message hang pattern (no more 10-minute timeouts)
- Enables parallel sub-agent execution via delegate_task
- Tool tracking now reports accurate tool usage counts
- All sub-agents remain in Agent SDK mode (as required)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-04 07:43:04 -07:00
cc7e623d74 Add detailed AssistantMessage content inspection
**Problem**: tool_calls=0 in both successful and failed requests, despite
16-31 turns of Agent SDK work. Tool tracking code finds no tool_use blocks.

**Debug addition**: Log content structure for first 10 AssistantMessages:
- Content type (str vs list)
- If list: number of blocks and types of first 3 blocks
- Will show if content contains tool_use blocks or different structure

**Expected**: Should see "list with X blocks: [ToolUseBlock, TextBlock, ...]"
**If broken**: Will see empty lists or unexpected structure

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-01 18:48:24 -07:00
d8b05173f7 Change debug logging to INFO level for visibility
**Problem**: Debug logs weren't appearing because DEBUG level not enabled
in logger configuration.

**Solution**: Changed logger.debug() to logger.info() for:
- Message type logging (every 20th message)
- ResultMessage tracking (captured data summary)
- Tool usage listing

**Impact**: Can now see what message types are being received and why
tool tracking isn't working (important for diagnosing empty results).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-01 18:21:56 -07:00
069a531064 Add debug logging to diagnose empty task results
**Problem**: Task "create a gitea repo" completed 86 messages but returned
generic fallback "Task completed (86 messages, $0.71)" with no actual work
done. Zero tool calls tracked, zero assistant messages captured.

**Debug additions**:
1. Log message type for first 5 messages and every 20th message to see what
   message types we're receiving from Agent SDK
2. Log ResultMessage contents: has_result, assistant_msgs count, tool_calls
   count to understand what was captured
3. Log tools used (if any) to verify tool tracking is working

**Next**: Restart bot and retry failing task. Check logs to see:
- What message types are actually being received (expecting AssistantMessage)
- Whether tool_use blocks are present in content
- Why tool_names list is empty despite work being done

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-01 16:25:46 -07:00
dd5beb11c2 Improve timeout error handling with actionable feedback
**Problem**: User frustrated that 10-minute timeout returned unhelpful
generic message "task may be too complex" when task "create a repo for
the dhcp course" timed out after 80 messages.

**Solution**: Enhanced timeout error to provide:
- Progress info (message count, last tool used)
- Complexity indicator (# of different tools)
- Actionable suggestions (break into sub-tasks, use delegate_task)

**Changes**:
- Track _last_message_count and _last_tool_names as instance vars
  (survive timeout unlike local vars in canceled async function)
- Update tracking variables in message loop
- Build multi-line error message with progress summary and suggestions
- Use chr(10) for newlines to avoid string literal corruption

**Impact**: Users now get helpful guidance instead of generic error when
complex tasks timeout, including suggestion to use new delegate_task tool
for parallel work.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-01 14:25:20 -07:00
6d62fa6314 Refactor: Organize Obsidian MCP into proper package structure
Moved obsidian_mcp.py into mcp_servers/obsidian/ to align with other
MCP server organization (SSH, Cloudflare, Loki). This improves code
discoverability and maintains consistent package structure.

Changes:
- Moved obsidian_mcp.py → mcp_servers/obsidian/obsidian_mcp.py
- Updated lazy import in llm_interface.py (line 396)
- Updated lazy import in tools.py (line 550)
- Deleted empty scripts/ directory

All imports verified with runtime tests. Zero functional changes.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-24 13:31:25 -07:00
58de3e55dc Add SSH MCP server and Gmail attachment download
Features:
- SSH MCP server with two tools:
  * ssh_execute: Run commands on remote hosts via SSH
  * ssh_file_upload: Upload files via SFTP
- Support for both password and SSH key authentication
- Auto-accept SSH host keys (AutoAddPolicy) for homelab use
- Gmail attachment download functionality
- Added download_attachment tool for Gmail API

Technical changes:
- Created mcp_servers/mcp_ssh.py with MCP-compliant text output
- Updated llm_interface.py to load SSH MCP server
- Added paramiko>=3.4.0 to requirements.txt
- Updated .env.example with SSH configuration template
- Enhanced gmail_client.py with download_attachment() method
- Added download_attachment tool handler in tools.py

SSH credentials configured via environment variables:
- PROXMOX_SSH_HOST, PROXMOX_SSH_USER, PROXMOX_SSH_PORT
- PROXMOX_SSH_PASSWORD (or) PROXMOX_SSH_KEY_FILE

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-24 12:32:05 -07:00
a9efdc0a01 feat: Add Loki MCP server scaffold, fix adapter blocking, upgrade model
- Scaffold mcp_servers/loki/ with config and async HTTP client
- Fix Slack/Telegram adapters to use non-blocking connections
- Upgrade default model to claude-sonnet-4-6
- Improve Agent SDK message collection for empty ResultMessage cases
- Add Message-ID to email summaries, increase body truncation limit
- Fix .gitignore inline comments that broke sensitive file exclusions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 21:19:28 -07:00
fe7c146dc6 feat: Add Gitea MCP integration and project cleanup
## New Features
- **Gitea MCP Tools** (zero API cost):
  - gitea_read_file: Read files from homelab repo
  - gitea_list_files: Browse directories
  - gitea_search_code: Search by filename
  - gitea_get_tree: Get directory tree
- **Gitea Client** (gitea_tools/client.py): REST API wrapper with OAuth
- **Proxmox SSH Scripts** (scripts/): Homelab data collection utilities
- **Obsidian MCP Support** (obsidian_mcp.py): Advanced vault operations
- **Voice Integration Plan** (JARVIS_VOICE_INTEGRATION_PLAN.md)

## Improvements
- **Increased timeout**: 5min → 10min for complex tasks (llm_interface.py)
- **Removed Direct API fallback**: Gitea tools are MCP-only (zero cost)
- **Updated .env.example**: Added Obsidian MCP configuration
- **Enhanced .gitignore**: Protect personal memory files (SOUL.md, MEMORY.md)

## Cleanup
- Deleted 24 obsolete files (temp/test/experimental scripts, outdated docs)
- Untracked personal memory files (SOUL.md, MEMORY.md now in .gitignore)
- Removed: AGENT_SDK_IMPLEMENTATION.md, HYBRID_SEARCH_SUMMARY.md,
  IMPLEMENTATION_SUMMARY.md, MIGRATION.md, test_agent_sdk.py, etc.

## Configuration
- Added config/gitea_config.example.yaml (Gitea setup template)
- Added config/obsidian_mcp.example.yaml (Obsidian MCP template)
- Updated scheduled_tasks.yaml with new task examples

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-18 20:31:32 -07:00
50cf7165cb Add sub-agent orchestration, MCP tools, and critical bug fixes
Major Features:
- Sub-agent orchestration system with dynamic specialist spawning
  * spawn_sub_agent(): Create specialists with custom prompts
  * delegate(): Convenience method for task delegation
  * Cached specialists for reuse
  * Separate conversation histories and focused context

- MCP (Model Context Protocol) tool integration
  * Zettelkasten: fleeting_note, daily_note, permanent_note, literature_note
  * Search: search_vault (hybrid search), search_by_tags
  * Web: web_fetch for real-time data
  * Zero-cost file/system operations on Pro subscription

Critical Bug Fixes:
- Fixed max tool iterations (15 → 30, configurable)
- Fixed max_tokens error in Agent SDK query() call
- Fixed MCP tool routing in execute_tool()
  * Routes zettelkasten + web tools to async handlers
  * Prevents "Unknown tool" errors

Documentation:
- SUB_AGENTS.md: Complete guide to sub-agent system
- MCP_MIGRATION.md: Agent SDK migration details
- SOUL.example.md: Sanitized bot identity template
- scheduled_tasks.example.yaml: Sanitized task config template

Security:
- Added obsidian vault to .gitignore
- Protected SOUL.md and MEMORY.md (personal configs)
- Sanitized example configs with placeholders

Dependencies:
- Added beautifulsoup4, httpx, lxml for web scraping
- Updated requirements.txt

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-16 07:43:31 -07:00
911d362ba2 Optimize for Claude Agent SDK: Memory, context, and model selection
## Memory & Context Optimizations

### agent.py
- MAX_CONTEXT_MESSAGES: 10 → 20 (better conversation coherence)
- MEMORY_RESPONSE_PREVIEW_LENGTH: 200 → 500 (richer memory storage)
- MAX_CONVERSATION_HISTORY: 50 → 100 (longer session continuity)
- search_hybrid max_results: 2 → 5 (better memory recall)
- System prompt: Now mentions tool count and flat-rate subscription
- Memory format: Changed "User (username)/Agent" to "username/Garvis"

### llm_interface.py
- Added claude_agent_sdk model (Sonnet) to defaults
- Mode-based model selection:
  * Agent SDK → Sonnet (best quality, flat-rate)
  * Direct API → Haiku (cheapest, pay-per-token)
- Updated logging to show active model

## SOUL.md Rewrite

- Added Garvis identity (name, email, role)
- Listed all 17 tools (was missing 12 tools)
- Added "Critical Behaviors" section
- Emphasized flat-rate subscription benefits
- Clear instructions to always check user profiles

## Benefits

With flat-rate Agent SDK:
-  Use Sonnet for better reasoning (was Haiku)
-  2x context messages (10 → 20)
-  2.5x memory results (2 → 5)
-  2.5x richer memory previews (200 → 500 chars)
-  Bot knows its name and all capabilities
-  Zero marginal cost for thoroughness

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 10:22:23 -07:00
a8665d8c72 Refactor: Clean up obsolete files and organize codebase structure
This commit removes deprecated modules and reorganizes code into logical directories:

Deleted files (superseded by newer systems):
- claude_code_server.py (replaced by agent-sdk direct integration)
- heartbeat.py (superseded by scheduled_tasks.py)
- pulse_brain.py (unused in production)
- config/pulse_brain_config.py (obsolete config)

Created directory structure:
- examples/ (7 example files: example_*.py, demo_*.py)
- tests/ (5 test files: test_*.py)

Updated imports:
- agent.py: Removed heartbeat module and all enable_heartbeat logic
- bot_runner.py: Removed heartbeat parameter from Agent initialization
- llm_interface.py: Updated deprecated claude_code_server message

Preserved essential files:
- hooks.py (for future use)
- adapters/skill_integration.py (for future use)
- All Google integration tools (Gmail, Calendar, Contacts)
- GLM provider code (backward compatibility)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-15 09:57:39 -07:00
8afff96bb5 Add API usage tracking and dynamic task reloading
Features:
- Usage tracking system (usage_tracker.py)
  - Tracks input/output tokens per API call
  - Calculates costs with support for cache pricing
  - Stores data in usage_data.json (gitignored)
  - Integrated into llm_interface.py

- Dynamic task scheduler reloading
  - Auto-detects YAML changes every 60s
  - No restart needed for new tasks
  - reload_tasks() method for manual refresh

- Example cost tracking scheduled task
  - Daily API usage report
  - Budget tracking ($5/month target)
  - Disabled by default in scheduled_tasks.yaml

Improvements:
- Fixed tool_use/tool_result pair splitting bug (CRITICAL)
- Added thread safety to agent.chat()
- Fixed N+1 query problem in hybrid search
- Optimized database batch queries
- Added conversation history pruning (50 messages max)

Updated .gitignore:
- Exclude user profiles (memory_workspace/users/*.md)
- Exclude usage data (usage_data.json)
- Exclude vector index (vectors.usearch)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-13 23:38:44 -07:00
a99799bf3d Initial commit: Ajarbot with optimizations
Features:
- Multi-platform bot (Slack, Telegram)
- Memory system with SQLite FTS
- Tool use capabilities (file ops, commands)
- Scheduled tasks system
- Dynamic model switching (/sonnet, /haiku)
- Prompt caching for cost optimization

Optimizations:
- Default to Haiku 4.5 (12x cheaper)
- Reduced context: 3 messages, 2 memory results
- Optimized SOUL.md (48% smaller)
- Automatic caching when using Sonnet (90% savings)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-13 19:06:28 -07:00