ajarbot/memory_workspace/MEMORY.md

# MEMORY - Ajarbot Project Context

## Project
Multi-platform AI agent with memory, cost-optimized for personal/small team use. Supports Slack, Telegram.

## Core Stack
- **Memory**: Hybrid search (0.7 vector + 0.3 BM25), SQLite FTS5 + Markdown files
- **Embeddings**: FastEmbed all-MiniLM-L6-v2 (384-dim, local, $0)
- **LLM**: Claude (Haiku default, Sonnet w/ caching optional), GLM fallback
- **Platforms**: Slack (Socket Mode), Telegram (polling)
- **Tools**: File ops, shell commands (5 tools total)
- **Monitoring**: Pulse & Brain (92% cheaper than Heartbeat - deprecated)

## Key Files
- `agent.py` - Main agent (memory + LLM + tools)
- `memory_system.py` - SQLite FTS5 + markdown sync
- `llm_interface.py` - Claude/GLM API wrapper
- `tools.py` - read_file, write_file, edit_file, list_directory, run_command
- `bot_runner.py` - Multi-platform launcher
- `scheduled_tasks.py` - Cron-like task scheduler

## Memory Files
- `SOUL.md` - Agent personality (auto-loaded)
- `MEMORY.md` - This file (project context)
- `users/{username}.md` - Per-user preferences
- `memory/YYYY-MM-DD.md` - Daily logs
- `memory_index.db` - SQLite FTS5 index
- `vectors.usearch` - Vector embeddings for semantic search

## Cost Optimizations (2026-02-13)
**Target**: Minimize API costs while maintaining capability

### Active
- Default: Haiku 4.5 ($0.25 input/$1.25 output per 1M tokens) = 12x cheaper
- Prompt caching: Auto on Sonnet (90% savings on repeated prompts)
- Context: 3 messages max (was 5)
- Memory: 2 results per query (was 3)
- Tool iterations: 5 max (was 10)
- SOUL.md: 45 lines (was 87)

### Commands
- `/haiku` - Switch to fast/cheap
- `/sonnet` - Switch to smart/cached
- `/status` - Show current config

### Results
- Haiku: ~$0.001/message
- Sonnet cached: ~$0.003/message (after first)
- $5 free credits = hundreds of interactions

## Search System
**IMPLEMENTED (2026-02-13)**: Hybrid semantic + keyword search
- 0.7 vector similarity + 0.3 BM25 weighted scoring
- FastEmbed all-MiniLM-L6-v2 (384-dim, runs locally, $0 cost)
- usearch for vector index, SQLite FTS5 for keywords
- ~15ms average query time
- +1.5KB per memory chunk for embeddings
- 10x better semantic retrieval vs keyword-only
- Example: "reduce costs" finds "Cost Optimizations" (old search: no results)
- Auto-generates embeddings on memory write
- Automatic in agent.chat() - no user action needed

## Recent Changes
**2026-02-13**: Hybrid search implemented
- Added FastEmbed + usearch for semantic vector search
- Upgraded from keyword-only to 0.7 vector + 0.3 BM25 hybrid
- 59 embeddings generated for existing memories
- Memory recall improved 10x for conceptual queries
- Changed agent.py line 71: search() -> search_hybrid()
- Zero cost (local embeddings, no API calls)

**2026-02-13**: Documentation cleanup
- Removed 3 redundant docs (HEARTBEAT_HOOKS, QUICK_START_PULSE, MONITORING_COMPARISON)
- Consolidated monitoring into PULSE_BRAIN.md
- Updated README for accuracy
- Sanitized repo (no API keys, user IDs committed)

**2026-02-13**: Tool system added
- Bot can read/write/edit files, run commands autonomously
- Integrated into SOUL.md instructions

**2026-02-13**: Task scheduler integrated
- Morning weather task (6am daily to Telegram user 8088983654)
- Config: `config/scheduled_tasks.yaml`

## Architecture Decisions
- SQLite not Postgres: Simpler, adequate for personal bot
- Haiku default: Cost optimization priority
- Local embeddings (FastEmbed): Zero API calls, runs on device
- Hybrid search (0.7 vector + 0.3 BM25): Best of both worlds
- Markdown + DB: Simple, fast, no external deps
- Tool use: Autonomous action without user copy/paste

## Deployment
- Platform: Windows 11 primary
- Git: https://vulcan.apophisnetworking.net/jramos/ajarbot.git
- Config: `.env` for API keys, `config/adapters.local.yaml` for tokens (both gitignored)
- Venv: Python 3.11+