Initial commit: Ajarbot with optimizations
Features: - Multi-platform bot (Slack, Telegram) - Memory system with SQLite FTS - Tool use capabilities (file ops, commands) - Scheduled tasks system - Dynamic model switching (/sonnet, /haiku) - Prompt caching for cost optimization Optimizations: - Default to Haiku 4.5 (12x cheaper) - Reduced context: 3 messages, 2 memory results - Optimized SOUL.md (48% smaller) - Automatic caching when using Sonnet (90% savings) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
148
docs/README_MEMORY.md
Normal file
148
docs/README_MEMORY.md
Normal file
@@ -0,0 +1,148 @@
|
||||
# Simple Memory System
|
||||
|
||||
A lightweight memory system inspired by OpenClaw, using SQLite + Markdown.
|
||||
|
||||
## Features
|
||||
|
||||
- **SQLite database** for fast indexing and search
|
||||
- **Markdown files** as the source of truth
|
||||
- **Full-text search** (FTS5) for keyword queries
|
||||
- **File watching** for auto-sync
|
||||
- **Chunking** for manageable pieces
|
||||
- **Daily logs** + long-term memory
|
||||
- **SOUL.md** - Agent personality and core identity
|
||||
- **User files** - Per-user preferences and context
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
memory_workspace/
|
||||
├── SOUL.md # Agent personality/identity
|
||||
├── MEMORY.md # Long-term curated memory
|
||||
├── users/ # User-specific memories
|
||||
│ ├── alice.md # User: alice
|
||||
│ ├── bob.md # User: bob
|
||||
│ └── default.md # Default user template
|
||||
├── memory/ # Daily logs
|
||||
│ ├── 2026-02-12.md
|
||||
│ ├── 2026-02-13.md
|
||||
│ └── ...
|
||||
└── memory_index.db # SQLite index
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from memory_system import MemorySystem
|
||||
|
||||
# Initialize
|
||||
memory = MemorySystem()
|
||||
|
||||
# Sync all markdown files
|
||||
memory.sync()
|
||||
|
||||
# === SOUL (Agent Personality) ===
|
||||
memory.update_soul("""
|
||||
## New Trait
|
||||
- I am patient and thorough
|
||||
""", append=True)
|
||||
|
||||
soul_content = memory.get_soul()
|
||||
|
||||
# === User-Specific Memory ===
|
||||
memory.update_user("alice", """
|
||||
## Preferences
|
||||
- Likes Python
|
||||
- Timezone: EST
|
||||
""")
|
||||
|
||||
alice_prefs = memory.get_user("alice")
|
||||
users = memory.list_users() # ['alice', 'bob', 'default']
|
||||
|
||||
# Search user-specific
|
||||
results = memory.search_user("alice", "python")
|
||||
|
||||
# === General Memory ===
|
||||
memory.write_memory("Important note", daily=True)
|
||||
memory.write_memory("Long-term fact", daily=False)
|
||||
|
||||
# Search all memory
|
||||
results = memory.search("keyword")
|
||||
for r in results:
|
||||
print(f"{r['path']}:{r['start_line']} - {r['snippet']}")
|
||||
|
||||
# Read file
|
||||
content = memory.read_file("MEMORY.md", from_line=10, num_lines=5)
|
||||
|
||||
# Status
|
||||
print(memory.status())
|
||||
|
||||
# Auto-sync with file watching
|
||||
memory.start_watching()
|
||||
|
||||
# Cleanup
|
||||
memory.close()
|
||||
```
|
||||
|
||||
## Database Schema
|
||||
|
||||
### files
|
||||
- `path` - relative path to markdown file
|
||||
- `hash` - content hash for change detection
|
||||
- `mtime` - last modified timestamp
|
||||
- `size` - file size
|
||||
|
||||
### chunks
|
||||
- `id` - unique chunk identifier
|
||||
- `path` - source file
|
||||
- `start_line`, `end_line` - line range
|
||||
- `text` - chunk content
|
||||
- `updated_at` - timestamp
|
||||
|
||||
### chunks_fts
|
||||
- Full-text search index (FTS5)
|
||||
- Enables fast keyword search
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Markdown is source of truth** - all data lives in `.md` files
|
||||
2. **SQLite indexes for speed** - database only stores chunks for search
|
||||
3. **Chunking** - splits files into ~500 char paragraphs
|
||||
4. **FTS5** - SQLite's full-text search for keyword matching
|
||||
5. **File watching** - detects changes and triggers re-indexing
|
||||
6. **Hash-based sync** - only re-indexes changed files
|
||||
|
||||
## Differences from OpenClaw
|
||||
|
||||
**Simpler:**
|
||||
- ❌ No vector embeddings (no AI model needed)
|
||||
- ❌ No hybrid search (BM25 + vector)
|
||||
- ❌ No embedding cache
|
||||
- ❌ No session memory
|
||||
- ✅ Just FTS5 keyword search
|
||||
- ✅ Smaller, easier to understand
|
||||
|
||||
**Same concepts:**
|
||||
- ✅ SQLite database
|
||||
- ✅ Markdown files
|
||||
- ✅ File watching
|
||||
- ✅ Chunking
|
||||
- ✅ Daily logs + MEMORY.md
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install watchdog
|
||||
```
|
||||
|
||||
## OpenClaw's Approach
|
||||
|
||||
OpenClaw uses a more sophisticated system:
|
||||
- **Vector embeddings** for semantic search
|
||||
- **Hybrid search** combining BM25 + vector similarity
|
||||
- **Embedding cache** to avoid re-computing
|
||||
- **Multiple providers** (OpenAI, Gemini, local)
|
||||
- **Batch processing** for large indexes
|
||||
- **Session memory** (optional conversation indexing)
|
||||
|
||||
This implementation strips out the complexity for a simple, fast, local-only solution.
|
||||
Reference in New Issue
Block a user