Details
1
# Free Agent - Features Overview
**Your Autonomous Desktop Assistant – Powered by Free Web AI**
---
## 1. Core AI Engine
### Multi-Model Support
- **Gemini** – Google's flagship model (via web UI)
- **HuggingChat** – Open-source models (Llama, Mistral, etc.)
- **DeepSeek** – Powerful reasoning model
- **Perplexity** – Search-enhanced AI
- **Grok** – xAI's conversational model
- **Qwen** – Alibaba's advanced model
- **Comet** – Local web-based model
- **Ollama** – Run local models (Llama, Mistral, Phi, etc.) via API
- **Custom** – Add any web-based AI with a profile
### Intelligent Prompting
- Context-aware prompts with memory, plan, and conversation history
- Automatic tool calling from model responses
- On-the-fly model switching
---
## 2. Advanced Memory System
### SQLite-Based Persistent Memory
- Stores Facts, Preferences, Episodic experiences, and Procedural knowledge
- Each memory has an importance score (1-10) and trigger keywords
### Semantic Search via Embeddings
- Local embedding models (e.g., all-MiniLM-L6-v2) for similarity search
- Natural language memory retrieval with memoryrecall
### Coprocessor for Automatic Storage
- Analyzes every model response and automatically stores critical information
- Stores with appropriate type, importance, and metadata
### Trigger Matching Modes
- **Exact** – Input must exactly equal the trigger phrase
- **Phrase** – Input must contain the trigger phrase (default, recommended)
- **Word** – Each word in the trigger must appear in input (any order)
- **Semantic** – Uses AI embeddings to match by meaning
### Memory Commands (Voice & Text)
- remember – Store with trigger keywords
- memorystore – Store with explicit type and importance
- memoryrecall – Search for memories
- memorystats – View memory statistics
- memory_suspend / memory_resume – Pause/resume operations
---
## 3. Desktop Automation Toolkit
### UI Discovery (Scanners)
| Scanner | Description | Best For |
|---------|-------------|----------|
| desktop_scanner_fast | ~4s scan, ~246 items | Daily automation, speed matters |
| desktop_scanner_ultra | ~9s scan, ~387 items | Deep discovery, extracting lists |
| desktop_scanner_vision | Uses minicpm-v:8b for page description | Popup detection, page overview |
| desktop_scanner_recursive | Full control tree with menu crawling | Mapping complex menus |
| winlist | Lists all visible windows | Window discovery |
| ui_inspect | Lists UI controls for a window | Detailed inspection |
### UI Execution (Executors)
| Executor | Description | Best For |
|----------|-------------|----------|
| desk_executor_seq | Executes a sequence of actions from JSON | All multi-step automations |
| ui_action | Direct UI control interaction | Low-level control |
| capture_window | Screenshot a window | Visual verification |
### Supported Actions
- click – Left-click (uses BM_CLICK for buttons)
- double_click – Double-click
- right_click – Right-click
- type – Types text (uses WM_SETTEXT for reliability)
- click_xy – Click at exact screen coordinates
- scroll – Scroll up/down (positive = up, negative = down)
- scroll_safe – Scroll at specific coordinates without clicking
- hotkey – Press any key combination (e.g., Ctrl+C)
- wait – Pause for N seconds
- drag – Drag from a control by offset
### Process-Locked & Reliable
- Finds windows by process name and title (substring)
- Automatically focuses and maximizes windows before execution
- Uses multiple fallback methods (UIA -> OCR -> raw coordinates)
- Retry logic for navigation steps
---
## 4. Scheduling & Automation
### Schedule Module
- **15 repeat types**: None, Minute, Minutes5, Minutes10, Minutes15, Minutes30, Hourly, Hours2, Hours4, Hours6, Hours8, Hours12, Daily, Weekly, Monthly
- **Action types**: CustomPrompt, RunScript, LaunchProgram
- Persistent storage to C:/ai/schedules/schedule.json
- Pipeline integration – scheduled prompts processed with full memory context
### Eternal Workflows
- Combine schedules with memory and trigger SOPs
- Self-perpetuating automations (e.g., daily duty checks that reschedule)
---
## 5. Voice Control
- **Local Whisper model** – No cloud dependency
- **Wake word detection** – "Hey Agent" to activate
- **Amplitude threshold** – Adjustable sensitivity
- **Real-time transcription** – Spoken commands submitted to the pipeline
- **Visual feedback** – Button states indicate listening/processing
---
## 6. Watchdog Monitoring
- Monitors chat and log output for patterns (regex)
- Triggers actions (Prompt or RunScript) on match
- Cooldown protection – prevents spam (configurable)
- Persistent – triggers saved to C:/ai/masterwatch_triggers.json
### Use Cases
- Alert on error messages
- Auto-recover from UI popups
- React to specific chat messages
---
## 7. Multi-Step Prompt Scripts
- Define steps with ID, Instruction, CompletionCriteria, BranchCondition
- Branching logic – conditional execution based on previous results
- Context preservation – each step receives the previous response
- Execution with prompt_script_run
- Pause/Resume control
---
## 8. Visual Sequence Editor
- Action library with descriptions
- Scan integration – load scan files to see controls
- Drag-and-drop – add, edit, delete, reorder steps
- Auto-save – changes saved immediately
- JSON output – exact format expected by desk_executor_seq
---
## 9. File & System Operations
- file_read – Read text files
- file_write – Write text files (UTF-8)
- dir – List directory contents
- python_run – Execute Python code inline
- python_run_file – Execute a Python script file
- browser_open – Open a URL in the default browser
---
## 10. Real-World Use Cases
### Automated Social Media Management
- Accept Facebook friend requests automatically
- Post daily updates to groups
- Extract duty assignments from chat and generate reports
- Minimize specific chats, scroll safely
### Data Extraction & Analysis
- Scan chat logs for specific patterns
- Extract and deduplicate messages
- Save reports to JSON for further analysis
### System Monitoring & Recovery
- Monitor logs for errors and trigger recovery actions
- Auto-restart applications on crash
- Send alerts when certain conditions are met
### Personal Assistant
- Set reminders with schedules
- Retrieve stored facts and preferences
- Execute voice commands hands-free
### Research & Content Generation
- Upload images to Gemini for description
- Use web AI to summarize articles
- Automate browsing and data collection
---
## 11. What Makes Free Agent Unique?
| Feature | Free Agent | Traditional RPA / AI Tools |
|---------|------------|---------------------------|
| **AI Cost** | Free (uses web-based AI) | Expensive API keys |
| **Subscription** | One-time purchase (lifetime) | Monthly/annual fees |
| **Cloud Dependency** | None – fully local | Often requires cloud |
| **Ease of Use** | Natural language -> automation | Complex scripting |
| **Flexibility** | Works with any Windows app | Locked to specific platforms |
| **Transparency** | Full control; human-readable scripts | Black box logic |
| **Memory** | Semantic, persistent, with coprocessor | Limited or absent |
| **Scheduling** | 15 repeat types + pipeline integration | Basic or none |
| **Voice Control** | Local Whisper + wake word | Often cloud-based |
| **Watchdog** | Pattern-based monitoring and recovery | Rarely included |
| **Extensibility** | Open JSON scripts + custom tools | Vendor-locked |
---
## 12. Included in the Lifetime License
- Full access to all automation tools
- Visual Sequence Editor
- Memory system (semantic, persistent)
- Scheduler (15 repeat types)
- Voice control (Whisper + wake word)
- Watchdog monitoring
- Multi-step prompt scripts
- Full documentation and examples
- Free updates for version 1.x
- 2 years of priority support
- No recurring fees, no cloud costs
---
*Document version: 1.0 – August 9, 2026*
# 50+ Creative Uses for Free Agent – Beyond the Obvious
Free Agent is an incredibly flexible system. Here are creative applications you probably haven't thought of yet, organized by category:
---
## 🧠 Memory System – Creative Uses
### 1. **Personal Knowledge Graph**
Store everything as linked memories. The trigger system creates an automatic knowledge graph:
```
Memory: "Python" → Links to → "Scripts" → Links to → "Automation"
Memory: "Automation" → Links to → "Duty Check" → Links to → "4:30 PM"
```
**Effect:** The AI builds its own knowledge graph over time, discovering relationships.
### 2. **Memory as Version Control**
Store different versions of SOPs with timestamps:
```
Memory: "Duty SOP v1" (2026-07-01)
Memory: "Duty SOP v2" (2026-08-01)
Memory: "Duty SOP v3" (2026-08-06)
```
**Effect:** Rollback to previous versions, track improvements.
### 3. **Automatic Learning from Failures**
```
Memory: "FAILED: Scroll at x=400,y=500 - no result"
Memory: "SUCCESS: Scroll at x=200,y=300 - found 5 messages"
```
**Effect:** The AI learns what works and what doesn't, improving over time.
### 4. **Memory as Bug Tracker**
```
Memory: "BUG: desktop_scanner_vision fails on dark mode"
Memory: "BUG: desk_executor_seq times out after 120s"
```
**Effect:** Known issues are stored and can be recalled automatically.
### 5. **Memory as FAQ System**
```
Memory: "Q: How to accept friend requests? A: Run script..."
Memory: "Q: How to check duty? A: Say 'duty check'"
```
**Effect:** Automatic FAQ generation from conversation history.
### 6. **Memory as Dependency Tracker**
```
Memory: "Script A depends on Script B"
Memory: "Script B requires Ollama running"
```
**Effect:** Automatic dependency resolution for scripts.
### 7. **Memory as To-Do List**
```
Memory: "TODO: Fix popup detection - priority high"
Memory: "TODO: Optimize scroll speed - priority medium"
```
**Effect:** Persistent task tracking across sessions.
### 8. **Memory as Decision Log**
```
Memory: "Decision: Use vision scanner over ultra for popups"
Memory: "Reason: Vision scanner detects overlays better"
```
**Effect:** Decision history for reference and auditing.
### 9. **Memory as Inventory**
```
Memory: "Inventory: Scripts: 12, Configs: 8, Scans: 45"
Memory: "Inventory: Models installed: 7 of 10 required"
```
**Effect:** Automatic inventory tracking.
### 10. **Memory as Brainstorming Log**
```
Memory: "Idea: Voice-activated macros"
Memory: "Idea: Cross-app workflows"
Memory: "Idea: Self-modifying schedules"
```
**Effect:** Ideas are never lost; can be revisited later.
---
## 🎯 Trigger System – Creative Uses
### 11. **Natural Language Commands**
```
Trigger: "scan it" → Run scan on current window
Trigger: "fix it" → Load error recovery SOP
Trigger: "explain" → Load explanation SOP
```
**Effect:** Any phrase can be a command.
### 12. **Context-Aware Triggers**
```
Memory: "When scanning Facebook → check for popups first"
```
**Effect:** Trigger fires only in specific contexts.
### 13. **Chain Reactions**
```
Trigger: "duty check" → Load SOP → Run scan → Extract data → Generate report
```
**Effect:** Multi-step workflows triggered by a single word.
### 14. **Conditional Triggers**
```
Trigger: "if error → load recovery SOP"
Trigger: "if success → load next SOP"
```
**Effect:** Branching logic based on results.
### 15. **Temporal Triggers**
```
Memory: "At 4:30 PM → check duty"
Memory: "After scan → run popup detection"
```
**Effect:** Time-based and sequence-based triggers.
### 16. **Voice-Activated Triggers**
```
Trigger: "Hey Agent" → Start recording
Trigger: "Hey Agent, duty check" → Run duty check workflow
```
**Effect:** Hands-free operation.
### 17. **Emotion-Aware Triggers**
```
Trigger: "frustrated" → Load patience SOP
Trigger: "stuck" → Load help SOP
```
**Effect:** AI responds to user's emotional state.
### 18. **Trigger as Shortcut to System State**
```
Trigger: "save state" → Save all current state (scroll, data, flags)
Trigger: "resume state" → Load saved state
```
**Effect:** Resume interrupted workflows.
### 19. **Trigger as Live Documentation**
```
Trigger: "help" → Load documentation overview
Trigger: "tools" → Load tool reference
```
**Effect:** Help on demand.
### 20. **Trigger as Onboarding**
```
Trigger: "new user" → Load tutorial SOP
Trigger: "setup" → Load installation guide
```
**Effect:** Self-onboarding.
---
## 🔧 Automation System – Creative Uses
### 21. **Automated Browser Automation**
Use `web_model_chat` + `browser_open` to:
- Scrape websites automatically
- Fill forms with AI-generated data
- Test web applications
### 22. **Cross-App Workflows**
```
1. Extract data from Excel
2. Process data with Python
3. Create report in Word
4. Email report automatically
```
**Effect:** End-to-end workflows across applications.
### 23. **Self-Healing Automation**
When automation fails:
1. Detect failure via watchdog
2. Analyze error
3. Attempt recovery
4. If recovery fails, log and alert user
### 24. **Predictive Automation**
Memory learns patterns:
- "Popups appear after 3 minutes of inactivity"
- "Scrolling works best at coordinates x=200,y=300"
**Effect:** Automation proactively adapts.
### 25. **Automated Debugging**
```
1. Automation fails
2. System captures state (screenshot, scan, logs)
3. AI analyzes failure
4. AI suggests fix
```
**Effect:** Self-debugging automation.
### 26. **Visual Automation Builder**
User clicks through UI → system records actions → generates script automatically:
```
Click → Record coordinates → Add to sequence
Type → Record text → Add to sequence
Scroll → Record position → Add to sequence
```
### 27. **Automated Documentation Generator**
```
1. Run automation
2. Capture all steps
3. AI generates documentation
4. Save documentation as memory
```
**Effect:** Self-documenting automation.
### 28. **Parallel Automation**
```
Instance 1: Monitor duty chat
Instance 2: Process research
Instance 3: Generate reports
```
**Effect:** Multi-threaded automation.
### 29. **Automated Regression Testing**
```
1. Run all automation scripts
2. Check for failures
3. Compare to expected results
4. Generate test report
```
### 30. **Automated Rollback**
```
1. Detect automation failure
2. Rollback to previous working state
3. Resume from last known good state
```
---
## 🗣️ Voice System – Creative Uses
### 31. **Voice-Activated Macros**
```
"Hey Agent, check duty" → Run duty script
"Hey Agent, accept friends" → Run friend request script
"Hey Agent, report" → Generate report
```
**Effect:** Full hands-free operation.
### 32. **Voice Journaling**
User speaks → Whisper transcribes → AI summarizes → stored as memory:
```
"Today I worked on..."
→ Summary: "Worked on automation script for duty check"
→ Stored as Fact memory
```
### 33. **Voice-Activated Search**
```
"Hey Agent, find duty script" → Search for "duty" in memory
"Hey Agent, show last 20 exchanges" → Show chat history
```
### 34. **Voice Dictation for Long Prompts**
Speak long commands → transcribed → sent to AI:
```
"Write a script that scans Facebook..."
→ System transcribes → sends to AI
```
### 35. **Voice Notifications**
```
"Task complete" → Voice notification
"Automation failed" → Voice alert
```
**Effect:** Audio feedback without looking at screen.
---
## 🐾 Watchdog System – Creative Uses
### 36. **System Health Monitor**
```
Watchdog: "memory usage > 80%" → Alert user
Watchdog: "Ollama not responding" → Restart Ollama
```
### 37. **Automation Quality Monitor**
```
Watchdog: "automation failed" → Trigger recovery
Watchdog: "automation took > 60s" → Alert user
```
### 38. **User Behavior Analysis**
```
Watchdog: "user said 'frustrated' twice" → Trigger help SOP
Watchdog: "user typed '?' after error" → Trigger explanation
```
### 39. **Security Monitor**
```
Watchdog: "unauthorized access" → Alert user
Watchdog: "failed login attempts" → Alert user
```
### 40. **Anomaly Detection**
```
Watchdog: "tool execution time 5x normal" → Investigate
Watchdog: "unexpected output pattern" → Log and alert
```
### 41. **Notification System**
```
Watchdog: "task complete" → Notify user
Watchdog: "report ready" → Notify user
```
### 42. **Pattern Detection**
```
Watchdog: "popup appears after 3 scrolls" → Learn pattern
Watchdog: "error occurs every Monday" → Learn pattern
```
---
## 🗓️ Scheduler System – Creative Uses
### 43. **Self-Modifying Schedules**
```
Schedule: "Every day at 4:30 PM → update next_trigger"
```
**Effect:** Schedules that adapt themselves.
### 44. **Batch Processing**
```
Schedule: "Every 5 minutes → check for new files"
Schedule: "Every hour → process pending tasks"
```
### 45. **Health Checks**
```
Schedule: "Every 30 minutes → check if model is alive"
Schedule: "Every hour → check if Ollama is running"
```
### 46. **Data Backup**
```
Schedule: "Daily at 2 AM → backup memory database"
Schedule: "Weekly at 3 AM → archive logs"
```
### 47. **Aggregation Tasks**
```
Schedule: "Every hour → aggregate metrics"
Schedule: "Daily at 5 PM → generate summary report"
```
### 48. **On-Demand Schedules**
```
Memory: "When user says 'check duty' → create schedule"
Memory: "When popup appears → schedule popup check"
```
### 49. **Conditional Schedules**
```
Memory: "If automation success → schedule next task"
Memory: "If automation fail → schedule retry"
```
---
## 🧩 Integration & Cross-Cutting
### 50. **Eternal Workflows**
```
Schedule: "Every day at 4:30 PM → check duty"
Schedule updates itself → never ends
```
**Effect:** Workflows that run forever.
### 51. **Self-Healing System**
```
Monitor: Watchdog detects failure
Trigger: "failure" → Load recovery SOP
Action: Run recovery
Result: System restores itself
```
### 52. **Proactive Assistance**
```
Memory: "User works on Project Alpha" → Load project context
Memory: "User usually needs duty check at 4:30 PM" → Suggest
Memory: "User prefers concise answers" → Format accordingly
```
### 53. **On-Demand Documentation**
```
Trigger: "documentation" → Load full docs
Trigger: "tools" → Load tool reference
Trigger: "examples" → Load examples
```
### 54. **Cross-Instance Communication**
```
Instance A: "I'm checking duty"
Instance B: "I'm processing research"
Instance C: "I'm generating report"
```
**Effect:** Multiple agents collaborate.
### 55. **Automatic Dependency Resolution**
```
Memory: "Script requires Python 3.10"
Memory: "Script requires Ollama"
→ System checks dependencies → warns if missing
```
---
## 🎯 **Top 10 Most Impactful to Build**
| # | Feature | Why | Difficulty |
|---|---------|-----|------------|
| 1 | **Voice-Activated Macros** | Hands-free operation | Medium |
| 2 | **Self-Healing Automation** | Reduces manual intervention | Medium |
| 3 | **Eternal Workflows** | Set and forget | Easy |
| 4 | **Automatic Learning from Failures** | Continuous improvement | Medium |
| 5 | **Cross-App Workflows** | End-to-end automation | High |
| 6 | **Proactive Assistance** | Smarter than reactive | Medium |
| 7 | **Parallel Automation** | Faster execution | High |
| 8 | **Automated Documentation Generator** | Self-documenting | Easy |
| 9 | **Memory as FAQ System** | Self-serve help | Easy |
| 10 | **Predictive Automation** | Anticipates needs | Medium |
---
## 💡 Summary
You already have all the pieces. The system can already do most of these things – they just need:
- **Memories** storing the knowledge
- **Triggers** making them discoverable
- **SOPs** guiding the behavior
- **Schedules** making them automatic
**The creative leap is just connecting what you already have in new ways.**
---