browsing 0.1.2

Lightweight MCP/API for browser automation: navigate, get content (text), screenshot. Parallelism via RwLock.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271

# Migration TODO

## Summary

**Overall Progress**: ~100% Complete (Core Functionality + CLI + MCP + Library + Documentation + All Tests Passing)

### Recent Updates (February 2025)
- ✅ Fixed JSON extractor to handle nested JSON structures using brace counting
- ✅ Added IBM content download example demonstrating web scraping
- ✅ All 317 tests passing (0 failures, 26 ignored)
- ✅ Improved error handling in agent execution tests
- ✅ Updated README and TODO with current status

### Current Status
- **Library**: Production-ready with comprehensive test coverage
- **CLI**: Fully functional with run, launch, and connect commands
- **MCP Server**: Complete with tools, prompts, and resources
- **Documentation**: Complete with README, usage guides, and examples
- **Tests**: 317 tests passing across all modules

### Core Functionality ✅
- ✅ Browser launch and CDP connection
- ✅ DOM extraction and serialization
- ✅ LLM integration (ChatModel trait)
- ✅ Agent execution loop
- ✅ Screenshot support
- ✅ Actor system (Page, Element, Mouse, Keyboard)
- ✅ Signal handling for graceful shutdown
- ✅ Comprehensive test suite (74+ tests)

### Three Usage Modes ✅
- ✅ **CLI Tool** - Command-line interface for autonomous browsing
- ✅ **MCP Server** - Model Context Protocol server for AI assistants
- ✅ **Rust Library** - Embeddable library for custom applications

### Documentation ✅
- ✅ README.md updated with all three usage modes
- ✅ CLI usage guide (docs/CLI_USAGE.md)
- ✅ MCP usage guide (docs/MCP_USAGE.md)
- ✅ Library usage guide (docs/LIBRARY_USAGE.md)
- ✅ Examples for library usage
- ✅ AGENTS.md for agentic coding tools

### Remaining Work ⏳
- ⏳ Advanced DOM features (paint order, enhanced markdown) - Optional optimizations
- ⏳ Cost calculation (token counting implemented, cost calculation can be added)
- [x] Additional actions (extract - LLM-based content extraction) - ✅ Complete
- [x] Signal handling (SIGINT/SIGTERM) - ✅ Complete
- [x] Comprehensive integration tests (24 tests) - ✅ Complete
- [x] CLI interface - ✅ Complete
- [x] MCP server - ✅ Complete
- [x] Library interface - ✅ Complete
- [x] Documentation (CLI, MCP, Library usage guides) - ✅ Complete

## Completed ✅

### Core Library
- [x] Scaffold Rust project structure (single crate)
- [x] Basic module structure (agent, browser, llm, tools, dom, config, error, utils, views)
- [x] Core error types
- [x] Configuration system (from env vars)
- [x] Basic type definitions
- [x] Core view types (Agent, Browser, DOM, Token views)
- [x] Tools/actions registry system
- [x] Browser session and CDP client integration
- [x] DOM serialization and extraction (complete implementation with CDP)
- [x] DOM service integration with agent service
- [x] LLM base trait (ChatModel trait complete)
- [x] Agent service basic structure (execution loop skeleton)
- [x] Agent service implementation (action parsing, execution, history tracking)
- [x] Logging setup (tracing integration)
- [x] Configuration with .env support
- [x] JSON repair for LLM responses (anyrepair integration)
- [x] Actor implementation (page, element, mouse interactions)
- [x] Keyboard input support (key combinations, modifiers)
- [x] Action execution in tools service (search, navigate, click, input, done)
- [x] Element clicking and input using Page/Element actors
- [x] Selector map integration for element lookup by index

### CLI Interface
- [x] CLI binary with clap argument parsing
- [x] Run command for autonomous browsing tasks
- [x] Launch command for browser management
- [x] Connect command for existing browser instances
- [x] Configuration via environment variables and files
- [x] CLI documentation (docs/CLI_USAGE.md)

### MCP Server
- [x] MCP server binary using rmcp
- [x] Tools: navigate, get_content, click, input, screenshot
- [x] Prompts: browse_task template
- [x] Resources: browser://current for page content
- [x] Lazy browser initialization
- [x] MCP documentation (docs/MCP_USAGE.md)

### Library Interface
- [x] Public API exports in lib.rs
- [x] Browser, Agent, Config re-exports
- [x] ChatModel trait for LLM integration
- [x] Example: library_usage.rs - Basic library usage
- [x] Example: custom_actions.rs - Custom action handlers
- [x] Example: ibm_content_download.rs - Web scraping demo
- [x] Example: comprehensive_showcase.rs - Full feature demonstration
- [x] Example: basic_navigation.rs - Simple navigation example
- [x] Example: simple_navigation.rs - Navigation-focused demo
- [x] Library documentation (docs/LIBRARY_USAGE.md)

## In Progress 🚧

- [x] Documentation (API docs, examples, README updates) - ✅ Complete with 317 tests passing
- [ ] Optional enhancements:
  - [ ] Enhanced markdown extraction (paint order filtering)
  - [ ] Cost calculation from token counts
  - [ ] Telemetry module (optional)

## Pending 📋

### Core Types and Models
- [x] Action types (ActionModel, ActionResult) - Basic implementation complete
- [x] DOM element types (EnhancedDOMTreeNode, EnhancedSnapshotNode, EnhancedAXNode)
- [x] Browser profile types (BrowserProfile with basic fields)
- [x] Additional view types from Python (BrowserStateSummary, etc.) - get_browser_state_summary implemented

### Browser Session
- [x] CDP client implementation (WebSocket connection)
- [x] Browser connection via CDP URL
- [x] Navigation handling
- [x] Page actor access
- [x] Browser launch and management (local browser - basic implementation)
- [x] Browser launcher (executable detection, port finding, process management)
- [x] Page state capture (full DOM extraction via get_serialized_dom_tree)
- [x] Screenshot support (Page, Element, and Browser session)
- [x] Tab management (list, switch, close, create tabs with actions)
- [x] Browser state summary (get_browser_state_summary with DOM, tabs, screenshot)

### DOM Service
- [x] Basic HTML parsing and text extraction
- [x] Page state retrieval (placeholder)
- [x] Selector map structure (placeholder)
- [x] Core CDP tree extraction (_get_all_trees - snapshot, DOM tree, AX tree, device pixel ratio)
- [x] Viewport ratio calculation (_get_viewport_ratio)
- [x] CDP client session_id support (send_command_with_session)
- [x] Full DOM tree building (get_dom_tree, enhanced node construction)
- [x] Enhanced snapshot lookup (build_snapshot_lookup)
- [x] Enhanced DOM tree node types (EnhancedDOMTreeNode, EnhancedSnapshotNode, EnhancedAXNode)
- [x] DOM serializer for LLM representation (basic implementation)
- [x] get_serialized_dom_tree method
- [x] Element extraction with indices (selector map)
- [x] JSON extraction with brace counting (handles nested objects/arrays)
- [ ] Markdown extraction (enhanced)
- [ ] Paint order filtering (advanced)
- [ ] Enhanced DOM snapshot optimizations

### LLM Integration
- [x] LLM base trait (ChatModel)
- [x] ChatMessage types
- [x] ChatInvokeCompletion types
- [x] ChatModel trait with streaming support
- [x] Token counting (usage information extraction from response)

### Tools/Actions
- [x] Action registry system
- [x] Default actions (click, input, navigate, search, done, switch, close, scroll, wait, send_keys, evaluate, find_text, dropdown_options, select_dropdown, upload_file, extract)
- [x] Action execution (basic implementation)
- [x] Element interaction (click, input using Page/Element actors)
- [x] Selector map integration (get element by index, lookup backend_node_id)
- [x] Custom action registration (ActionHandler trait and registration system)

### Agent Service
- [x] Agent execution loop (complete)
- [x] Step management
- [x] LLM interaction
- [x] Action parsing from LLM response (with JSON repair)
- [x] Action execution via tools
- [x] History tracking
- [x] Task completion detection

### Actor (Low-level browser interactions)
- [x] Page actor (navigation, evaluation, screenshot, keyboard)
- [x] Element actor (click, fill, text extraction, screenshot, bounding box)
- [x] Mouse actor (click, move, scroll)
- [x] Keyboard input (press keys, key combinations)

### Utilities
- [x] URL detection and parsing
- [x] Logging setup (tracing)
- [x] Configuration with .env support
- [x] Signal handling (SIGINT/SIGTERM for graceful shutdown)
- [ ] Telemetry (optional)

### Testing
- [x] Unit tests for core modules (317 tests passing, 0 failures)
- [x] Integration tests (50+ comprehensive tests passing)
- [x] Actor module tests (page, element, mouse, keyboard operations) - 23 passed
- [x] Browser managers tests (navigation, screenshot, tab management) - 10 passed
- [x] Tools handlers tests (navigation, interaction, content, tabs, advanced) - 49 passed
- [x] Agent service tests (execution logic, history tracking, usage tracking) - 32 passed
- [x] Agent execution tests (workflow, configuration, token tracking) - 11 passed
- [x] Traits tests (BrowserClient, DOMProcessor implementations) - 24 passed
- [x] Utilities tests (URL extraction, domain matching, signal handling) - 49 passed
- [x] Signal handling tests (2 tests passing)
- [x] JSON extraction tests (4 tests with nested JSON support)
- [x] Comprehensive test files:
  - [actor_test.rs](tests/actor_test.rs) - 23 passed, 6 ignored (require browser)
  - [tools_handlers_test.rs](tests/tools_handlers_test.rs) - 49 passed, 8 ignored
  - [agent_service_test.rs](tests/agent_service_test.rs) - 32 passed, 6 ignored
  - [agent_execution_test.rs](tests/agent_execution_test.rs) - 11 passed
  - [traits_test.rs](tests/traits_test.rs) - 24 passed, 3 ignored
  - [utils_test.rs](tests/utils_test.rs) - 49 passed, 3 ignored
  - Plus 10+ additional test files covering all components

### Documentation
- [x] API documentation (cargo doc --open)
- [x] Examples (5 working examples demonstrating all features)
  - ibm_content_download.rs - Web scraping demo
  - comprehensive_showcase.rs - Full agent capabilities
  - library_usage.rs - Basic library usage
  - custom_actions.rs - Extensibility demo
  - basic_navigation.rs - Simple navigation
- [x] README.md - Complete with architecture and usage
- [x] CLI, MCP, and Library usage guides in docs/

## Notes

- Using single crate structure (not multi-crate workspace)
- LLM integration via ChatModel trait
- Using anyrepair for JSON repair
- Using rmcp for MCP support
- Using clap for CLI argument parsing
- Rust edition 2024

## Usage Modes

### 1. CLI Tool
```bash
# Install
cargo install --path . --bin browsing

# Run autonomous task
browsing run "Find the latest news" --url https://news.ycombinator.com --headless

# Launch browser
browsing launch --headless

# Connect to existing browser
browsing connect ws://localhost:9222/devtools/browser/abc123
```

### 2. MCP Server
```bash
# Build
cargo build --release --bin browsing-mcp

# Run (communicates via stdio)
./target/release/browsing-mcp

# Configure in Claude Desktop
# See docs/MCP_USAGE.md for configuration details
```

### 3. Rust Library
```rust
use browsing::{Browser, Config};

let mut browser = Browser::new(Config::from_env().browser_profile);
browser.start().await?;
browser.navigate("https://example.com").await?;
```

See `docs/LIBRARY_USAGE.md` for complete API documentation.