Rustboxd

A fast, async Rust library for scraping Letterboxd data.

Features

Async/Await Support: Built on tokio for high-performance concurrent operations
Type Safety: Full Rust type safety with serde serialization
Comprehensive API: Access users, movies, films, lists, and search functionality
Error Handling: Detailed error types with context
Rate Limiting: Built-in respect for server resources
No Dependencies on Python: Pure Rust implementation

Quick Start

Add this to your Cargo.toml:

[dependencies]
rustboxd = "0.1.0"
tokio = { version = "1.0", features = ["full"] }

Basic Usage

use rustboxd::{User, Movie, Search};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Get user information
    let user = User::new("username").await?;
    println!("Hello, {}!", user.display_name);
    
    // Get movie details
    let movie = Movie::new("the-matrix").await?;
    println!("Movie: {} ({})", movie.title, movie.year.unwrap_or(0));
    
    // Search for content
    let search = Search::new("kubrick", Some("films")).await?;
    println!("Found {} Kubrick films", search.results.films.len());
    
    Ok(())
}
}

User Operations

Basic User Information

use rustboxd::User;

let user = User::new("username").await?;

// Basic profile information
println!("Display Name: {}", user.display_name);
println!("Bio: {}", user.bio.unwrap_or_default());
println!("Location: {}", user.location.unwrap_or_default());

// Statistics
if let Some(stats) = &user.stats {
    println!("Films: {}", stats.films);
    println!("Reviews: {}", stats.reviews);
    println!("Lists: {}", stats.lists);
}

User Film Data

// Get all films watched by user
let films = user.get_films().await?;
println!("Total films: {}", films.len());

// Get films by specific rating
let five_star_films = user.get_films_by_rating(5.0).await?;
println!("5-star films: {}", five_star_films.len());

// Get watchlist
let watchlist = user.get_watchlist().await?;
println!("Watchlist: {}", watchlist.len());

// Get liked films
let liked_films = user.get_films_liked().await?;
println!("Liked films: {}", liked_films.len());

User Activity and Social

// Get user's diary
let diary = user.get_diary(Some(2024), None, None, None).await?;
println!("2024 diary entries: {}", diary.len());

// Get reviews
let reviews = user.get_reviews().await?;
println!("Reviews written: {}", reviews.len());

// Get social connections
let followers = user.get_followers().await?;
let following = user.get_following().await?;
println!("Followers: {}, Following: {}", followers.len(), following.len());

// Get lists created by user
let lists = user.get_lists().await?;
println!("Lists created: {}", lists.len());

Movie Operations

Movie Details

use rustboxd::Movie;

let movie = Movie::new("the-matrix").await?;

// Basic information
println!("Title: {}", movie.title);
println!("Year: {}", movie.year.unwrap_or(0));
println!("Runtime: {} minutes", movie.runtime.unwrap_or(0));
println!("Rating: {:.1}/5", movie.rating.unwrap_or(0.0));

// Genres and themes
println!("Genres: {}", movie.genres.join(", "));
println!("Themes: {}", movie.themes.join(", "));

// Cast and crew
println!("Cast: {} people", movie.cast.len());
println!("Crew: {} people", movie.crew.len());

Movie Community Data

// Get reviews for the movie
let reviews = movie.get_reviews().await?;
println!("Reviews: {}", reviews.len());

// Get similar films
let similar = movie.get_similar().await?;
println!("Similar films: {}", similar.len());

// Get users who watched this movie
let watchers = movie.get_watchers().await?;
println!("Watchers: {}", watchers.len());

Search Operations

Film Search

use rustboxd::Search;

// Search for films
let search = Search::new("christopher nolan", Some("films")).await?;
println!("Found {} films", search.results.films.len());

for film in search.results.films.iter().take(5) {
    println!("- {} ({})", film.title, film.year.unwrap_or(0));
}

Multi-type Search

// Search all content types
let search = Search::new("kubrick", None).await?;

println!("Search results for 'kubrick':");
println!("- Films: {}", search.results.films.len());
println!("- Reviews: {}", search.results.reviews.len());
println!("- Lists: {}", search.results.lists.len());
println!("- Members: {}", search.results.members.len());

Lists

List Information

use rustboxd::List;

let list = List::new("author", "list-slug").await?;

println!("List: {}", list.title);
println!("Author: {}", list.author);
println!("Films: {}", list.film_count);
println!("Likes: {}", list.likes);

// Access films in the list
for film in list.films.iter().take(5) {
    println!("{}. {}", film.position.unwrap_or(0), film.title);
}

List Comments

// Get comments on the list
let comments = list.get_comments().await?;
println!("Comments: {}", comments.len());

for comment in comments.iter().take(3) {
    println!("{}: {}", comment.author, comment.content);
}

Films Collection

Collection Analysis

use rustboxd::Films;

let films = Films::new("https://letterboxd.com/films/popular/").await?;

println!("Total films: {}", films.count);

// Filter by rating
let high_rated = films.filter_by_rating(4.0);
println!("Films rated 4.0+: {}", high_rated.len());

// Filter by year
let recent = films.filter_by_year(2020);
println!("Films from 2020+: {}", recent.len());

// Filter by genre
let horror = films.filter_by_genre("Horror");
println!("Horror films: {}", horror.len());

Error Handling

Rustboxd provides detailed error types:

use rustboxd::{User, Error};

match User::new("username").await {
    Ok(user) => println!("Success: {}", user.display_name),
    Err(Error::InvalidUsername(username)) => {
        println!("Invalid username format: {}", username);
    }
    Err(Error::UserNotFound(username)) => {
        println!("User not found: {}", username);
    }
    Err(Error::PageLoad { url, message }) => {
        println!("Failed to load page {}: {}", url, message);
    }
    Err(Error::ParseError { field, message }) => {
        println!("Parse error in {}: {}", field, message);
    }
    Err(e) => println!("Other error: {}", e),
}

Async Best Practices

Concurrent Operations

use tokio;

// Run multiple operations concurrently
let (user, movie, search) = tokio::try_join!(
    User::new("username"),
    Movie::new("the-matrix"),
    Search::new("kubrick", Some("films"))
)?;

Rate Limiting

use tokio::time::{sleep, Duration};

// Be respectful to the server
for username in usernames {
    let user = User::new(username).await?;
    println!("Loaded: {}", user.display_name);
    
    // Wait between requests
    sleep(Duration::from_millis(500)).await;
}

Advanced Examples

For more comprehensive examples, see the examples documentation and performance guide.

Documentation

API Documentation - Detailed API reference
Examples - Comprehensive usage examples
Performance Guide - Optimization and best practices

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Disclaimer

This library is for educational and personal use. Please respect Letterboxd's robots.txt and terms of service. Consider implementing appropriate rate limiting and caching to minimize server load.

Ok(())

}


### Search and Discovery

```rust
use rustboxd::Search;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Search films
    let film_search = Search::new("christopher nolan", Some("films")).await?;
    
    // Search reviews
    let review_search = Search::new("masterpiece", Some("reviews")).await?;
    
    // Search users
    let user_search = Search::new("film critic", Some("members")).await?;
    
    // Search lists
    let list_search = Search::new("best of 2024", Some("lists")).await?;
    
    Ok(())
}

Project Structure

rustboxd/
├── src/
│   ├── core/           # Core functionality (HTTP client, errors, constants)
│   ├── models/         # Data models (User, Movie, Search, Films, List)
│   ├── pages/          # Page-specific scrapers and parsers
│   ├── utils/          # Utility functions for parsing and validation
│   └── lib.rs         # Library entry point
├── examples/           # Usage examples
├── tests/             # Integration and unit tests
└── docs/              # Documentation

Features by Module

Core

HTTP client with proper headers and error handling
Comprehensive error types with detailed messages
Constants for Letterboxd URLs, selectors, and validation

Models

User: Complete user profile data and statistics
Movie: Full film metadata including cast, crew, and reviews
Search: Multi-type search with filtering capabilities
Films: Collection handling with pagination support
List: User list data with film entries and metadata

Utils

HTML parsing utilities with error handling
Data transformation and normalization functions
Input validation and sanitization
URL building and manipulation helpers

Error Handling

Rustboxd provides comprehensive error handling:

use rustboxd::{User, Error};

match User::new("invalid-user").await {
    Ok(user) => println!("User found: {}", user.display_name),
    Err(Error::PageLoad { url, message }) => {
        println!("Failed to load {}: {}", url, message);
    }
    Err(Error::InvalidUsername(username)) => {
        println!("Invalid username format: {}", username);
    }
    Err(e) => println!("Other error: {}", e),
}

Performance

Async/await for non-blocking I/O operations
HTTP connection reuse for multiple requests
Efficient HTML parsing with the scraper crate
Memory-efficient data structures with serde

Validation

Built-in validation for:

Usernames (alphanumeric and underscore only)
Film slugs (lowercase letters, numbers, hyphens)
Ratings (0.5-5.0 in 0.5 increments)
Search filters (predefined valid options)
URLs and date formats

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Development

Clone the repository:

git clone https://github.com/Pranav-Karra-3301/rustboxd.git
cd rustboxd

Install Rust and dependencies:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo build

Run tests:
```
cargo test
```

Run examples:

cargo run --example basic_usage
cargo run --example advanced_usage

License

This project is licensed under the MIT License - see the LICENSE file for details.

Disclaimer

This is an unofficial library for educational and research purposes. Please respect Letterboxd's terms of service and implement appropriate rate limiting in your applications.

Related Projects

letterboxdpy - Python library that inspired this project
letterboxd-api - Official Letterboxd API (requires authentication) letterboxd webscraper in rust

rustboxd 0.2.0

Rustboxd

Features

Quick Start

Basic Usage

User Operations

Basic User Information

User Film Data

User Activity and Social

Movie Operations

Movie Details

Movie Community Data

Search Operations

Film Search

Multi-type Search

Lists

List Information

List Comments

Films Collection

Collection Analysis

Error Handling

Async Best Practices

Concurrent Operations

Rate Limiting

Advanced Examples

Documentation

Contributing

License

Disclaimer

Project Structure

Features by Module

Core

Models

Pages

Utils

Error Handling

Performance

Validation

Contributing

Development

License

Disclaimer

Related Projects