# bioRxiv and medRxiv Preprint API Clients
This module provides async clients for fetching preprints from **bioRxiv.org** (life sciences) and **medRxiv.org** (medical sciences), converting them to `SemanticVector` format for RuVector discovery.
## Features
- **Free API access** - No authentication required
- **Rate limiting** - Automatic 1 req/sec rate limiting (conservative)
- **Pagination support** - Handles large result sets automatically
- **Retry logic** - Built-in retry for transient failures
- **Domain separation** - bioRxiv → `Domain::Research`, medRxiv → `Domain::Medical`
- **Rich metadata** - DOI, authors, categories, publication status
## API Details
- **Base URL**: `https://api.biorxiv.org/details/[server]/[interval]/[cursor]`
- **Servers**: `biorxiv` or `medrxiv`
- **Interval**: Date range like `2024-01-01/2024-12-31`
- **Response**: JSON with collection array
## BiorxivClient (Life Sciences)
### Methods
```rust
use ruvector_data_framework::BiorxivClient;
let client = BiorxivClient::new();
// Get recent preprints (last N days)
let recent = client.search_recent(7, 100).await?;
// Search by date range
let start = NaiveDate::from_ymd_opt(2024, 1, 1).unwrap();
let end = NaiveDate::from_ymd_opt(2024, 12, 31).unwrap();
let papers = client.search_by_date_range(start, end, Some(200)).await?;
// Search by category
let neuro = client.search_by_category("neuroscience", 100).await?;
```
### Categories
- `neuroscience` - Neural systems and computation
- `genomics` - Genome sequencing and analysis
- `bioinformatics` - Computational biology
- `cancer-biology` - Oncology research
- `immunology` - Immune system studies
- `microbiology` - Microorganisms
- `molecular-biology` - Molecular mechanisms
- `cell-biology` - Cellular processes
- `biochemistry` - Chemical processes
- `evolutionary-biology` - Evolution and phylogenetics
- `ecology` - Ecosystems and populations
- `genetics` - Heredity and variation
- `developmental-biology` - Organism development
- `synthetic-biology` - Engineered biological systems
- `systems-biology` - System-level understanding
## MedrxivClient (Medical Sciences)
### Methods
```rust
use ruvector_data_framework::MedrxivClient;
let client = MedrxivClient::new();
// Get recent medical preprints
let recent = client.search_recent(7, 100).await?;
// Search by date range
let papers = client.search_by_date_range(start, end, Some(200)).await?;
// Search COVID-19 related papers
let covid = client.search_covid(100).await?;
// Search clinical research
let clinical = client.search_clinical(50).await?;
```
### Specialized Searches
- **COVID-19**: Filters for "covid", "sars-cov-2", "coronavirus", "pandemic" keywords
- **Clinical Research**: Filters for "clinical", "trial", "patient", "treatment", "therapy", "diagnosis"
## SemanticVector Output
Both clients convert preprints to `SemanticVector` with:
```rust
SemanticVector {
id: "doi:10.1101/2024.01.01.123456",
embedding: Vec<f32>, // Generated from title + abstract
domain: Domain::Research, // or Domain::Medical for medRxiv
timestamp: DateTime<Utc>, // Preprint publication date
metadata: {
"doi": "10.1101/2024.01.01.123456",
"title": "Paper title",
"abstract": "Full abstract text",
"authors": "John Doe; Jane Smith",
"category": "Neuroscience",
"server": "biorxiv",
"published_status": "preprint" or journal name,
"corresponding_author": "John Doe",
"institution": "MIT",
"version": "1",
"type": "new results",
"source": "biorxiv" or "medrxiv"
}
}
```
## Example Usage
See `examples/biorxiv_discovery.rs` for a complete example:
```bash
cargo run --example biorxiv_discovery
```
## Rate Limiting
- **Default**: 1 request per second (conservative)
- **Configurable**: Modify `BIORXIV_RATE_LIMIT_MS` constant if needed
- **Retry logic**: 3 retries with exponential backoff
## Pagination
Both clients handle pagination automatically:
- Fetches up to the specified `limit`
- Uses cursor-based pagination
- Safety limit of 10,000 records per query
- Handles empty result sets gracefully
## Integration with RuVector
Use the generated `SemanticVector`s with:
1. **Vector similarity search**: Find related preprints using HNSW index
2. **Graph coherence analysis**: Detect emerging research trends
3. **Cross-domain discovery**: Find connections between life sciences and medical research
4. **Time-series analysis**: Track research evolution over time
## Error Handling
The clients include comprehensive error handling:
- **Network errors**: Automatic retry with exponential backoff
- **Rate limiting**: Built-in delays between requests
- **Parsing errors**: Graceful handling of malformed responses
- **Empty results**: Returns empty vector instead of error
## Testing
Run the unit tests:
```bash
# Run all tests (excluding integration tests)
cargo test --lib biorxiv_client::tests
# Run integration tests (requires network access)
cargo test --lib biorxiv_client::tests -- --ignored
```
Unit tests cover:
- Client creation
- Embedding dimension configuration
- Record to vector conversion
- Date parsing
- Domain assignment
- Metadata extraction
Integration tests (ignored by default):
- Search recent papers
- Search by category
- COVID-19 search
- Clinical research search
## Dependencies
- `reqwest` - Async HTTP client
- `serde` / `serde_json` - JSON parsing
- `chrono` - Date/time handling
- `tokio` - Async runtime
- `urlencoding` - URL encoding for queries
- `SimpleEmbedder` - Text to vector embedding
## Custom Embedding Dimension
```rust
// Default 384 dimensions
let client = BiorxivClient::new();
// Custom dimension
let client = BiorxivClient::with_embedding_dim(512);
```
## Best Practices
1. **Respect rate limits**: The clients enforce conservative rate limiting
2. **Use date ranges**: For large datasets, query by date ranges
3. **Filter locally**: Use category filters for more specific searches
4. **Handle errors**: Network requests can fail, use proper error handling
5. **Cache results**: Consider caching SemanticVectors for repeated use
6. **Batch processing**: Process results in batches for better performance
## Publication Status
The `published_status` metadata field indicates:
- `"preprint"` - Not yet published in journal
- Journal name - Accepted and published (e.g., "Nature Medicine")
This helps distinguish between preliminary and peer-reviewed research.
## Cross-Domain Analysis
Combine bioRxiv and medRxiv for comprehensive analysis:
```rust
let biorxiv = BiorxivClient::new();
let medrxiv = MedrxivClient::new();
let bio_papers = biorxiv.search_recent(7, 100).await?;
let med_papers = medrxiv.search_recent(7, 100).await?;
let mut all_papers = bio_papers;
all_papers.extend(med_papers);
// Use RuVector's discovery engine to find cross-domain patterns
```
## Resources
- **bioRxiv**: https://www.biorxiv.org/
- **medRxiv**: https://www.medrxiv.org/
- **API Docs**: https://api.biorxiv.org/
- **RuVector**: https://github.com/ruvnet/ruvector