extrablatt
Customizable article scraping & curation library and CLI. Also runs in Wasm.
Basic Wasm example with some CORS limitations: https://mattsse.github.io/extrablatt/
Inspired by newspaper.
Html Scraping is done via select.rs.
Features
- News url identification
- Text extraction
- Top image extraction
- All image extraction
- Keyword extraction
- Author extraction
- Publishing date
- References
Customizable for specific news sites/layouts via the Extractor
trait.
Documentation
Full Documentation https://docs.rs/extrablatt
Example
Extract all Articles from news outlets.
use Extrablatt;
use StreamExt;
async
Command Line
Install
Usage
USAGE:
extrablatt <SUBCOMMAND>
SUBCOMMANDS:
article Extract a set of articles
category Extract all articles found on the page
help Prints this message or the help of the given subcommand(s)
site Extract all articles from a news source.
Extract a set of specific articles and store the result as json
License
Licensed under either of these:
- Apache License, Version 2.0, (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)