Module fetch

bms_table

Module fetch 

Source
Available on crate feature scraper only.
Expand description

Data fetching and HTML parsing helpers

Provides HTML parsing when the scraper feature is enabled, used to extract the header JSON URL from <meta name="bmstable" content="..."> in a page. Also provides a unified entry to parse a response string into the header JSON or its URL.

§Examples

let html = r#"
<!DOCTYPE html>
<html>
  <head>
    <meta name="bmstable" content="header.json">
  </head>
  <body></body>
</html>
"#;
match get_web_header_json_value::<serde_json::Value>(html).unwrap() {
    HeaderQueryContent::Url(u) => assert_eq!(u, "header.json"),
    _ => unreachable!(),
}

Modules§

reqwestreqwest
Network fetching module based on reqwest

Structs§

FetchedTable
Result of fetching a table with its raw JSON strings.
FetchedTableList
Result of fetching a table list with its raw JSON string.

Enums§

HeaderQueryContent
Return type of get_web_header_json_value.

Traits§

TableFetcher
Unified interface for fetching BMS tables.

Functions§

get_web_header_json_value
Parse a response string into the header JSON or its URL.
header_query_with_fallback
Extract the header query content from a response string with a fallback cleaning step.
parse_json_str_with_fallback
Parse JSON from a raw string with a cleaning fallback.
replace_control_chars
Remove non-printable control characters from JSON text.
try_extract_bmstable_from_html
Extract the JSON file URL pointed to by the bmstable field from HTML page content.