Struct StackExchange 

pub struct StackExchange { /* private fields */ }

Scrape questions from StackExchange

Implementations

impl StackExchange

pub fn new() -> Self

Create a new StackExchange instance with a custom client that generates UA (user-agent in order to avoid getting rate limited by DuckDuckGO).

Examples

use falion::search::stackexchange;

let se = stackexchange::StackExchange::new();

pub fn with_client(client: Client) -> Self

Create a new StackExchange instance with a provided client. Note: DuckDuckGO will limit your requests if you don’t provide a user-agent.

use falion::search::stackexchange;

let se = stackexchange::StackExchange::with_client(reqwest::Client::new());

pub async fn get_question_content( &self, question_url: &str, ) -> Result<Vec<String>, SeError>

Get the contents of a StackExchange question inside a vector, the first item being the question itself and the rest the answers.

Arguments

• question_url - The StackExchange absolute url, specifically like this https://stackexchange.com/questions/[0-9]*/the-question, to the question

Examples

use falion::search::ddg;
use falion::search::stackexchange;

const STACKEXCHANGE_INVALID: [&str; 2] = [
    "stackexchange.com/questions/tagged",
    "stackexchange.com/tag",
];

let ddg = ddg::Ddg::new();
let se = stackexchange::StackExchange::new();
let link = &ddg.get_links("Rust threading", Some("stackexchange.com/questions/"), Some(true), Some(&STACKEXCHANGE_INVALID), Some(1)).await.unwrap()[0];

let question_content = se.get_question_content(&link).await.unwrap();

Errors

returns stackexchange::SeError

• NotSeQuestion - The given url does not correspond to a StackExchange question.
• InvalidRequest - Reqwest returned an error when processing the request. This can be due to rate limiting, bad internet etc.
• InvalidResponseBody - The response content you got back is corrupted, usually bad internet.
• InvalidQuestionContent - Usually this means the content returned by the website is corrupted because it did return 200 OK.
• ErrorCode - The website returned an error code

pub async fn get_multiple_questions_content( &self, query: &str, limit: Option<usize>, ) -> Result<Vec<(String, JoinHandle<Result<Vec<String>, SeError>>)>, SeError>

Search for stackexchange results using duckduckgo and a provided query. This function will go through ALL of those results and crate a future for each one which will start getting the content asynchronously for ALL of them. Each of this Futures is associated with the title of the question and returned inside a Vec for preserved order.

PLEASE READ: While setting a limit is optional, doing 100 requests to StackExchange at once will probably get you rate limited.

Arguments

• query - The query to search for.
• limit - Optional, but doing 100 requests to StackExchange at once will probably get you rate limited. A recommended value is something like 10 for enough results and still good results.

Examples

use falion::search::stackexchange;

# async fn run() -> Result<(), stackexchange::SeError> {
let se = stackexchange::StackExchange::new();
let question_content = se
    .get_multiple_questions_content("Rust threading", Some(1))
    .await
    .unwrap();

for q in question_content {
   assert!(!q.1.await.unwrap().unwrap().is_empty())
}
# Ok(())
# }

Errors

returns stackexchange::SeError;

• DdgError - error with getting results from DuckDuckGO. (ddg::DdgError)

First error is for duckduckgo, second is for the future hanle, third is for the actual question content

Trait Implementations

impl Debug for StackExchange

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for StackExchange

fn default() -> Self

Returns the “default value” for a type.