requests2 0.1.2

# Requests2


A Rust library the crate help you Write a function similar to that of Python's request repository (**[`Python`]** **BS4** library).

- Each new requests initializes a cache instance which stores the parsed data in key value pairs

- When you get an instance of connect, you can call the parser method to parse the data in the form of closure

## Example


```rust
let data = Cache::new();
let client = Requests::new(&data);
let mut rq = client.connect("https://www.qq.com/", Headers::Default);

rq.parser(|p| {
    p.find_all("a", |x| {
        x.attr("href").map_or(false, |v| v.starts_with("http"))
    }, "href")
}, "href"); //
data.print()
```

Use `data.print` you can view the value stored as the [`href`] key. It is a value enumeration type that contains most data types.

- output
```rust
Key -- "href" Value -- LIST(["https://qzone.qq.com", "https://qzone.qq.com", "https://mail.qq.com", "https://mail.qq.com/cgi-bin/loginpage", "http://news.qq.com/", "https://v.qq.com/?isoldly=1", "http://gongyi.qq.com/",...]
```

## Headers


Headers defines three types of request headers. The default `Header::default` has only one `user agent`, or it can be without any `Headers::None`.also use JSON string to make a request header containing `useragent` and  `host`, this code:

```rust
let headers = r#"{"user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36", "host": "www.qq.com"}"#;
let store = Cache::new();
let client = Requests::new(&store);
let mut p = client.connect("https://www.qq.com", Headers::JSON(headers));
```

If you need more request header fields, you need to use add corresponding fields in [`headers.rs`](https://docs.rs/requests2/0.1.2/requests2/headers/enum.Headers.html)

## Parser


When you use the `connect` method to connect to a URL, you can use the parser method to write the work of parsing HTML, the current parser has `find` and `find_ all` method

the parser must return [`Value`](https://docs.rs/requests2/0.1.2/requests2/value/enum.Value.html) type

If you use find find_all the code:

```rust
rq.parser(|p| {
    p.find_all("a", |x| {
        x.attr("href").map_or(false, |v| v.starts_with("http"))
    }, "href")
}, "href")
```

The first parameter of parser is obtained by closure automatically saves the value to `value:: list`

In general, you may need to handle the parsing manually and customize the returned `Value`, please use `p.select` function , this example code:

```rust
let data = Cache::new();
    let client = Requests::new(&data);
    let mut parser = client.connect("https://www.qq.com", Headers::Default);
    parser.parser(|p| {
        let mut result = HashMap::new();

        let navs = p.select("li.nav-item", |nodes| {
            let navs = nodes.into_iter().map(|n| {
                let mut item = HashMap::new();
                n.find(Name("a")).next().map_or(HashMap::from([("".to_string(), Value::NULL)]), |a| {
                    let nav_name = a.text();
                    let nav_href = a.attr("href").map_or(String::from(""), |x| x.to_string());
                    item.insert("nav_name".to_string(), Value::STR(nav_name));
                    item.insert("nav_href".to_string(), Value::STR(nav_href));
                    item
                })
            }).collect::<Vec<HashMap<String, Value>>>();

            Value::VECMAP(navs)
        });

        let news = p.select("ul.yw-list", |nodes| {
            let mut news = Vec::new();
            
            for node in nodes {
                for n in node.find(Class("news-top")) {
                    for a in n.find(Name("a")) {
                        let title = a.text();
                        news.push(title);
                    }
                }
            }

            Value::LIST(news)
        });

        result.insert("titles".to_owned(), news);
        result.insert("nav".to_owned(), navs);

        Value::MAP(result)
    }, "index");

    data.print();
```


## Value


```rust
pub enum Value {
    /// 字符串类型
    STR(String),
    /// 字符串列表
    LIST(Vec<String>),
    INT(i32),
    /// 空数据
    NULL,
    /// bool
    BOOL(bool),

    /// map类型的列表
    VECMAP(Vec<HashMap<String, Value>>),

    /// map类型
    MAP(HashMap<String, Value>)
}
```
Add the data type you need in the **Value.rs**


## Concurrency support

use `rayon` library test the concurrency, this have a simple code: 

```rust
let data = Cache::new();
    let client = Requests::new(&data);
    let urls =  ["https://www.baidu.com", "https://www.qq.com", "https://www.163.com"];
    let _ =  urls.par_iter().map(|url| {
        let mut p = client.connect(url, Headers::Default);
        p.parser(|p| {
            p.find_all("a", |f| f.attr("href").map_or(false, |v| v.starts_with("http://")), "href")  
        }, format!("{}_link", url).as_str());

        p.parser(|p| {
            p.find("title", |f| f.text() != "", "text")
        }, format!("{}_title", url).as_str());
    })
    .map(|_| String::from("")).collect::<String>();


    match data.get("https://www.qq.com_title") {
        Value::STR(i) => assert_eq!(i, "腾讯首页"),
        _ => panic!("")
    };

    if let Value::STR(i) = data.get("https://www.163.com_title") {
        assert_eq!(i, "网易");
    }

    if let Value::STR(i) = data.get("https://www.baidu.com_title") {
        assert_eq!(i, "百度一下，你就知道");
    }
```