aniscraper 0.1.0

Rust library designed for efficient web scraping and data extraction. It simplifies the process of fetching, parsing, and extracting data from websites.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146

<p align="center"><img src="https://github.com/user-attachments/assets/87ace4a6-a3ed-4c7c-903c-2b055628cc54" width="175"/></p>

<h1 align="center">aniscraper</h1>

<p align="center">
  A Rust library for scraping data with proxy support and error handling.
</p>

## Index

- [Installation](#installation)
- [Configuration](#configuration)
  - [Using `.env` file](#using-env-file)
  - [Manual Configuration](#manual-configuration)
- [Usage](#usage)
- [License](#license)
- [Contributing](#contributing)

##  <span id="installation">Installation</span>

To get started with `aniscraper`, first add it to your `Cargo.toml`:

```toml
[dependencies]
aniscraper = "0.1"  # Replace with the latest version
```

Or

```bash
cargo add aniscraper
```

Then, run `cargo build` to fetch and compile the library.

##  <span id="configuration">Configuration</span>

Before using `aniscraper`, you need to configure it with the necessary parameters. You can do this in two ways:

##  <span id="using-env-file">Using `.env` File</span>

Create a `.env` file in the root of your project directory and add the following environment variables:

```env
# Required
USER_AGENT_HEADER=xxx
ACCEPT_ENCODING_HEADER=xxx
ACCEPT_HEADER=xxx

# Optional
MAX_RETRIES_ATTEMPTS=100
SOCK5_URL=https://xxx/socks5.txt
SOCK4_URL=https://xxx/socks4.txt
HTTP_URL=https://xxx/http.txt

UNKNOWN_ERROR_WEBHOOK=xxx
REQWEST_ERROR_WEBHOOK=xxx
NO_PROXIES_AVAILABLE_ERROR_WEBHOOK=xxx
FAILED_TO_FETCH_AFTER_RETRIES_ERROR_WEBHOOK=xxx
UTILS_ERROR_WEBHOOK=xxx

HIANIME_DOMAINS=xxx
```

>[!NOTE]
>Required variables are `USER_AGENT_HEADER`, `ACCEPT_ENCODING_HEADER`, and `ACCEPT_HEADER`. All other variables are optional and can be left empty if not needed.

##  <span id="manual-configuration">Manual Configuration</span>

Alternatively, you can configure `aniscraper` manually using a struct. First, ensure you have `aniscraper` included in your project:

```rust
use aniscraper::env::SecretConfig;
```

Then, define and initialize the `SecretConfig` struct:

```rust
#[derive(Debug, Clone)]
pub struct SecretConfig {
    pub max_retries_attempts: String,
    pub reqwest_error_webhook: String,
    pub no_proxies_available_error_webhook: String,
    pub failed_to_fetch_after_retries_error_webhook: String,
    pub utils_error_webhook: String,
    pub unknown_error_webhook: String,
    pub http_url: String,
    pub sock4_url: String,
    pub sock5_url: String,
    pub hianime_domains: String,
    pub user_agent_header: String,
    pub accept_encoding_header: String,
    pub accept_header: String,
}
```

Then pass `SecretConfig` as an argument to the `new` method:

```rust
#[tokio::main]
async fn main() {
    let secret_config = SecretConfig {
        max_retries_attempts: "100".to_string(),
        reqwest_error_webhook: "".to_string(),
        no_proxies_available_error_webhook: "".to_string(),
        failed_to_fetch_after_retries_error_webhook: "".to_string(),
        utils_error_webhook: "".to_string(),
        unknown_error_webhook: "".to_string(),
        http_url: "".to_string(),
        sock4_url: "".to_string(),
        sock5_url: "".to_string(),
        hianime_domains: "".to_string(),
        user_agent_header: "your-user-agent".to_string(),
        accept_encoding_header: "your-accept-encoding".to_string(),
        accept_header: "your-accept-header".to_string(),
    };
    let hianime = HiAnimeRust::new(secret_config).await;
}
```

##  <span id="usage">Usage</span>

Once configured, you can use `aniscraper` in your project to start scraping with proxy support and error handling. Here's an example of how to use the library:

```rust
#[tokio::main]
async fn main() {
    let hianime = HiAnimeRust::new(None).await;
    
    let hianime_home_info = hianime.scrape_home().await;
    let hianime_about_anime_info = hianime.scrape_about_anime("one-piece-100").await;
    let hianime_episodes_info = hianime.scrape_episodes("one-piece-100").await;
    let hianime_search_query_info = hianime.scrape_search("jojo", 1).await;
    let hianime_category_info = hianime.scrape_category("ova", 3).await;
}
```

##  <span id="license">License</span>

`aniscraper` is licensed under the MIT License. See the [LICENSE](LICENSE) file for more details.

##  <span id="contributing">Contributing</span>

Contributions to `aniscraper` are welcome! If you have suggestions, improvements, or bug fixes, please submit a pull request or open an issue on our [GitHub repository](https://github.com/your-username/aniscraper). Your feedback and contributions are greatly appreciated.

---