1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
//! # Filter list manager library core
//!
//! ## Overview
//!
//! This crate represents a library for managing AdGuard filter lists.
//!
//! This library can:
//!
//! - Fetch filter lists
//! - Store the downloaded filter list
//! - Perform filter lists updates
//! - ... and more
//!
//! ## Filters analysis
//!
//! ### List of meta tags that the library parses from filter content
//!
//! - `! Title` - Name of the filter.
//! - `! Description` - Detailed description of the filter.
//! - `! Version` - Current version of the filter.
//! - `! Expires` - Filter expiration period. Will be converted into seconds. [See the tests for an example](./src/filters/parser/metadata/parsers/expires.rs) If this field is missing in the metadata, the global value from the [configuration](./src/manager/models/configuration/mod.rs) will be used. Before updating the filter, the value will be checked and aligned to the lower boundary ([3600](./src/manager/models/configuration/mod.rs)) if it is less than this value.
//! - `! Homepage` - Filter website/homepage.
//! - `! TimeUpdated` - When this filter was updated in registry. Format: `2024-08-13T13:30:53+00:00`.
//! - `! Last modified` - Alias for `TimeUpdated`. Format: `2024-08-13T12:01:26.703Z`. You can choose one format for both fields.
//! - `! Diff-Path` - [Differential updates](https://github.com/ameshkov/diffupdates?tab=readme-ov-file#-diff-path) information
//! - `! License` - Link to filter license.
//! - `! Checksum` - Filter's base64(md5-checksum). Before update/install filter, checksum will be calculated and compared. See the source [here](./src/filters/parser/checksum_validator.rs)
//!
//! ### List of filter preprocessor directives supported by the library
//!
//! [See AdGuard preprocessor directives](https://adguard.com/kb/general/ad-filtering/create-own-filters/#preprocessor-directives)
//!
//! The library supports:
//!
//! - `!#include file_path` - Includes contents of file into filter and process. `file_path` must be:
//! - Absolute url with the [same origin](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy) as the parent filter.
//! - Relative url.
//! - File url (only if the parent filter's url has `file` scheme).
//! - `!#if/!#endif/!#else` - Condition compilation directives. They can be nested. Supported tokens:
//! - `()` - parentheses
//! - `true/false` - boolean values
//! - `&& ||` - AND/OR operators
//! - `!` - NOT operator
//! - Literal compiler constant from [configuration](./src/manager/models/configuration). For example, `windows`,`mac`, etc... It works like this: if the constant encountered is in the `configuration.compiler_conditional_constants` list, then the condition becomes **true**, **false** otherwise
//!
//! **See the tests for more information:**
//!
//! - [All directives](./src/filters/parser.rs)
//! - [!#include](./src/filters/parser/include_processor.rs)
//! - [!#if / !#endif / !#else](./src/filters/parser/boolean_expression_parser.rs)
//!
//! ## Usage
//!
//! ### Create and setup configuration for library facade
//!
//!
//! // Every instance of FilterListManager must have its own configuration
//! `let mut configuration = Configuration::default();`
//!
//! // Sets urls for filters indices.
//! `configuration.metadata_url = "https://filters.adtidy.org/extension/safari/filters.json".to_string();`
//! `configuration.metadata_locales_url = "https://filters.adtidy.org/extension/safari/filters_i18n.json".to_string();`
//!
//! // Sets locale. Will be used for returning localized strings for filters,
//! // groups, and tags, where applicable.
//! `configuration.locale = "pt_PT".to_string();`
//!
//! // Sets type of filters lists.
//! // By default, FilterListType::STANDARD will be selected.
//! configuration.filter_list_type = FilterListType::DNS;
//!
//! // Creates facade instance
//! `let flm = FilterListManagerImpl::new(configuration);`
//!
//!
//! #### Example references
//!
//! [Configuration reference](./src/manager/models/configuration/mod.rs)\
//! [FilterListManager reference](./src/manager/mod.rs)
//!
//! ---
//!
//! ### How to create and fill up standard filters database
//!
//!
//! // Creates and configures the database. Populates the database with information
//! // from the filter indexes (filters metadata), the paths to which are specified
//! // in the configuration.
//! // In addition, this method applies migrations that have not yet been applied.
//! // See the lift_up_database method for details on "lifting" a database.
//! `flm.pull_metadata();`
//!
//! // Then, downloads the contents of the filters.
//! `flm.update_filters(true, 0, true);`
//!
//!
//! > [!NOTE]
//! > By default, the application operates with a database located in the current
//! > working directory (**cwd**), and the database file name is generated based on
//! > the format `agflm_{configuration.filter_list_type.to_string()}`. For standard
//! > filters, the file path will be `$CWD/agflm_standard.db`.
//!
//! ---
//!
//! ### Database scheme updates
//!
//! Database schema updates (migrations) are possible using the `flm.lift_up_database()` method.
//! The method “raises” the state of the database to the working state.
//!
//! **If the database doesn't exist:**
//! - Creates database
//! - Rolls up the schema
//! - Rolls migrations
//! - Performs bootstrap.
//!
//! **If the database is an empty file:**
//! - Rolls the schema
//! - Rolls migrations
//! - Performs bootstrap.
//!
//! ... and so on.
//!
//! ### Usage notes
//!
//! First connection to database almost always "lift" the database.
//! So you need this only in special cases, like old database backups
//!
//! ---
//!
//! ### Operations with custom filters
//!
//! The library categorizes all filters into three types:
//!
//! 1. **Index Filters** - Filters created by parsing the index (registry).
//! 2. **Custom Filters** - Filters added (and edited) by the user using the
//!
//! library's methods.
//! 3. **Special Filters** - Custom filters preconfigured by the library's scripts.
//!
//! You can refer to the [db constants file][constants] to check the indicators for
//! special and custom filters.
//!
//! // Installs a custom filter.
//! ```
//! use adguard_flm::{Configuration, FilterListManagerImpl, FilterListManager};
//! let flm = FilterListManagerImpl::new(Configuration::default())
//! .unwrap();
//!
//! let custom_filter = flm.install_custom_filter_list(
//! String::from("https://example.com/custom_filter.txt"),
//! true, // The filter list is marked as trusted.
//! Some(String::from("Custom title")),
//! Some(String::from("Custom description"))
//! ).unwrap();
//!
//! // Edit metadata.
//! flm.update_custom_filter_metadata(
//! custom_filter.id,
//! String::from("new title"),
//! false // The filter list is marked as not trusted.
//! ).unwrap();
//!
//! // Turn on this filter.
//! flm.enable_filter_lists(vec![custom_filter.id], true).unwrap();
//!
//! // Remove this filter.
//! flm.delete_custom_filter_lists(vec![custom_filter.id]).unwrap();
//! ```
//!
//! #### Installing a custom filter from a string instead of downloading it
//!
//! ```
//! use adguard_flm::{Configuration, FilterListManagerImpl, FilterListManager};
//! let flm = FilterListManagerImpl::new(Configuration::default())
//! .unwrap();
//!
//! let string_contents = String::from(r###"
//! ! Checksum: ecbiyIyplBZKLeNzi64pGA
//! ...
//! ! JS API START
//! #%#var AG_onLoad=function(func){if(document.readyState==="complete"||document.readyState==="interactive")func();else
//! ...
//! "###);
//!
//! flm.install_custom_filter_from_string(
//! String::new(), // download url
//! 1719505304i64, // last_download_time value. Explanation: Can we update filter? Answer: (filter.last_download_time + filter.expires < now())
//! true, // Enabled
//! true, // Trusted
//! string_contents, // Filter body
//! None, // Filter title - Option<String>
//! None // Filter description - Option<String>
//! ).unwrap();
//! ```
//!
//! [constants]: ./crates/filter-list-manager/src/storage/constants.rs
//!
//! #### Operations with custom filters rules
//!
//! ```
//! // Saves the structure containing the filter rules.
//! flm.save_custom_filter_rules(/* FilterListRules */ rules_for_new_local_custom_filter);
//!
//! // You can save only disabled rules for the filter list
//! flm.save_disabled_rules(filter.id, /* Vec<String> */ disabled_rules_list);
//! ```
//!
//! #### Example references
//!
//! [FilterListRules reference](./src/manager/models/filter_list_rules.rs)
//!
//! ---
//!
//! ### Get operations
//!
//! ```
//! // Retrieves all filters metadata from the database **with** theirs rules.
//! // Returns Vec<FullFilterList>.
//! flm.get_full_filter_lists();
//!
//! // Retrieves a filter metadata by its ID from the database **with** its rules.
//! // Returns Optional<FullFilterList>.
//! flm.get_full_filter_list_by_id(id);
//!
//! // Retrieves all enabled filters as ActiveRulesInfo.
//! flm.get_active_filters();
//!
//! // Retrieves all filters metadata from the database **without** theirs rules.
//! // Returns Vec<StoredFilterMetadata>
//! flm.get_stored_filters_metadata();
//!
//! // Retrieves a filter metadata by its ID from the database **without** its rules.
//! // Returns Optional<StoredFilterMetadata>.
//! flm.get_stored_filter_metadata_by_id(id);
//! ```
//!
//! #### Example references
//!
//! [FullFilterList reference](./src/manager/models/full_filter_list.rs)\
//! [StoredFilterMetadata reference](./src/manager/models/stored_filter_metadata.rs)\
//! [ActiveRulesInfo reference](./src/manager/models/active_rules_info.rs)
//!
//! ### Other (All) operations
//!
//! [Facade Interface](./src/manager/mod.rs)
//!
pub use crateFilterParserError;
pub use crateIOError;
pub use crateHttpClientError;
/// # Re-exports
pub use crateFilterListManagerImpl;
pub use crateActiveRulesInfo;
pub use crateActiveRulesInfoRaw;
pub use crateConfiguration;
pub use crateFilterListType;
pub use crateLocale;
pub use crateRequestProxyMode;
pub use crateDisabledRulesRaw;
pub use crateFilterGroup;
pub use crateFilterListRules;
pub use crateFilterListRulesRaw;
pub use crateFilterTag;
pub use crateFLMError;
pub use crateRulesCountByFilter;
pub use crateStoredFilterMetadata;
pub use crateUpdateFilterError;
pub use crateFilterId;
pub use crateFilterListMetadata;
pub use crateFilterListMetadataWithBody;
pub use crateFullFilterList;
pub use crateUpdateResult;
pub use crate;
pub use crateFilterListManager;
pub use crate*;
pub use crategenerate_random_key;
pub
pub
/// Customized [`Result`] with error [`FLMError`]
pub type FLMResult<T> = ;