Expand description
dyer is designed for reliable, flexible and fast Request-Response based service, including data processing, web-crawling and so on, providing some friendly, interoperable, comprehensive features without compromising speed.
dyer
provides some high-level features:
- asynchronous, lock-free, concurrent streaming and I/O, make the best of thread pool, network, and system resource.
- Event-driven, once you set the initials and recursive generator,
dyer
will handle the rest of it interoperably. - User-friendly and flexible,
dyer
offers high-level, easy to use wrappers and APIs what does a lot for you.
Feature Flag
To reduce code redundancy and speed up compilation, dyer use feature flag to mark the necessary modules/functions, Currently here are some supported Features:
xpath
: Enable parse the html response with xpathcompression
: Enable HTTP Compression:br
,deflate
,gzip
proxy
: Enable use proxiesfull
: Enable all features
Get started by installing dyer-cli and looking over the examples.
Modules
Contains some data structure that related to the content of HTTP
attribute macro of dyer
A structure that carries basic meta-data, including its origin, stime stamp, privilege, encoding and so on.
A lightweight logging facade.
Instructions of plugins including middleware, pipeline and their usage.
proxy
A proxy that will re-route the request to
Note that it currently supports HTTP
This module contains structs related to HTTP requests, notably the Request type itself as well as a builder to create requests. Typically you’ll import the http::Request type rather than reaching into this module itself.
HTTP response types.
This module contains structs related to HTTP requests,
some utilities that useful and convenience for dealing with data flow.
xpath
implement the xpath for the response more to see Response::xpath
Structs
generally, it provide extra infomation , meta data required by server,
including User-Agent
, Accept-Encoding
and so on.
An abstraction and collection of data flow
To control the workflow of engine in dealing with Affix including using affix or not, the amount to use/generate
some infomation about dyer
at rumtime where speed and error-handler based on
proxy
Basic authentication
proxy
Bearer authentication
proxy
Custom authentication
A cheaply cloneable and sliceable chunk of contiguous memory.
A type map of protocol extensions.
basic meta data related 3 ranging
Meta Data of the Request
Meta Data of the Response
Meta Data of the Task
Represents a medium that handles the dataflow of App
Serve as an medium to create an instance of MiddleWare
the parsed result returned by parser
.
Serve as an medium to create an instance of PipeLine
proxy
data type that represent proxy
An Wrapper of http::Request
An Wrapper of http::Response
An Task
consists of a head and a potentially optional body. The body component is
generic, enabling arbitrary types to represent the HTTP body. For example, the body could be
Vec
Enums
proxy
proxy authentication it support 3 types:
represent an client that invoke make requests
Statics
the Pool contains constructed Client
A vector of tuple which contians function name(&str) and function pointer (*const ()),
Traits
Although it starts a project, the work to do here is not very complicated it is as simple as setting up initial condition and other basic things.
Read bytes from a buffer.