Crate dyer[−][src]

Expand description

dyer is designed for reliable, flexible and fast Request-Response based service, including data processing, web-crawling and so on, providing some friendly, flexible, comprehensive features without compromising speed.

dyer provides some high-level features:

asynchronous, lock-free, concurrent streaming and I/O, make the best of thread pool, network, and system resource.
Event-driven, once you set the initials and recursive generator, dyer will handle the rest of it.
User-friendly and flexible, dyer offers high-level, easy to use wrappers and APIs what does a lot for you.

Get started by installing dyer-cli and looking over the examples.

Quick Start

dyer-cli is a handy tool for your easy and fast use of dyer, and recommanded to intergrate

Prerequisite

rust and cargo must be installed,
OpenSSL library is required to compile openssl-sys(if not installed, see here)

the following code helps you get the tool:

cargo install dyer-cli

Once installed, run dyer in your terminal or cmd prompt, it prints some info like

Handy tool for dyer

then dyer-cli is successfully installed.

Create A Project

Dyer-cli generates a template that contains many useful instances and instructions when using dyer with following code:

dyer new myproject

It will create a project called myproject and the files layout displays:

|___Cargo.toml
|___Readme.md
|___data/
|___data/tasks/
|___src/
    |___src/affix.rs
    |___src/entity.rs
    |___src/parser.rs
    |___src/actor.rs
    |___src/middleware.rs
    |___src/pipeline.rs

Main functionality of each file:

the affix.rs serves as an actor to adjust and satisfy additional requirement
the entity.rs contains entities/data structure to be used/collected
the parser.rs contains functions that extract entities from response
the actor.rs contains initial when opening and final things to do when closing
the middleware.rs contains Some middlewares that process data at runtime
the pipeline.rs contains entities manipulation including data-storage, displsying and so on
the lib.rs exports all modules inside the directory, just do nothing here normally
Cargo.toml is the basic configuration of the project
README.md contains some instructions of the project
data/ place to store/load files of App when load-balancing and backup

Then it is your show time, basically there are simple example items(function, enum, struct) in each file you can follow. After that check your code

dyer check

if you run it the first time, dyer-cli will download the crates and then check the code. if some warning happens such as unused import or dead code the command does a lot for you:

dyer fix

A wraper of cargo fix, if some warning happens such as unused import or dead code the command does a lot for you. However it won’t help if some errors occur, if so, you have to debug the code manually.

Edit dyer.cfg file in the root directory

the file contains some configurations of ArgApp that will update periodically, for more details see dyer.cfg Configuration

When the program compiles, haha run it:

dyer run

Short as it seems, here represents the normal procedure to write your program. It is believed that learning by example is the best, and some examples are provided to illustrate how to use.

Features to be added

Dyer-cli Command-Line Tool Support
Interrupt and Resume Support
Lock-free WorkFlow
Multiple Actors, MiddleWares, PipeLines
More to go

Problem And Feedback

It is, of course, probable that bugs and errors lie in somewhere, and defects may appear in an unexpected way, if you got any one, comments and suggestions are welcome, please new a issue in my github.

Modules

affix

This module contains structs that adjusts Task before consume it

body

Contains some data structure that related to the content of HTTP

client

the Client that asynchronously executes Request, with specified connect-timeout, read-timeout and write-timeout.

component

Instructions of components including client, affix, request, response, task, utils.

couple

A joints of a Task and Affix in practice, it is used to reproduce a Request

dyer_macros

attribute macro of dyer

engine

An instruction of App and its configuration ArgApp

info

A structure that carries basic meta-data, including its origin, stime stamp, privilege, encoding and so on.

log

A lightweight logging facade.

parsed

The result of parser when parsing Response

plugin

Instructions of plugins including middleware, pipeline and their usage.

request

This module contains structs related to HTTP requests, notably the Request type itself as well as a builder to create requests. Typically you’ll import the http::Request type rather than reaching into this module itself.

response

HTTP response types.

task

This module contains structs related to HTTP requests,

utils

some utilities that useful and convenience for dealing with data flow.

Structs

Affix

generally, it provide extra infomation , meta data required by server, including User-Agent, Accept-Encoding and so on.

App

An abstraction and collection of data flow

ArgAffix

To control the workflow of engine in dealing with Affix including using affix or not, the amount to use/generate

ArgApp

Arguments that control the App at runtime, including using history or not,
Task Affix Request Response entities consuming and generating There shall be an introduction to every member(maybe coming soon).

ArgRate

some infomation about dyer at rumtime where speed and error-handler based on

Body

series of Bytes that receives from network or to be sent

Bytes

A reference counted contiguous slice of memory.

Client

Client that take Request and execute, return Response

Couple

Group a Task and a optional Affix to make this it is the raw stuff before turned into a Request

Info

basic meta data related 3 ranging

MetaRequest

Meta Data of the Request

MetaResponse

Meta Data of the Response

MetaTask

Meta Data of the Task

MiddleWare

Represents a medium that handles the dataflow of App

MiddleWareBuilder

Serve as an medium to create an instance of MiddleWare

Parsed

the parsed result returned by parser.

PipeLine

Represents a medium that manipulates the collected data structure of App

PipeLineBuilder

Serve as an medium to create an instance of PipeLine

Request

An Wrapper of http::Request

Response

An Wrapper of http::Response

Task

An Task consists of a head and a potentially optional body. The body component is generic, enabling arbitrary types to represent the HTTP body. For example, the body could be Vec, a Stream of byte chunks, or a value that has been deserialized.