This create contains client bindings for Noria.
Noria is a new streaming data-flow system designed to act as a fast storage backend for read-heavy web applications based on this paper from OSDI'18. It acts like a databases, but pre-computes and caches relational query results so that reads are blazingly fast. Noria automatically keeps cached results up-to-date as the underlying data, stored in persistent base tables change. Noria uses partially-stateful data-flow to reduce memory overhead, and supports dynamic, runtime data-flow and query change.
Like most databases, Noria follows a server-client model where many clients connect to a
(potentially distributed) server. The server in this case is the
binary, and must be started before clients can connect. See
noria-server --help for details
and the Noria repository README for details. Noria uses
Apache ZooKeeper to announce the location of its servers, so
ZooKeeper must also be running.
If you just want to get up and running quickly, here's some code to dig into. Note that this requires a nightly release of Rust to run for the time being.
let mut db = ControllerHandle::from_zk(zookeeper_addr).unwrap(); // if this is the first time we interact with Noria, we must give it the schema db.install_recipe(" CREATE TABLE Article (aid int, title varchar(255), url text, PRIMARY KEY(aid)); CREATE TABLE Vote (aid int, uid int); "); // we can then get handles that let us insert into the new tables let mut article = db.table("Article").unwrap(); let mut vote = db.table("Vote").unwrap(); // let's make a new article let aid = 42; let title = "I love Soup"; let url = "https://pdos.csail.mit.edu"; article .insert(vec![aid.into(), title.into(), url.into()]) .unwrap(); // and then vote for it vote.insert(vec![aid.into(), 1.into()]).unwrap(); // we can also declare views that we want want to query db.extend_recipe(" VoteCount: \ SELECT Vote.aid, COUNT(uid) AS votes \ FROM Vote GROUP BY Vote.aid; QUERY ArticleWithVoteCount: \ SELECT Article.aid, title, url, VoteCount.votes AS votes \ FROM Article LEFT JOIN VoteCount ON (Article.aid = VoteCount.aid) \ WHERE Article.aid = ?;"); // and then get handles that let us execute those queries to fetch their results let mut awvc = db.view("ArticleWithVoteCount").unwrap(); // looking up article 42 should yield the article we inserted with a vote count of 1 assert_eq!( awvc.lookup(&[aid.into()], true).unwrap(), vec![vec![DataType::from(aid), title.into(), url.into(), 1.into()]] );
Noria accepts a set of parameterized SQL queries (think prepared statements), and produces a data-flow program that maintains materialized views for the output of those queries. Reads now become fast lookups directly into these materialized views, as if the value had been directly read from a cache (like memcached). The views are automatically kept up-to-date by Noria through the data-flow.
Reads work quite differently in Noria compared to traditional relational databases. In
particular, a query, or view, must be registered before it can be executed, much like SQL
prepared statements. Use
ControllerHandle::extend_recipe to register new base tables and
views. Once a view has been registered, you can get a handle that lets you execute the
corresponding query by passing the view's name to
ControllerHandle::view. The returned
View can be used to query the view with different values for its declared parameters
(values in place of
? in the query) through
Writes are fairly similar to those in relational databases. To add a new table, you extend the
ControllerHandle::extend_recipe) with a
CREATE TABLE statement, and then
ControllerHandle::table to get a handle to the new base table. Base tables support
similar operations as SQL tables, such as
Table::delete, and also more esoteric operations like
Noria provides a MySQL adapter that implements the binary MySQL protocol, which provides a compatibility layer for applications that wish to continue to issue ad-hoc MySQL queries through existing MySQL client libraries.
Types used when debugging Noria.
The prelude contains most of the types needed in everyday operation.
Represents the result of a recipe activation.
A handle to a Soup controller.
A pointer that lets you construct a new
Marker for a handle that has its own connection to Noria.
Marker for a handle that shares its underlying connection with other handles owned by the same thread.
Coordinator that shares connection information between workers and clients using ZooKeeper.
The main type used for user data throughout the codebase.
A modification to make to a column in an existing row.
A modification to make to an existing value.
An operation to apply to a base table.