octopux-cli 0.18.2

CLI to generate base models for octopux
octopux-cli-0.18.2 is not a library.

octopux

Generate JSON CRUD endpoints for Actix Web from your structs, with little boilerplate.

  • 5 REST routes per model (GET, GET list, POST, PUT, DELETE) generated by derives and macros

  • Persistence with sqlx (SQLite, PostgreSQL, MySQL): the queries are written for you at compile time

  • OpenAPI documentation with apistos and Swagger UI

  • Has-many and many-to-many relations, listed on GET /{model}/{id}/{relation}

  • A CLI generating the models, relations and SQL migrations

  • Octopux requires actix-web v4.

Table of contents

Installation

The library

cargo add --git https://github.com/ctaque/octopux --tag v0.17.3 octopux
cargo add actix-web
cargo add serde --features derive

serde is needed because the generated models derive Serialize / Deserialize. The generated code reaches async_trait and anyhow through octopux, which re-exports them, so they don't need to be dependencies of your crate.

Optional features:

Feature Enables Extra dependencies
sqlx The SqlxModel, SqlxNewModel, SqlxUpdatableModel and SqlxFilter derives (see sqlx models) cargo add sqlx --features runtime-tokio,sqlite,macros,migrate,chrono (pick your database driver)
openapi The documented routes (see OpenAPI documentation) cargo add apistos --features chrono,swagger-uicargo add schemars --rename schemars --package apistos-schemars
cargo add --git https://github.com/ctaque/octopux --tag v0.17.3 octopux --features openapi,sqlx

Add chrono (cargo add chrono --features serde) when your models have date fields or use --timestamps.

The CLI

Once per machine:

cargo install --git https://github.com/ctaque/octopux --tag v0.17.3 octopux-cli   # installs the `octopux` binary
cargo install sqlx-cli                                                                        # optional, for `sqlx migrate run`

Quick start

This walkthrough builds a Project API persisted in SQLite with sqlx and documented with OpenAPI. It needs the openapi and sqlx features and their dependencies (see Installation).

1. Generate a model

cargo new my-api && cd my-api/src
octopux generate-model --name Project --fields --sqlx --migration --timestamps --openapi

(The menu of field types printed before each type is omitted here, see generate-model.)

This produces:

  • src/project.rs: the Project, NewProject and UpdatableProject structs, their query structs, the sqlx derives implementing the CRUD queries, and a configure function registering the documented routes;
  • migrations/<timestamp>_create_project.sql: the migration creating the project table.

2. Wire it into main.rs

Tip: octopux --bootstrap --openapi, run at the root of the crate, writes the src/main.rs and src/helpers.rs below and installs their dependencies, see --bootstrap.

The generated model imports the application state from the crate root (use crate::AppState;), so main.rs declares it, with the sqlx pool:

2.1) Create a src/helpers.rs file with the AppState struct, which will be shared between models files and the main.rs file.

2.2) Wire into main.rs

mod project;
use crate::shared::AppState;
use actix_web::web;
use apistos::app::{BuildConfig, OpenApiWrapper};
use apistos::info::Info;
use apistos::spec::Spec;
use apistos::SwaggerUIConfig;
use sqlx::SqlitePool;

#[actix_web::main]
async fn main() -> std::io::Result<()> {
    let pool = SqlitePool::connect("sqlite://data.db?mode=rwc").await.unwrap();
    sqlx::migrate!().run(&pool).await.unwrap();
    let state = web::Data::new(AppState { pool });

    actix_web::HttpServer::new(move || {
        let spec = Spec {
            info: Info {
                title: "MyApp REST API".to_string(),
                version: "1.0.0".to_string(),
                ..Default::default()
            },
            ..Default::default()
        };
        actix_web::App::new()
            .document(spec)
            // Actix does not fall through between scopes sharing a prefix,
            // so resources living under the same scope must be registered together
            .service(
                apistos::web::scope("v1")
                    .configure(project::configure), // Where the magic operates
            )
            .app_data(state.clone())
            .build_with(
                "/openapi.json",
                BuildConfig::default().with(SwaggerUIConfig::new(&"/swagger")),
            )
    })
    .bind(("127.0.0.1", 8085))?
    .run()
    .await
}

3. Run it

cargo run
Route Action
GET /v1/project List a page of projects
GET /v1/project/{id} Find a project
POST /v1/project Create a project
PUT /v1/project/{id} Update a project
DELETE /v1/project/{id} Delete a project

The OpenAPI document is served on /openapi.json and Swagger UI on /swagger.

Note: when the updatable struct has an id field, PUT /v1/project/{id} answers 400 ID_MISMATCH if the payload id differs from the path {id}.

How it works

A resource is made of three structs, each with a derive generating its HTTP handlers and a trait holding your logic:

Struct Derive Trait to implement Routes
Project HttpFindListDelete Model (find, list, delete) GET /project/{id}, GET /project, DELETE /project/{id}
NewProject HttpCreate NewModel (save) POST /project
UpdatableProject HttpUpdate UpdatableModel (update) PUT /project/{id}

The traits can be implemented by hand, see examples/simple, or by the sqlx derives.

Guides

OpenAPI documentation with apistos

  1. Create the main.rs as described in the above section.
  2. Add required dependancies :
[dependencies]
octopux = { version = "0.9", git = "https://github.com/ctaque/octopux", features = ["openapi"] }
apistos = { version = "0.9", features = ["chrono", "swagger-ui"] }
# apistos relies on its fork of schemars
schemars = { package = "apistos-schemars", version = "0.8" }
  1. Then run octopux generate-model with the --openapi flag. This will generate the openapi spec at /swagger.

See the Swagger UI generated for the Project resource.

Troubleshooting: apistos parses route paths with a regex syntax introduced in regex 1.9, but accepts older versions. If the app panics with path name regex, run cargo update -p regex.

sqlx models

With the sqlx feature, three derives implement the model traits with sqlx queries, written at compile time from the struct fields (a field is a column of the same name, r#type the type column). They run on the pool field of the application state, and read the query and state types from the http_* attribute of the struct.

Your application depends on sqlx itself, for sqlx::FromRow and the pool, with the driver of its database and a runtime.

// Don't touch this file.
// It is automatically generated by the octopux crate cli.

use octopux::{HttpCreate, HttpFindListDelete, HttpUpdate, SqlxModel, SqlxNewModel, SqlxUpdatableModel, octopux_info};

#[derive(Deserialize)]
pub struct ListQuery {
    pub offset: Option<usize>, // read by SqlxModel's list
    pub limit: Option<usize>,
}

#[derive(Serialize, Deserialize, sqlx::FromRow, HttpFindListDelete, SqlxModel)]
#[http_find_list_delete(Id, FindQuery, ListQuery, DeleteQuery, AppState)]
#[sqlx_model(database = "postgres", timestamps, soft_delete)]
#[octopux_info(path = "project")]
pub struct Project {
    pub id: Id,
    pub title: String,
    pub created_at: Option<DateTime<Utc>>,
    pub updated_at: Option<DateTime<Utc>>,
    pub deleted_at: Option<DateTime<Utc>>,
}

#[derive(Serialize, Deserialize, HttpCreate, SqlxNewModel)]
#[http_create(SaveQuery, AppState)]
#[sqlx_model(database = "postgres", model = "Project", timestamps)]
pub struct NewProject {
    pub title: String,
}

#[derive(Serialize, Deserialize, sqlx::FromRow, HttpUpdate, SqlxUpdatableModel)]
#[http_update(Id, UpdateQuery, Project, FindQuery, AppState)]
#[sqlx_model(database = "postgres", timestamps, soft_delete)]
pub struct UpdatableProject {
    pub id: Id,
    pub title: String,
    pub updated_at: Option<DateTime<Utc>>,
}
Derive Generated queries
SqlxModel find selects the row by id; list selects a page ordered by id (offset, and limit of 20 by default and 100 at most); delete removes the row and returns it
SqlxNewModel save inserts the fields of the struct and returns the model row
SqlxUpdatableModel update sets every field but id on the row id, and returns the fields of the struct

#[sqlx_model(...)] options

Key Description
database Required. "sqlite", "postgres" or "mysql". SQLite and PostgreSQL use $N placeholders and RETURNING, MySQL ? placeholders and a SELECT of the row (with last_insert_id() after the insert)
table The table, the lowercase name of the model by default
model Required on SqlxNewModel. The model returned by save
pool The field of the application state holding the sqlx pool, pool by default
timestamps save sets created_at and updated_at to Utc::now(), update refreshes updated_at, and the payload does not set them
soft_delete delete sets deleted_at instead of removing the row, and find, list and update skip the rows whose deleted_at is set
default_limit, max_limit The page size of list without limit, and its maximum
filter On SqlxModel: list applies the filters and the sort of its list query, which derives SqlxFilter (see below), and orders by id without sort
before_save On SqlxNewModel and SqlxUpdatableModel: save and update first pass the payload through your BeforeSave implementation, see below

Transforming the payload with before_save

With before_save, the struct implements BeforeSave, which transforms the payload before it is written, e.g. to hash a password. It is async and receives the application state; an error aborts the query and answers 500:

use octopux::{BeforeSave, anyhow::Result, async_trait};

#[derive(Serialize, Deserialize, HttpCreate, SqlxNewModel)]
#[http_create(SaveQuery, AppState)]
#[sqlx_model(database = "postgres", model = "User", before_save)]
pub struct NewUser {
    pub email: String,
    pub password: String,
}

#[async_trait]
impl BeforeSave<AppState> for NewUser {
    async fn before_save(mut self: Self, _state: &AppState) -> Result<Self> {
        let password = std::mem::take(&mut self.password);
        // CPU-bound hashing runs off the actix workers
        self.password = actix_web::rt::task::spawn_blocking(move || hash(&password)).await??;
        Ok(self)
    }
}

Filtering and sorting with SqlxFilter

The SqlxFilter derive turns the Option fields of a list query into the conditions of a WHERE clause, one for each field that is set. The column and the operator come from the field name, name filters name = ... and price_gte filters price >= ... (suffixes _ne, _gt, _gte, _lt, _lte and _like), or from #[sqlx_filter(column = "...", op = "...")]. offset and limit are left to the pagination, #[sqlx_filter(skip)] leaves out another field. The values are bound, never written in the SQL:

use octopux::SqlxFilter;

#[derive(Deserialize, SqlxFilter)]
#[sqlx_filter(database = "postgres")]
pub struct ListQuery {
    pub offset: Option<usize>,
    pub limit: Option<usize>,
    pub name: Option<String>,      // ?name=...
    pub price_gte: Option<i32>,    // ?price_gte=...
    #[sqlx_filter(column = "name", op = "like")]
    pub q: Option<String>,         // ?q=alpha%
    #[sqlx_filter(sort = "name, price")]
    pub sort: Option<String>,      // ?sort=-price,name
    #[sqlx_filter(sort_direction)]
    pub order: Option<String>,     // ?order=desc
}

let mut qb = sqlx::QueryBuilder::new("SELECT * FROM project");
let mut has_where = false; // true when the query already has a WHERE clause
query.push_filters(&mut qb, &mut has_where);
if !query.push_order_by(&mut qb)? {
    qb.push(" ORDER BY id"); // without `sort`
}

The Option<String> field marked #[sqlx_filter(sort = "...")] gives the ORDER BY clause: the columns of its value separated by commas, prefixed with - in descending order. Only the columns listed by the attribute are accepted, each once; any other value makes push_order_by return an error.

The Option<String> field marked #[sqlx_filter(sort_direction)] takes asc or desc (in any case) and orders the columns that are not prefixed with -: ?sort=price,name&order=desc gives ORDER BY price DESC, name DESC. They are ascending without it, and another value is an error. It requires the sort field.

Has-many relations

The CLI can generate relations for you, see generate-relation.

A relation lists the children of a model on GET /{path}/{id}/{relation}, e.g. the books of a project on GET /v1/project/{id}/books, paginated by its query rather than embedded in the parent.

The HasMany trait is implemented on a type standing for the relation, so a model can have several relations, even towards the same child model:

// Don't touch this file.
// It is automatically generated by the octopux crate cli.

use octopux::{HasMany, anyhow::Result, async_trait, gen_relation_endpoint};

pub struct ProjectBooks;

#[async_trait]
impl HasMany for ProjectBooks {
    type Parent = Project;          // its `octopux_info` path prefixes the route
    type Id = Id;
    type Query = ProjectBooksQuery; // e.g. `offset` and `limit`
    type Result = Vec<Book>;
    type State = AppState;
    const RELATION: &'static str = "books";

    async fn list_related(id: Id, query: &ProjectBooksQuery, state: &AppState) -> Result<Option<Vec<Book>>> {
        // `None` when no project has this id (404 ENTITY_NOT_FOUND), the page of its books otherwise
    }
}

CLI reference

The octopux binary writes the models and relations in the src folder of the working directory when it exists, in the working directory otherwise (e.g. when run from src). The generated files only rely on octopux and serde, plus sqlx, chrono and apistos depending on the options.

--bootstrap

cargo init my-api && cd my-api
octopux --bootstrap [--openapi]

Run at the root of the crate, --bootstrap writes the files of an actix server ready to mount generated models:

  • src/main.rs: connects a SQLite pool on data.db, shares it in the AppState, and serves an empty v1 scope on 127.0.0.1:8085;
  • src/helpers.rs: the AppState struct, holding the sqlx pool.
Option Description
--openapi Serve the routes on an apistos app, with the OpenAPI document on /openapi.json and Swagger UI on /swagger, to mount models generated with --openapi. Requires --bootstrap

The project is not bootstrapped when src/helpers.rs already exists, and an existing src/main.rs (such as the one of cargo init) is only overwritten once confirmed.

The CLI then offers to install the dependencies with cargo add (it needs a Cargo.toml in the working directory):

Dependency Added
octopux, from the tag of the CLI version, with the sqlx feature (and openapi with --openapi) Always
actix-web@4, serde@1 (derive), chrono@0.4 (serde) Always
sqlx@0.9 (runtime-tokio, sqlite, chrono, macros, migrate) Always
apistos@0.9 (chrono, swagger-ui), apistos-schemars@0.8 renamed schemars With --openapi

Then generate a model from the crate root (it is written in src), declare it with mod project; and mount it with .configure(project::configure) in the v1 scope of src/main.rs:

octopux generate-model --name Project --fields --sqlx --migration --openapi

Uncomment sqlx::migrate!().run(&pool) in src/main.rs once the crate has migrations.

generate-model

octopux generate-model --name Project [OPTIONS]

Generates project.rs, to declare with mod project;. Its structs are public, and it imports AppState from the crate root (use crate::AppState;): declare your application state in main.rs or lib.rs, or change this import.

Option Description
-n, --name The model name
--fields Ask interactively for the fields, see below
--sqlx Implement the queries with the sqlx derives (requires --fields)
--migration Create the SQL migration of the table (requires --fields)
--foreign-keys Ask, for each field, for the table and the column it references, see below (requires --migration)
--unique Ask, for each field, whether its column is unique, see below (requires --migration)
--sqlite (default), --postgres, --mysql The database targeted by --sqlx and --migration
--timestamps Add created_at, updated_at and deleted_at fields, with soft delete
--openapi Derive JsonSchema and ApiComponent, and generate the configure function mounting the documented routes
--force Overwrite an existing project.rs

Without --openapi, mount the routes with gen_endpoint!(Project, NewProject, UpdatableProject) (see How it works).

--fields

The CLI asks for the name and type of each field. The type is picked by its number in the menu, or typed as any Rust type, and defaults to String; an empty name ends the input. The fields are added to Project, NewProject and UpdatableProject, next to the id: Id field that is always generated:

$ octopux generate-model --name Project --fields
? Field 1 name › title
  1) String  2) i32  3) i64  4) f64  5) bool  6) Option<String>  7) DateTime<Utc>  ...
? Type of `title` › (number or custom type) [String]
  ✔ title: String
? Field 2 name › stars:2?
  ✔ stars: Option<i32> (nullable)
? Field 3 name ›

Shortcuts:

  • name:type gives the type with the name, without the menu (stars:i32 or stars:2)
  • a trailing ? makes the type optional: 2? gives Option<i32>, ? alone Option<String>
  • - removes the last field

The declared fields and the files to write (created, or overwritten with --force) are summed up, then the CLI asks for saving them: only n cancels, nothing is written then. The output is colored in a terminal, set NO_COLOR to disable it.

--sqlx

Requires --fields with at least one field, and the sqlx feature of octopux. Project, NewProject and UpdatableProject derive SqlxModel, SqlxNewModel and SqlxUpdatableModel instead of leaving the find, list, delete, save and update functions to fill: the queries on the project table run on the pool field of your AppState. Project and UpdatableProject also derive sqlx::FromRow, and ListQuery gets the offset and limit of the page.

octopux generate-model --name Project --fields --sqlx --postgres

--migration

Requires --fields. Creates <timestamp>_create_project.sql in the migrations folder next to src (the parent migrations folder when run from inside src), ready for sqlx migrate run or sqlx::migrate!().

Each field type is mapped to the column type sqlx declares for it (<T as sqlx::Type<DB>>::type_info().name()), so the columns always decode into the model fields. Option<T> fields are nullable.

With PostgreSQL and MySQL, String columns are VARCHAR: after the type, the CLI asks for their length, 255 when empty (up to 10485760 with PostgreSQL, 16383 with MySQL).

? Field 1 name › title:String
? Length of `title` › (VARCHAR, 1 to 10485760) [255] 80
  ✔ title: String (length 80)
Rust type SQLite PostgreSQL MySQL
String TEXT VARCHAR(255) VARCHAR(255)
i8 INTEGER TINYINT
i16 INTEGER INT2 SMALLINT
i32 INTEGER INT4 INT
i64 INTEGER INT8 BIGINT
u8, u16, u32 INTEGER TINYINT UNSIGNED, SMALLINT UNSIGNED, INT UNSIGNED
u64 BIGINT UNSIGNED
f32 REAL FLOAT4 FLOAT
f64 REAL FLOAT8 DOUBLE
bool BOOLEAN BOOL BOOLEAN
DateTime<Utc> DATETIME TIMESTAMPTZ DATETIME(6)
NaiveDateTime DATETIME TIMESTAMP DATETIME(6)
NaiveDate DATE DATE DATE
NaiveTime TIME TIME TIME(6)
Vec<u8> BLOB BYTEA BLOB
Vec<String>, Vec<i16>, Vec<i32>, Vec<i64>, Vec<f64>, Vec<bool> TEXT[], INT2[], INT4[], INT8[], FLOAT8[], BOOL[]
octopux generate-model --name Project --fields --sqlx --migration --postgres

--foreign-keys

Requires --migration. After the type of each field, the CLI proposes the tables and their columns: the field becomes a foreign key of the migration. An empty answer means no reference, and the column defaults to the primary key.

The tables are read from the database of DATABASE_URL when it is set (the tables of the current schema with PostgreSQL and MySQL; a relative SQLite file is also looked up at the crate root, and opened read-only). Otherwise they are read from the CREATE TABLE statements of the migrations folder. The model itself is always proposed, for a self reference (parent_id), with the fields declared before.

$ octopux generate-model --name Book --fields --migration --foreign-keys --postgres
✔ 2 tables read from `DATABASE_URL`
? Field 1 name › author_id:i64
  1) author  2) project  3) book (this model)
? Table referenced by `author_id` › (number or name, empty for none) 1
  1) id INT8 (unique)  2) name TEXT
? Column of `author` referenced by `author_id` › (number or name) [id]
  ✔ author_id: i64 → author (id)
CREATE TABLE IF NOT EXISTS book (
    id BIGSERIAL PRIMARY KEY,
    author_id INT8 NOT NULL,
    FOREIGN KEY (author_id) REFERENCES author (id)
);

The CLI warns when the referenced column is neither a primary key nor unique, or when its type differs from the column of the field (i32 referencing a BIGSERIAL id), as the database would refuse the foreign key. Declare the referencing fields as i64 to reference the id of the generated models.

--unique

Requires --migration. After the type of each field (and its reference with --foreign-keys), the CLI asks whether its column is unique: the migration then declares a UNIQUE constraint on it. An empty answer means not unique. With --foreign-keys, the unique fields of the model are proposed as unique columns for a self reference.

$ octopux generate-model --name Author --fields --migration --unique --postgres
? Field 1 name › email:String
? Is `email` unique › (y/N) y
  ✔ email: String (unique)
CREATE TABLE IF NOT EXISTS author (
    id BIGSERIAL PRIMARY KEY,
    email VARCHAR(255) NOT NULL,
    UNIQUE (email)
);

With --mysql, the CLI warns when the column is a BLOB or a TEXT (Vec<u8>), on which MySQL refuses a unique index without a key length.

--timestamps

Project gets created_at, updated_at and deleted_at fields and UpdatableProject an updated_at field, all typed Option<DateTime<Utc>> from chrono. Add chrono with its serde feature to your dependencies, and the chrono feature of sqlx or apistos when you use --sqlx or --openapi.

  • With --sqlx, the derives get the timestamps and soft_delete options: save sets created_at and updated_at, update refreshes updated_at, and delete sets deleted_at instead of removing the row. find, list and update skip the deleted rows (a deleted row answers like a missing one).
  • With --migration, the table gets nullable created_at, updated_at and deleted_at columns.
octopux generate-model --name Project --fields --sqlx --migration --timestamps

--force

generate-model refuses to overwrite an existing project.rs; --force overwrites it. The code written in the model is then lost, and the new create_project migration does nothing on a table that already exists.

generate-relation

octopux generate-relation --parent Project --child Book [OPTIONS]

Generates a has-many relation in project_books.rs for the books of a Project, to declare with mod project_books; and mount with .configure(project_books::configure) in the scope of the parent routes. It imports Project and Id from crate::project and Book from crate::book, as generated by generate-model.

Option Description
--parent The parent model
--child The child model
--name The route segment, the plural of the child by default (books)
--foreign-key The column of the children referencing the parent, project_id by default
--through For a many-to-many relation: the join model
--child-key With --through: the column of the join model referencing the child, category_id by default
--sqlx list_related selects a page of the children ordered by id (offset, and limit of 20 by default and 100 at most), and looks the parent up only when the page is empty, to answer 404 for an unknown parent
--migration Create the migration indexing the foreign key (<timestamp>_index_book_project_id.sql). Generate it after the migration creating the table
--timestamps Skip the soft deleted children, join rows and parents
--openapi, --sqlite, --postgres, --mysql, --force Same as for generate-model

One-to-many: the books of a project, on GET /project/{id}/books:

octopux generate-relation --parent Project --child Book --foreign-key=project_id --sqlx --openapi --migration --postgres

Many-to-many: the categories of a project, through a ProjectCategory join model whose project_id and category_id columns reference the parent and the child, on GET /project/{id}/categories:

octopux generate-relation --parent Project --child Category --through ProjectCategory --sqlx --openapi --migration --postgres

Examples

The examples folder contains runnable projects:

Example Shows
simple The traits implemented by hand, with gen_endpoint!
pagination A paginated list
openapi sqlx models documented with OpenAPI and Swagger UI
relations_filters_sort Has-many and many-to-many relations generated by the CLI, and a list filtered and sorted with SqlxFilter (PostgreSQL)

They come with request collections for Hoppscotch (and Insomnia for some).

Issues

Sqlx fail to compile with strip. See : https://github.com/ctaque/octopux/issues/5