Crate restapi

Expand description

Rust Rest API Stack with User Management

A secure-by-default rest api stack implemented with hyper, tokio, bb8 and postgres. This project is focused on providing end-to-end encryption by default for 12-factor applications looking to customize functionality using environment variables as needed. Includes a working user management and authentication backend written in postgresql with async S3 uploading for POST-ed data files.

Examples

Please see the restapi/examples/server.rs for developing your own rest api.

Overview

User

User password reset and user email change support using one-time-use tokens that are stored in postgres.
Users can upload and manage files stored on AWS S3 (assuming valid credentials are loaded outside this rust project).
User passwords are hashed using argon2.

Auth

User authentication enabled by default
Default JWT signing keys included with documentation for building new keys as needed.

Database

The rest api server utilizes postgres with a bb8 client threadpool.
The postgres database requires each client connection include the postgres tls certificate authority file for encrypting data in-transit.
Includes pg4admin for database management in a browser (deployed with docker compose).

TLS Encryption

Includes a tls asset generator tool (./certs/generate-tls-assets.sh) for building self-signed tls assets (requires docker).

Ingress

Component	Status
Rest API Server	Listening for encrypted client connections on tcp port 3000
Postgres	Listening for encrypted client connections on tcp port 5432 (tls Certificate Authority required)
pgAdmin	Listening for encrypted HTTP client connections on tcp port 5433

Getting Started

Clone the repo

git clone https://github.com/jay-johnson/restapi
cd restapi

Generate TLS Assets

The repository restapi includes default tls assets, but for security purposes you should generate your own. Please refer to the Generate TLS Assets doc for more information.

Here’s how to generate them under the ./certs directory:

cd certs
./generate-tls-assets.sh -f -c ./configs/dev-network.yml
cd ..

Generate JWT Keys

This repo includes default JWT signing keys, but you should generate your own signing keys under the ./jwt directory with these commands:

cd jwt
openssl ecparam -name prime256v1 -genkey -out private-key.pem
openssl pkcs8 -topk8 -nocrypt -in private-key.pem -out private-key-pkcs8.pem
openssl ec -in private-key.pem -pubout -out public-key.pem
cd ..

Please refer to the How to build JWT private and public keys for the jsonwebtokens crate doc for more information.

Build the Postgres docker image

Please refer to the Build and Deploy a Secured Postgres backend doc for more information.

Build API Server

cargo build --example server

Run API Server

export RUST_BACKTRACE=1 && export RUST_LOG=info && ./target/debug/examples/server

Supported APIs

Here are the supported json contracts for each Request and Response based off the url. Each client request is handled by the handle_requests and returned as a response back to the client (serialization using serde_json)

User APIs

Create User

Create a single users record for the new user

URL path: /user
Method: POST
Handler: create_user
Request: ApiReqUserCreate
Response: ApiResUserCreate

Update User

Update supported users fields (including change user email and password)

URL path: /user
Method: PUT
Handler: update_user
Request: ApiReqUserUpdate
Response: ApiResUserUpdate

Get User

Get a single user by users.id - by default, a user can only get their own account details

URL path: /user/USERID
Method: GET
Handler: get_user
Request: ApiReqUserGet
Response: ApiResUserGet

Delete User

Delete a single users record (note: this does not delete the db record, just sets the users.state to inactive 1)

URL path: /user
Method: DELETE
Handler: delete_user
Request: ApiReqUserDelete
Response: ApiResUserDelete

Search Users in the db

Search for matching users records in the db

URL path: /user/search
Method: POST
Handler: search_users
Request: ApiReqUserSearch
Response: ApiResUserSearch

Create One-Time-Use Password Reset Token (OTP)

Create a one-time-use password reset token that allows a user to change their users.password value by presenting the token

URL path: /user/password/reset
Method: POST
Handler: create_otp
Request: ApiReqUserCreateOtp
Response: ApiResUserCreateOtp

Consume a One-Time-Use Password Reset Token (OTP)

Consume a one-time-use password and change the user’s users.password value to the new argon2-hashed password

URL path: /user/password/change
Method: POST
Handler: consume_user_otp
Request: ApiReqUserConsumeOtp
Response: ApiResUserConsumeOtp

Verify a User’s email

Consume a one-time-use verification token and change the user’s users.verified value verified (1)

URL path: /user/verify
Method: GET
Handler: verify_user
Request: ApiReqUserVerify
Response: ApiResUserVerify

User S3 APIs

Upload a file asynchronously to AWS S3 and store a tracking record in the db

Upload a local file on disk to AWS S3 asynchronously and store a tracking record in the users_data table. The documentation refers to this as a user data or user data file record.

URL path: /user/data
Method: POST
Handler: upload_user_data
Request: ApiReqUserUploadData
Response: ApiResUserUploadData

Update an existing user data file record for a file stored in AWS S3

Update the users_data tracking record for a file that exists in AWS S3

URL path: /user/data
Method: PUT
Handler: update_user_data
Request: ApiReqUserUpdateData
Response: ApiResUserUpdateData

Search for existing user data files from the db

Search for matching records in the users_data db based off the request’s values

URL path: /user/data/search
Method: POST
Handler: search_user_data
Request: ApiReqUserSearchData
Response: ApiResUserSearchData

User Authentication APIs

Log the user in and get a json web token (jwt) back for authentication on subsequent client requests

URL path: /login
Method: POST
Handler: login_user
Request: ApiReqUserLogin
Response: ApiResUserLogin

Interation Test Guide

This project focused on integration tests for v1 instead of only rust tests (specifically everything has been tested with curl):

Please refer to the latest Integration Tests Using curl Guide

Build and run the example server

cargo build --example server && export RUST_BACKTRACE=1 && export RUST_LOG=info && ./target/debug/examples/server

Integration Tests Using curl Guide

Set up bash curl tests

export API_TLS_DIR="./certs/tls/api"
export TLS_ARGS="--cacert ${API_TLS_DIR}/api-ca.pem \
    --cert ${API_TLS_DIR}/api.crt \
    --key ${API_TLS_DIR}/api.key"

User APIs

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/login" \
    -XPOST \
    -d '{"email":"user@email.com","password":"12345"}' | jq

Create user

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user" \
    -XPOST \
    -d '{"email":"user@email.com","password":"12345"}' | jq

export TOKEN=$(curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/login" \
    -XPOST \
    -d '{"email":"user@email.com","password":"12345"}' | jq -r '.token')

Get user

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user/1" \
    -XGET \
    -H "Bearer: ${TOKEN}" | jq

Update user

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user" \
    -H "Bearer: ${TOKEN}" \
    -XPUT \
    -d '{"user_id":1,"email":"somenewemail@gmail.com","password":"321123","state":0}'

Change user password

Change to a new password

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user" \
    -H "Bearer: ${TOKEN}" \
    -XPUT \
    -d '{"user_id":1,"password":"12345a"}' | jq

Change password back to the original

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user" \
    -H "Bearer: ${TOKEN}" \
    -XPUT \
    -d '{"user_id":1,"password":"12345"}' | jq

Create a one-time-use-password (otp) allowing a user to reset their users.password from the users.email

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user/password/reset" \
    -H "Bearer: ${TOKEN}" \
    -XPOST \
    -d '{"user_id":1,"email":"user@email.com"}' | jq

Consume user one-time-use-password token to reset the users.password (otp)

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user/password/change" \
    -H "Bearer: ${TOKEN}" \
    -XPOST \
    -d '{"user_id":1,"email":"user@email.com"}' | jq

Change user email

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user" \
    -H "Bearer: ${TOKEN}" \
    -XPUT \
    -d '{"user_id":1,"email":"unique@gmail.com"}' | jq

Verify user email

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user/verify?u=1&t=2" | jq

Search user (token must be for the POST-ed user id)

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user/search" \
    -XPOST \
    -H "Bearer: ${TOKEN}" \
    -d '{"email":"user","user_id":1}' | jq

Delete user

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user" \
    -XDELETE \
    -d '{"email":"user@email.com","user_id":1}' \
    -H "Content-type: application/json" \
    -H "Bearer: ${TOKEN}" | jq

export TOKEN_HEADER="Bearer"

Token Org (embedded in the jwt)

export TOKEN_ORG="Org Name";

Token Lifetime Duration

export TOKEN_EXPIRATION_SECONDS_INTO_FUTURE=2592000;
export TOKEN_EXPIRATION_SECONDS_INTO_FUTURE=604800;
export TOKEN_EXPIRATION_SECONDS_INTO_FUTURE=86400;

JWT Signing Keys

export TOKEN_ALGO_KEY_DIR="./jwt"
export TOKEN_ALGO_PRIVATE_KEY_ORG="${TOKEN_ALGO_KEY_DIR}/private-key.pem"
export TOKEN_ALGO_PRIVATE_KEY="${TOKEN_ALGO_KEY_DIR}/private-key-pkcs8.pem"
export TOKEN_ALGO_PUBLIC_KEY="${TOKEN_ALGO_KEY_DIR}/public-key.pem"

Generate your own jwt keys with these commands

These commands were tested on ubuntu 21.10 using bash:

openssl ecparam -name prime256v1 -genkey -out "${TOKEN_ALGO_PRIVATE_KEY_ORG}"
openssl pkcs8 -topk8 -nocrypt -in private-key.pem -out "${TOKEN_ALGO_PRIVATE_KEY}"
openssl ec -in "${TOKEN_ALGO_PRIVATE_KEY_ORG}" -pubout -out "${TOKEN_ALGO_PUBLIC_KEY}"

S3

Setting up AWS credentials

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html

export AWS_ACCESS_KEY_ID=ACCESS_KEY
export AWS_SECRET_ACCESS_KEY=SECRET_KEY

S3 Upload a user data file (no file type restrictions + s3 archival)

export UPLOAD_FILE="./README.md"
export DATA_TYPE="file"
export S3_DATA_BUCKET="BUCKET_NAME"
export S3_DATA_PREFIX="user/data/file"

curl -s ${TLS_ARGS} \
    -XPOST \
    --data-binary "@${UPLOAD_FILE}" \
    "https://0.0.0.0:3000/user/data" \
    -H "Bearer: ${TOKEN}" \
    -H 'user_id: 1' \
    -H 'comments: this is a test comment' \
    -H 'encoding: na' \
    -H 'Content-type: text/txt' \
    -H 'filename: README.md' \
    -H "data_type: ${DATA_TYPE}" | jq

Search user data (token must be for the POST-ed user id)

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user/data/search" \
    -XPOST \
    -H "Bearer: ${TOKEN}" \
    -d '{"user_id":1}' | jq

Update a single user data record (token must be for the PUT user id)

curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/user/data" \
    -XPUT \
    -H "Bearer: ${TOKEN}" \
    -d '{"user_id":1,"data_id":1,"comments":"updated comment using curl"}' | jq

export TOKEN=$(curl -s ${TLS_ARGS} \
    "https://0.0.0.0:3000/login" \
    -XPOST \
    -d '{"email":"user@email.com","password":"12345"}' | jq -r '.token')

psql --set=sslmode=require -h 0.0.0.0 -p 5432 -U postgres -d mydb

Get public tables in the mydb

SELECT table_name FROM information_schema.tables WHERE table_schema='public';

Modules

core

handle_request

Handle all Client Requests