atomic-server
https://user-images.githubusercontent.com/2183313/139728539-d69b899f-6f9b-44cb-a1b7-bbab68beac0c.mp4
Status: Alpha. Not ready for production time. Prone to changes and corrupt databases when upgrading. Changelog
Atomic-server is a graph database server for storing and sharing Atomic Data. Demo on atomicdata.dev
- âïļ Dynamic schema validation / type checking using Atomic Schema. Combines safety of structured data with the
- ð Fast (1ms responses on my laptop)
- ðŠķ Lightweight (15MB binary, no runtime dependencies)
- ðŧ Runs everywhere (linux, windows, mac, arm)
- ð Embedded server with support for HTTP / HTTPS / HTTP2.0 and Built-in LetsEncrypt handshake.
- ðïļ Browser GUI included powered by atomic-data-browser. Features dynamic forms, tables, authentication, theming and more.
- ðū Event-sourced versioning / history powered by Atomic Commits
- ð Synchronization using websockets: communicates state changes with a client. Send a
wss
request to/ws
to open a webscocket. - 𧰠Many serialization options: to JSON, JSON-AD, and various Linked Data / RDF formats (RDF/XML, N-Triples / Turtle / JSON-LD).
- ð Full-text search with fuzzy search and various operators, often <3ms responses.
- ð Pagination, sorting and filtering using Atomic Collections
- ð Authorization (read / write permissions) and Hierarchical structures powered by Atomic Hierarchy
- ðē Invite and sharing system with Atomic Invites
Powered by Rust, atomic-lib, actix-web, sled, tantivy and more.
README Contents
- When should you use this
- When not to use this
- Installation & getting started
- Initial setup and configuration
- Usage
- Testing
- Performance benchmarking
- FAQ & Troubleshooting
When should you use this
- You want to make (high-value) datasets as easily accessible as possible
- You want to specify and share a common vocabulary / ontology / schema for some specific domain or dataset. Example classes here.
- You want to use and share linked data, but don't want to deal with most of the complexities of RDF, SPARQL, Triple Stores, Named Graphs and Blank Nodes.
- You are interested in re-decentralizing the web or want want to work with tech that improves data ownership and interoperability.
- You like living on the edge (this application is not production ready)
When not to use this
- If you need stability or reliability, look further (for now).
- You're dealing with non-public / private data. As of now, this server does not have methods to prevent access to content.
- Complex query requirements. Check out NEO4j, Apache Jena or maybe TerminusDB.
Installation & getting started
You can run atomic-server
in four ways:
- From a published binary (probably the quickest)
- Using docker
- Using Cargo from crates.io:
cargo install atomic-server
- Manually from source
When you're running atomic-server
, go to Initial setup and configuration
Run as binary
After installing from cargo install atomic-server
or as a binary, run atomic-server
.
Run atomic-server --help
to learn more about the available commands, such as export
, import
and various flags.
Run using docker
The dockerfile
is located in the project root, above this server
folder.
- Run:
docker run -p 80:80 -p 443:443 -v atomic-storage:/atomic-storage joepmeneer/atomic-server
- If you want to update, run
docker pull joepmeneer/atomic-server
and docker should fetch the latest version.
Install from source
# Clone this repoo
# Optional, but recommended: Create a new .env using the template.
# Run the server. It creates a store in ~/.config/atomic/db by default
# Or tun the extra-cool desktop version with a presence in your app tray
# If you don't need HTTPS (or don't have OpenSSL available on your device)
Troubleshooting compiling from source:
# If pkg-config or libssl-dev is not installed, make sure to install them
Initial setup and configuration
- You can configure the server by passing arguments (see
atomic-server --help
), or by setting ENV variables. - The server loads the
.env
from the current path by default. Create a.env
file from the default template in your current directory withatomic-server setup-env
. - After running the server, check the logs and take note of the
Agent Subject
andPrivate key
. You should use these in theatomic-cli
and atomic-data-browser clients for authorization. - A directory is made:
~/.config/atomic
, which stores your newly created Agent keys, your data, the HTTPS certificates and a folder for public static files. - Visit
http://localhost:9883/setup
to register your first (admin) user. You can use an existing Agent, or create a new one.
HTTPS Setup
You'll probably want to make your Atomic Data available through HTTPS. You can use the embedded HTTPS / TLS setup powered by LetsEncrypt, acme_lib and rustls.
You can do this by passing these flags:
Run the server: atomic-server --https --email some@example.com --domain example.com
.
You can also set these things using a .env
or by setting them some other way.
Make sure the server is accessible at ATOMIC_DOMAIN
at port 80, because Let's Encrypt will send an HTTP request to this server's /.well-known
directory to check the keys.
The default Ports are 9883
for HTTP, and 9884
for HTTPS.
If you're running the server publicly, set these to 80
and 433
: atomic-server --https --port 80 --port-https 433
.
It will now initialize the certificate.
Read the logs, watch for errors.
HTTPS certificates are automatically renewed when the server is restarted, and the certs are 4 weeks or older.
They are stored in your .config/atomic/
dir.
Usage
There are three ways to interact with this server:
- GUI: Use the
atomic-data-browser
JS frontend by visitinglocalhost
. - API: Check out ./example_requests.http for various HTTP requests to the server. Also, read the docs. You can also try the react boilerplate to build your own front-end app using @tomic/lib and @tomic/react.
- CLI: The
atomic-cli
terminal app
Use atomic-cli
as client
atomic-cli
is a useful terminal tool for interacting with atomic-server
.
It makes it easy to query and edit Atomic Data from the command line.
Check it out.
API
You can fetch individual items by sending a GET request to their URL.
# Fetch as JSON-AD (de facto standard for Atomic Data)
# Fetch as JSON-LD
# Fetch as JSON
# Fetch as Turtle / N3
Check out ./example_requests.http for more things that you can do. Also, read the Atomic Data Docs to learn more about Collections, Commits, JSON-AD and other concepts used here.
Testing
# This also makes sure that cli and server work, plus it test the db feature
Performance benchmarking
# Install drill
FAQ & Troubleshooting
Can / should I create backups?
Yes! You should. Especially before installing a newer Atomic-Server version, it might be imcompatible with the previous database model and could corrupt the database.
Run atomic-server export
to create a backup in your ~/.config/atomic/backups
folder.
Import them using atomic-server import -p ~/.config/atomic/backups/${date}.json
.
I lost the key / secret to my Root Agent, and the /setup
invite is no longer usable! What now?
You can run atomic-server --initialize
to recreate the /setup
invite. It will be reset to 1
usage.
How do I migrate my data to a new domain?
There are no helper functions for this, but you could atomic-server export
your JSON-AD, and find + replace your old domain with the new one.
This could especially be helpful if you're running at localhost
and want to move to a live server.
How do I reset my database?
Remove the db
folder in your atomic
config: rm -rf ~/.config/atomic/db
.
How do I make my data private, yet available online?
This is not yet possible. See #13.
Collections are empty / TPF is not working / search is empty
You might have a problem with your indexes.
Try rebuilding the indexes using atomic-server --rebuild-index
.
I get a failed to retrieve
error when opening
Try re-initializing atomic server atomic-server --initialize
.