# AntTP
## Background
Autonomi Network (a.k.a. Safe Network) is a distributed data network where both mutable and immutable data can be stored. It can
considered as a best of class web3 experience.
AntTP is an HTTP server for the Autonomi Network. This allows regular web browsers (and other apps) to retrieve data from Autonomi
without needing any client libraries, CLIs, etc.
Users can either spin up a local AntTP service or deploy one to a public environment. This enables developers to
integrate with Autonomi in a more conventional way and gives end users a conventional browsing experience.
AntTP was formally known as sn_httpd.
## Overview
AntTP is designed to allow data to be browsed over HTTP directly from the Autonomi Network. To do this, it follows similar principles
to hosting data from a regular file system.
Archives (public or tarchive) are treated as file containers. Whether these are uploaded from the `ant` CLI or from AntTP, they can be
retrieved by AntTP. If only the archive address is specified in a URL, a file listing will be shown, similar to that of a regular web
server. If an archive and a filename within it are specified, then the file will be downloaded, much like a regular web server.
Examples:
- Archive URL: http://localhost:18888/a33082163be512fb471a1cca385332b32c19917deec3989a97e100d827f97baf/
- Archive and file URL: Example archive URLs look like: http://localhost:18888/a33082163be512fb471a1cca385332b32c19917deec3989a97e100d827f97baf/public-archive.png
File addresses can also be specified directly in a URL too, if they either have no archive acting as a file container or if they just
need addressing directly. As the network address is used as the filename in these cases, an optional (friendly) filename can be appended
to the URL to make it easier for the browser to recognise the file type and/or provide a name for downloading.
Examples:
- File URL with no filename: http://localhost:18888/304b74b76536e89910262ade48020a4ab2724bdaf353f3b42de625fee2477228
- File URL with filename: http://localhost:18888/304b74b76536e89910262ade48020a4ab2724bdaf353f3b42de625fee2477228/BegBlag.mp3
Pointers and Registers can also be used as addresses. These are mutable and allow the owner of the pointer/register to link to other
addresses (like the above). With Autonomi, to change immutable data, you copy and edit it, then save it to a new address. By using
pointers/registers, the same address can be linked to the new address of the immutable data.
Pointers/registers in URLs function a bit like regular DNS on the clear net, but any data item can be addressed, not just hosts.
Examples:
- Pointer URL to archive: http://localhost:18888/8e16406561d0c460f3dbe37fef129582d6410ec7cb9d5aebdf9cbb051676624c543a315f7e857103cd71088a927c9085/
- Pointer URL to archive with filename: http://localhost:18888/8e16406561d0c460f3dbe37fef129582d6410ec7cb9d5aebdf9cbb051676624c543a315f7e857103cd71088a927c9085/imim-github.png
AntTP Bookmarks can also be used as aliases to mutable or immutable types. They only apply to the AntTP instance with the bookmarks defined
but it can be a nice way to store commonly accessed sites. In the future they may form the basis of a broader DNS style integration.
Examples:
- Bookmark URL to a pointer with a filename: http://localhost:18888/traktion-blog/imim-github.png
## Features
`AntTP` currently provides the following:
- Data retrieval from Autonomic using archives for human readable naming `/[ARCHIVE_XOR_ADDRESS]/[MY_FILE_NAME]`. Enables
regular static sites to be uploaded as an archive, with files browsed by file name. E.g.
- http://localhost:18888/91d16e58e9164bccd29a8fd8d25218a61d8253b51c26119791b2633ff4f6b309/autonomi/david-irvine-autonomi-founder.jpg
- Proxy server to allow `http://[ARCHIVE_XOR_ADDRESS]/[MY_FILE_NAME]` to be resolved. Allows
sites to pivot from a 'root' directory and a smoother user experience. E.g.
- - http://91d16e58e9164bccd29a8fd8d25218a61d8253b51c26119791b2633ff4f6b309/autonomi/david-irvine-autonomi-founder.jpg
- Routing from URLs to specific `[XOR_ADDRESS]` or `[FILE_NAME]`. Enables SPA (single page apps) such as Angular or
React to be hosted (once a routeMap is provided - see [example-config](resources/app-conf.json)
- Native integration of the `autonomi` libraries into Actix web framework. These are both written in Rust to provide
smooth integration. As Actix is core to `AntTP`, it can be extended for specific use cases easily.
For a complete list of features, please review the roadmap below.
## Run instructions
List help from binary:
`anttp --help`
```
AntTP is an HTTP server for the Autonomi Network
Usage: anttp [OPTIONS]
Options:
-l, --listen-address <LISTEN_ADDRESS>
[default: 0.0.0.0:18888]
-s, --static-file-directory <STATIC_FILE_DIRECTORY>
[default: ]
-w, --wallet-private-key <WALLET_PRIVATE_KEY>
[default: ]
-d, --download-threads <DOWNLOAD_THREADS>
[default: 8]
-a, --app-private-key <APP_PRIVATE_KEY>
[default: ]
-b, --bookmarks <BOOKMARKS>
[default: traktion-blog=8e16406561d0c460f3dbe37fef129582d6410ec7cb9d5aebdf9cbb051676624c543a315f7e857103cd71088a927c9085,imim=959c2ba5b84e1a68fedc14caaae96e97cfff19ff381127844586b2e0cdd2afdfb1687086a5668bced9f3dc35c03c9bd7,gimim=82fb48d691a65e771e2279ff56d8c5f7bc007fa386c9de95d64be52e081f01b1fdfb248095238b93db820836cc88c67a,index=b970cf40a1ba880ecc27d5495f543af387fcb014863d0286dd2b1518920df38ac311d854013de5d50b9b04b84a6da021,gindex=879d061580e6200a3f1dbfc5c87c13544fcd391dfec772033f1138a9469df35c98429ecd3acb4a9ab631ea7d5f6fae0f,cinema=953ff297c689723a59e20d6f80b67233b0c0fe17ff4cb37a2c8cfb46e276ce0e45d59c17e006e4990deaa634141e4c77]
-u, --uploads-disabled
-c, --cached-mutable-ttl <CACHED_MUTABLE_TTL>
[default: 5]
-p, --peers <PEERS>
-m, --map-cache-directory <MAP_CACHE_DIRECTORY>
[default: /tmp/anttp/cache/]
-e, --evm-network <EVM_NETWORK>
[default: evm-arbitrum-one]
--immutable-disk-cache-size <IMMUTABLE_DISK_CACHE_SIZE>
[default: 1024]
--immutable-memory-cache-size <IMMUTABLE_MEMORY_CACHE_SIZE>
[default: 32]
-i, --idle-disconnect <IDLE_DISCONNECT>
[default: 30]
-h, --help
Print help
-V, --version
Print version
```
Run binary with defaults
`anttp`
Build and run from source code:
`cargo run` OR `cargo run -- <args>`
Where arguments are:
- `-l, --listen-address` is the IP address and port to listen on.
- `-s, --static-file-directory` is a directory to host local/static files in.
- `-w, --wallet-private-key` is a secret key for a wallet used for uploads.
- `-d, --download-threads` is the number of parallel threads used for chunk downloads.
## Proxy Configuration
Using AntTP as a proxy is optional, but it improves the user experience.
Configuring AntTP as a proxy is more secure, as it blocks calls out to clear net sites. All traffic is directed to
Autonomi, ensuring no data is leaked beyond AntTP and Autonomi.
Using a proxy also enables shorter URLs, where the target web application XOR is considered the 'host' in a traditional
web sense. For some web applications, this may be a requirement, due to how they route their links.
Configuring a browser to use AntTP as a proxy is easy. Any regular web browser that has proxy settings can be used
(e.g. Firefox) or allows CLI arguments to enable them (e.g. Brave).
### Firefox Configuration
- Go to `Settings` from the burger drop down
- Type 'proxy' in the `Find in Settings` input box
- Click `Settings...` button
- Click `Manual proxy configuration`
- Enter `127.0.0.1` in the `HTTP Proxy` input box and `18888` in the `Port` input box (or whichever non-default IP/port you are using)
- Check the `Also use this proxy for HTTPS` check box
- Check the `SOCKS v5` check box
- Check the `Proxy DNS when using SOCKS v5` check box
- Then click `OK` and start browsing Autonomi over AntTP!
See the example screenshot below:
### Brave Configuration
Brave browser only uses system wide proxy settings, unless it is launched with proxy arguments:
`brave --proxy-server="127.0.0.1:18888" http://a0f6fa2b08e868060fe6e57018e3f73294821feaf3fdcf9cd636ac3d11e7e2ac/BegBlag.mp3`
(or whichever non-default IP/port you are using)
### Security
If you're running AntTP on your own, personal, machine, you can ignore the security warnings for using HTTP. All data
transmitted between your browser and AntTP will remain on your machine only.
If you use a remote AntTP, ensure you use HTTPS, as your data will be transmitted to/from that remote proxy.
## Build Instructions
### Dependencies
On Ubuntu:
Install Rust
`sudo apt-get install rustup`
Download latest stable release:
`rustup default stable`
### Linux Target
It is recommended that the MUSL target is used to prevent runtime dependency issues.
On Ubuntu:
`sudo apt-get install musl-tools`
Then add target:
`rustup target add x86_64-unknown-linux-musl`
Then build release:
`cargo build --release --target x86_64-unknown-linux-musl`
### Windows Target
On Ubuntu:
`sudo apt-get install mingw-w64`
Then add target:
`rustup target add x86_64-pc-windows-gnu`
Then build release:
`cargo build --release --target x86_64-pc-windows-gnu`
### ARM Target
On Ubuntu:
`sudo apt install gcc make gcc-arm* gcc-aarch64* binutils-arm* binutils-aarch64* pkg-config libssl-dev`
Then add target:
`rustup target add arm-unknown-linux-musleabi`
`rustup target add gcc-arm-linux-gnueabi`
Then update the environment:
`export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-gnu-gcc`
`export CC=aarch64-linux-gnu-gcc`
Then build release:
`cargo build --release --target aarch64-unknown-linux-musl`
## Archive Upload
To upload a directory to Autonomi as an archive, do the following:
- `cd your/directory`
- `ant file upload -p <directory>`
This command will return information about the uploads and summarise with something like:
`Uploading file: "./1_bYTCL7G4KbcR_Y4rd78OhA.png"
Upload completed in 5.57326318s
Successfully uploaded: ./
At address: 600d4bbc3d7f316c2fe014ca6986c6ea62200be316e34bd307ae3aa68f8e3cfc`
The 'At address' is the archive address, which you can now reference the uploaded files like:
Via a proxy (to localhost:18888):
`http://a0f6fa2b08e868060fe6e57018e3f73294821feaf3fdcf9cd636ac3d11e7e2ac/BegBlag.mp3`
Or via direct request:
`http://localhost:18888/a0f6fa2b08e868060fe6e57018e3f73294821feaf3fdcf9cd636ac3d11e7e2ac/BegBlag.mp3`
## Web Application Customisation
See [example-config](resources/app-conf.json) for customising how your website/app behaves on `AntTP`:
```
{
"routeMap": {
"": "index.html",
"blog/*": "index.html",
"blog/*/article/*": "index.html"
}
}
```
- Create an app-config.json file in the directory you intend to upload/publish to Autonomi
- Add the `routeMap` key
- Add any routes that should be mapped to a file
- Use "" as a key to serve the target file for the root URL, e.g. index.html
- Use "/blog/*" as a key to serve the target file for any URL with blog followed by a filename
- Use "/blog/*/article/*" as a key to serve the target file for any URL with a blog and article specified
- The blog/article above are not keywords. Any names can be used to suit the routing approach needed
- Upload the directory as an archive to Autonomi (see above for more details)
- `ant file upload -p <directory>`
- Browse to the archive XOR address with your browser and confirm the routing is correct
- Why add routes?
- Many modern frameworks expect all requests to be routed through a single HTML file, which then pull in Javascript
dependencies, which then handles the routing of your app components. Angular, for example, requires this sort of routing.
- If you just want an index instead of a file listing being rendered, providing a `routeMap` will also enable this.
This is handy when you want a default page/app/script to load for a URL, without needing to specify the filename too.
## Tarchive Support
The Tarchive format is a more efficient way to upload/download many smaller files (< 4 MB).
A Tarchive is simply a tar file containing the associated files with an index appended to the end.
Note that the file order is important for establishing chronologic sequence, which may be important for some applications.
Either create the tar using a specific ordering or append file(s) with `tar -rf` to define the ordering.
Tarindexer (https://github.com/devsnd/tarindexer) is used to generate the index and then the following commands can be
run to create and upload the tarchive.
```shell
cd mydirectory
tar -cf ../archive.tar *
cd ..
tarindexer.py -i archive.tar archive.tar.idx
tar -rf archive.tar archive.tar.idx
ant file upload -p archive.tar
```
Files can be added to the tarchive by adding them to the end of the tar file (to preserve chronological order), along with
an updated index:
```shell
cd mydirectory
tar -rf ../archive.tar new_data.txt new_image.png
cd ..
tarindexer.py -i archive.tar archive.tar.idx
tar -rf archive.tar archive.tar.idx
ant file upload -p archive.tar
```
## Pointer Name Resolver (PNR) [EXPERIMENTAL!]
__IMPORTANT: The resolver key will be CHANGED after initial testing! Names will be temporary until after this point.__
The Pointer Name Resolver is a simple, first come, first server resolver.
To create a name, you set the `x-data-key` header to `resolver` and then create a pointer to your chosen target pointer.
This will use the shared key chosen for PNR to store the pointer.
Anyone can use this shared key to create or update pointers, so it is important that the `counter` is set to the maximum value
once confirmation that PNR is working as expected.
`PNR Shared Key Pointer (max counter) ⇒ PNR Current User Key Pointer`
To transfer the name to another, update the target pointer to point to the other user's pointer and set the 'counter' to
the maximum value. The other user will then have full control, as prior pointers can no longer be changed.
`PNR Shared Key Pointer (max counter) ⇒ PNR Old User Key Pointer (max counter) ⇒ PNR Current User Key Pointer`
The maximum counter value that can be set via the REST API is `18,446,744,073,709,551,614`, which automatically increments
to the full u64 maximum value on create/update.
Note that each time a new owner is assigned, the chain of pointers increases in size. This will cause the resolution time to
slow. However, AntTP caches pointers, so the impact will be unnoticeable after the initial retrieval.
Additionally, chains of pointers can be used to manage (or rent) names without transferring them. Consider the following:
`PNR Shared Key Pointer (max counter) ⇒ PNR Manager User Key Pointer ⇒ PNR Consumer User Key Pointer`
Experimental! Additional resolution features may be added in the future, to allow more flexibility/control. For example,
allowing sub-names to be managed from a lookup table:
`PNR Shared Key Pointer (max counter) ⇒ PNR Current User Key Pointer => Immutable Lookup Table`
## MCP Tools API [EXPERIMENTAL!]
LLMs (AI) can now interface with AntTP via MCP tools. This allows agents to interact with Autonomi Network
to create mutable and immutable data, similar to the REST API.
The MCP tools access the same underlying code, including the caching layer to optimise performance.
To learn more about MCP, see here: https://modelcontextprotocol.io/docs/getting-started/intro
To configure your agent (Antigravity, Claude Code, etc) to use AntTP as an MCP server, specify the following URL
(when running AntTP locally with the default port): http://localhost:18888/mcp-0
Antigravity example:
```json
{
"mcpServers": {
"local-anttp": {
"serverUrl": "http://localhost:18888/mcp-0",
"headers": {
"Authorization": "Bearer unknown",
"Content-Type": "application/json"
},
"disabled": false
}
}
}
```
## PubAnt.com - Publish your Website
For more information on how to publish a website on Autonomi Network, [PubAnt.com](https://pubant.com/) is an excellent resource.
Once your site has been published, it can be accessed through a regular browser, through AntTP.
## Example site - IMIM!
A sister application for AntTP is the IMIM blog. The source code is located at [IMIM](https://github.com/traktion/i-am-immutable-client), and enabled authors
to write Markup text files and publish them on Autonomi. Using `AntTP`, these blogs can be viewed anywhere that an
instance is running.
IMIM includes examples of route maps and how Angular apps can be integrated with AntTP. It also gives a realistic example
of performance and how immutable file caching can effectively reduce latency to near zero in most cases. IMIM is also
a great place to create a blog.
If your browser is configured as an AntTP proxy, take a look here at an example blog all about Autonomi:
http://62003e683b5a792f425a75c5d7d99d06e80f7047be8de8176b7d295e510b3b4c/blog/705a5fa9b2b2ee9d1ec88f7f6cae45a9e40d4cf8ea202252c9d7e68eb6e17c8b#home
Why not take a look and start your own immutable blog today?
## Swagger UI
Developers can explore the REST API by accessing the Swagger UI. This can be found on any AntTP server instance at the `/swagger-ui/` endpoint,
e.g. http://localhost:18888/swagger-ui/ or http://anttp.antsnest.site/swagger-ui/.
There are a number of different endpoints for uploading or downloading immutable and mutable data types. To upload data, uploads need to be enabled
and a valid wallet address must be provided (see 'Run Instructions').
Developers can use the `x-cache-only` header to only upload data to the AntTP instance, instead of uploading to the Autonomi Network. This
provides a number of use cases, from quickly/cheaply testing out new web apps, to local only storage for applications.
Note that when uploading as 'cache only', the files remain on the AntTP instance and there are no charges to upload to the Autonomi Network. If
the same data is subsequently uploaded without the `x-cache-only` header, it will charge the wallet and upload the data to the network.
Developers should consider how this could easily provide preview modes for uploaded data (e.g. preview a blog before committing to the network),
local per-user configuration data (e.g. arbitrary data that only the user needs to access), etc.
## Roadmap
- [ ] Documentation
- [x] Basic README
- [x] Improved README
- [ ] Add tutorials / API details
- [ ] Link with IMIM as sample project
- [x] Files
- [x] Enable file downloads from XOR addresses
- [x] Enable file downloads from archives with friendly names
- [x] Download files directly from tarchives
- [x] Directories (archives)
- [x] Enable directory listing in HTML (default)
- [x] Enable directory listing with JSON (using `accept` header)
- [x] Enable multiple file uploads as multipart form data
- [x] Enable defaulting to index.html when present
- [x] Caching
- [x] Cache immutable archive indexes to disk to reduce lookups to Autonomi
- [x] Set response headers to encourage long term caching of XOR data
- [x] Add eTag header support to encourage long term caching of all immutable data (with/without XOR)
- [x] Proxy server
- [x] Resolve hostnames to XOR addresses for files
- [x] Resolve hostnames to XOR addresses for archives
- [x] HTTPS proxy support
- [x] Streaming downloads
- [x] Add streaming support for data requested with RANGE headers
- [x] Add streaming support for all other data requested
- [ ] Advanced Autonomi API integration
- [ ] REST API
- [x] Pointer
- [x] Scratchpad
- [x] Graph
- [x] Register
- [x] Chunk
- [x] Public Archive
- [x] Public Data
- [ ] BLS support
- [ ] Create, sign, verify
- [ ] Derived keys
- [ ] Analyze address support
- [ ] Vault support (CRUD, cost)
- [ ] Data upload cost
- [ ] Wallet support
- [x] Async command/upload queue
- [ ] Tarchive
- [x] Pointer name resolver
- [ ] gRPC API
- [ ] Pointer
- [ ] Scratchpad
- [ ] Graph
- [ ] Register
- [ ] Chunk
- [ ] Public Archive
- [ ] Public Data
- [ ] BLS support
- [ ] Create, sign, verify
- [ ] Derived keys
- [ ] Analyze address support
- [ ] Vault support (CRUD, cost)
- [ ] Data upload cost
- [ ] Wallet support
- [ ] MCP API
- [x] Pointer
- [x] Scratchpad
- [x] Graph
- [x] Register
- [x] Chunk
- [x] Public Archive
- [x] Public Data
- [ ] BLS support
- [ ] Create, sign, verify
- [ ] Derived keys
- [ ] Analyze address support
- [ ] Vault support (CRUD, cost)
- [ ] Data upload cost
- [ ] Wallet support
- [x] Async command/upload queue
- [ ] Tarchive
- [x] Pointer name resolver
- [ ] Websockets
- [ ] Stream immutable data types
- [ ] Stream changes to mutable data types
- [x] AntTP status
- [x] Async command/upload queue (listing)
- [ ] Async command/upload queue (CRUD)
- [ ] Testing
- [ ] Core unit test coverage
- [ ] Full unit test coverage
- [x] Immutable performance testing
- [ ] Mutable performance testing
- [x] Improve CLI arguments
- [x] Offline mode (requests without connected client library dependency)
- [x] Offline downloaded data
- [x] Offline uploaded data (with optional async publishing to Autonomi)
- [ ] Accounting features
- [ ] Bandwidth usage/tracking
- [ ] Payments for data uploads (i.e. for public proxies)
- [x] Access list (allow/deny controls)
- [ ] AntTP status page
- [ ] Async command/upload queue (listing)
- [ ] Async command/upload queue (CRUD)
- [x] Pointer name resolver
- [x] Create/update pointer names
- [x] Resolve pointer names
## Codebase to Tutorial
Codebase to Tutorial generated an excellent AI summary of the application and how the code fits together.
I'd encourage a read, especially if you'd like to contribute: https://code2tutorial.com/tutorial/1d641ea9-4262-4b45-a489-21eb249a406e/index.md
## Donations
If you like AntTP and want to support continued development, any tokens would be gratefully received:
ETH, ANT or other ETH based tokens: 0x2662A9d8889678cD46F483C950eDfe344456E04e