Crate tower_no_ai 

tower-no-ai

A very simple crate to extend tower with the ability to redirect all AI-scraper bot User-Agent headers to a user-defined URL.

This can be done with something like the following (with axum):

use tower_no_ai::NoAiLayer;
use axum::routing::{get, Router};

let route = Router::new()
	.route("/", get(hello_world))
	// route them to a hetzner 10GB speed test file
	.layer(NoAiLayer::new("https://fsn1-speed.hetzner.com/10GB.bin"));

As this is built on tower, it should work perfectly with all tower-based backends.

Contributions, bug reports, and suggestions are welcome.

Dual-Licensed MIT and Apache 2.0

Structs

NoAiLayer

The tower layer which can be added to something like an axum::Router

NoAiService

The service which will redirect the requests with matching user agents

Enums

ServiceFut

The Future type that NoAiService::call produces. This has the bounds necessary to work nicely with the tower_service::Service API requirements for the associated Future type.

Statics

AI_AGENTS

The User-Agent patterns checked for and redirected if present

Functions

bot_blocking_robots_txt

Returns the contents of a basic robots.txt file that explicitly disallows all the known AI bots from accessing anything under the root of this website. Can be added with something like: