website_crawler 0.1.0

crawl all urls on a website async & sync
1
web: ROCKET_PORT=$PORT CRAWL_URL=$CRAWL_URL ./target/release/crawler