Struct robotxt::RobotsBuilder
source · pub struct RobotsBuilder { /* private fields */ }
Available on crate feature
builder
only.Expand description
The set of formatted user-agent
groups that can be written
in the robots.txt
compliant format.
Implementations§
source§impl RobotsBuilder
impl RobotsBuilder
sourcepub fn new() -> Self
pub fn new() -> Self
Creates a new RobotsBuilder
with default settings.
sourcepub fn header(self, header: &str) -> Self
pub fn header(self, header: &str) -> Self
Adds a global header, usually used for permissions or legal notices.
use robotxt::RobotsBuilder;
let txt = RobotsBuilder::default()
.header("Note: Stop right there!")
.group(["*"], |u| u.disallow("/"))
.group(["foobot"], |u| u.allow("/"));
sourcepub fn group<'a>(
self,
group: impl IntoIterator<Item = &'a str>,
factory: impl FnOnce(GroupBuilder) -> GroupBuilder
) -> Self
pub fn group<'a>( self, group: impl IntoIterator<Item = &'a str>, factory: impl FnOnce(GroupBuilder) -> GroupBuilder ) -> Self
Adds a new user-agent
group from the provided list of user-agents.
use robotxt::RobotsBuilder;
let txt = RobotsBuilder::default()
.group(["*"], |u| u.disallow("/"))
.group(["foobot"], |u| u.allow("/"));
sourcepub fn sitemap(self, sitemap: Url) -> Self
pub fn sitemap(self, sitemap: Url) -> Self
Adds the Sitemap
directive from the URL address.
use url::Url;
use robotxt::RobotsBuilder;
let txt = RobotsBuilder::default()
.sitemap("https://example.com/sitemap_1.xml".try_into().unwrap())
.sitemap("https://example.com/sitemap_1.xml".try_into().unwrap());
Adds a global footer, usually used for notices.
use robotxt::RobotsBuilder;
let txt = RobotsBuilder::default()
.group(["*"], |u| u.disallow("/"))
.group(["foobot"], |u| u.allow("/"))
.footer("Note: Have a nice day!");
sourcepub fn parse(&self, user_agent: &str) -> Robots
Available on crate feature parser
only.
pub fn parse(&self, user_agent: &str) -> Robots
parser
only.Parses the constructed output.
See Robots::from_bytes
.
Trait Implementations§
source§impl Clone for RobotsBuilder
impl Clone for RobotsBuilder
source§fn clone(&self) -> RobotsBuilder
fn clone(&self) -> RobotsBuilder
Returns a copy of the value. Read more
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moresource§impl Debug for RobotsBuilder
impl Debug for RobotsBuilder
source§impl Default for RobotsBuilder
impl Default for RobotsBuilder
source§fn default() -> RobotsBuilder
fn default() -> RobotsBuilder
Returns the “default value” for a type. Read more
Auto Trait Implementations§
impl RefUnwindSafe for RobotsBuilder
impl Send for RobotsBuilder
impl Sync for RobotsBuilder
impl Unpin for RobotsBuilder
impl UnwindSafe for RobotsBuilder
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more