Struct StopWords

pub struct StopWords { /* private fields */ }

Contains words to be filtered out from the resulting set.

The list is used to mark potentially meaningless tokens and generally based on the language given as input.

Tokens with fewer than three characters are also considered a stopword.

Implementations

impl StopWords

pub fn custom(lowercased: HashSet<String>) -> Self

Use the passed set of lowercased strings as stopwords.

pub fn predefined(lang_iso_639_2: &str) -> Option<Self>

Load a predefined list of stopwords for the language given as argument.

The argument is a ISO 639 two-letter code. See the isolang crate.

Methods from Deref<Target = HashSet<String>>

pub fn capacity(&self) -> usize

Returns the number of elements the set can hold without reallocating.

Examples

use hashbrown::HashSet;
let set: HashSet<i32> = HashSet::with_capacity(100);
assert!(set.capacity() >= 100);

pub fn iter(&self) -> Iter<'_, T>

An iterator visiting all elements in arbitrary order. The iterator element type is &'a T.

Examples

use hashbrown::HashSet;
let mut set = HashSet::new();
set.insert("a");
set.insert("b");

// Will print in an arbitrary order.
for x in set.iter() {
    println!("{}", x);
}

pub fn len(&self) -> usize

Returns the number of elements in the set.

Examples

use hashbrown::HashSet;

let mut v = HashSet::new();
assert_eq!(v.len(), 0);
v.insert(1);
assert_eq!(v.len(), 1);

pub fn is_empty(&self) -> bool

Returns true if the set contains no elements.

Examples

use hashbrown::HashSet;

let mut v = HashSet::new();
assert!(v.is_empty());
v.insert(1);
assert!(!v.is_empty());

pub fn drain(&mut self) -> Drain<'_, T, A>

Clears the set, returning all elements in an iterator.

Examples

use hashbrown::HashSet;

let mut set: HashSet<_> = [1, 2, 3].into_iter().collect();
assert!(!set.is_empty());

// print 1, 2, 3 in an arbitrary order
for i in set.drain() {
    println!("{}", i);
}

assert!(set.is_empty());

pub fn retain<F>(&mut self, f: F)

where F: FnMut(&T) -> bool,

Retains only the elements specified by the predicate.

In other words, remove all elements e such that f(&e) returns false.

Examples

use hashbrown::HashSet;

let xs = [1,2,3,4,5,6];
let mut set: HashSet<i32> = xs.into_iter().collect();
set.retain(|&k| k % 2 == 0);
assert_eq!(set.len(), 3);

pub fn extract_if<F>(&mut self, f: F) -> ExtractIf<'_, T, F, A>

where F: FnMut(&T) -> bool,

Drains elements which are true under the given predicate, and returns an iterator over the removed items.

In other words, move all elements e such that f(&e) returns true out into another iterator.

If the returned ExtractIf is not exhausted, e.g. because it is dropped without iterating or the iteration short-circuits, then the remaining elements will be retained. Use retain() with a negated predicate if you do not need the returned iterator.

Examples

use hashbrown::HashSet;

let mut set: HashSet<i32> = (0..8).collect();
let drained: HashSet<i32> = set.extract_if(|v| v % 2 == 0).collect();

let mut evens = drained.into_iter().collect::<Vec<_>>();
let mut odds = set.into_iter().collect::<Vec<_>>();
evens.sort();
odds.sort();

assert_eq!(evens, vec![0, 2, 4, 6]);
assert_eq!(odds, vec![1, 3, 5, 7]);

pub fn clear(&mut self)

Clears the set, removing all values.

Examples

use hashbrown::HashSet;

let mut v = HashSet::new();
v.insert(1);
v.clear();
assert!(v.is_empty());

pub fn allocator(&self) -> &A

Returns a reference to the underlying allocator.

pub fn hasher(&self) -> &S

Returns a reference to the set’s BuildHasher.

Examples

use hashbrown::HashSet;
use hashbrown::DefaultHashBuilder;

let hasher = DefaultHashBuilder::default();
let set: HashSet<i32> = HashSet::with_hasher(hasher);
let hasher: &DefaultHashBuilder = set.hasher();

pub fn reserve(&mut self, additional: usize)

Reserves capacity for at least additional more elements to be inserted in the HashSet. The collection may reserve more space to avoid frequent reallocations.

Panics

Panics if the new capacity exceeds isize::MAX bytes and abort the program in case of allocation error. Use try_reserve instead if you want to handle memory allocation failure.

Examples

use hashbrown::HashSet;
let mut set: HashSet<i32> = HashSet::new();
set.reserve(10);
assert!(set.capacity() >= 10);

pub fn try_reserve(&mut self, additional: usize) -> Result<(), TryReserveError>

Tries to reserve capacity for at least additional more elements to be inserted in the given HashSet<K,V>. The collection may reserve more space to avoid frequent reallocations.

Errors

If the capacity overflows, or the allocator reports a failure, then an error is returned.

Examples

use hashbrown::HashSet;
let mut set: HashSet<i32> = HashSet::new();
set.try_reserve(10).expect("why is the test harness OOMing on 10 bytes?");

pub fn shrink_to_fit(&mut self)

Shrinks the capacity of the set as much as possible. It will drop down as much as possible while maintaining the internal rules and possibly leaving some space in accordance with the resize policy.

Examples

use hashbrown::HashSet;

let mut set = HashSet::with_capacity(100);
set.insert(1);
set.insert(2);
assert!(set.capacity() >= 100);
set.shrink_to_fit();
assert!(set.capacity() >= 2);

pub fn shrink_to(&mut self, min_capacity: usize)

Shrinks the capacity of the set with a lower limit. It will drop down no lower than the supplied limit while maintaining the internal rules and possibly leaving some space in accordance with the resize policy.

Panics if the current capacity is smaller than the supplied minimum capacity.

Examples

use hashbrown::HashSet;

let mut set = HashSet::with_capacity(100);
set.insert(1);
set.insert(2);
assert!(set.capacity() >= 100);
set.shrink_to(10);
assert!(set.capacity() >= 10);
set.shrink_to(0);
assert!(set.capacity() >= 2);

pub fn difference<'a>( &'a self, other: &'a HashSet<T, S, A>, ) -> Difference<'a, T, S, A>

Visits the values representing the difference, i.e., the values that are in self but not in other.

Examples

use hashbrown::HashSet;
let a: HashSet<_> = [1, 2, 3].into_iter().collect();
let b: HashSet<_> = [4, 2, 3, 4].into_iter().collect();

// Can be seen as `a - b`.
for x in a.difference(&b) {
    println!("{}", x); // Print 1
}

let diff: HashSet<_> = a.difference(&b).collect();
assert_eq!(diff, [1].iter().collect());

// Note that difference is not symmetric,
// and `b - a` means something else:
let diff: HashSet<_> = b.difference(&a).collect();
assert_eq!(diff, [4].iter().collect());

pub fn symmetric_difference<'a>( &'a self, other: &'a HashSet<T, S, A>, ) -> SymmetricDifference<'a, T, S, A>

Visits the values representing the symmetric difference, i.e., the values that are in self or in other but not in both.

Examples

use hashbrown::HashSet;
let a: HashSet<_> = [1, 2, 3].into_iter().collect();
let b: HashSet<_> = [4, 2, 3, 4].into_iter().collect();

// Print 1, 4 in arbitrary order.
for x in a.symmetric_difference(&b) {
    println!("{}", x);
}

let diff1: HashSet<_> = a.symmetric_difference(&b).collect();
let diff2: HashSet<_> = b.symmetric_difference(&a).collect();

assert_eq!(diff1, diff2);
assert_eq!(diff1, [1, 4].iter().collect());

pub fn intersection<'a>( &'a self, other: &'a HashSet<T, S, A>, ) -> Intersection<'a, T, S, A>

Visits the values representing the intersection, i.e., the values that are both in self and other.

Examples

use hashbrown::HashSet;
let a: HashSet<_> = [1, 2, 3].into_iter().collect();
let b: HashSet<_> = [4, 2, 3, 4].into_iter().collect();

// Print 2, 3 in arbitrary order.
for x in a.intersection(&b) {
    println!("{}", x);
}

let intersection: HashSet<_> = a.intersection(&b).collect();
assert_eq!(intersection, [2, 3].iter().collect());

pub fn union<'a>(&'a self, other: &'a HashSet<T, S, A>) -> Union<'a, T, S, A>

Visits the values representing the union, i.e., all the values in self or other, without duplicates.

Examples

use hashbrown::HashSet;
let a: HashSet<_> = [1, 2, 3].into_iter().collect();
let b: HashSet<_> = [4, 2, 3, 4].into_iter().collect();

// Print 1, 2, 3, 4 in arbitrary order.
for x in a.union(&b) {
    println!("{}", x);
}

let union: HashSet<_> = a.union(&b).collect();
assert_eq!(union, [1, 2, 3, 4].iter().collect());

pub fn contains<Q>(&self, value: &Q) -> bool

where Q: Hash + Equivalent<T> + ?Sized,

Returns true if the set contains a value.

The value may be any borrowed form of the set’s value type, but Hash and Eq on the borrowed form must match those for the value type.

Examples

use hashbrown::HashSet;

let set: HashSet<_> = [1, 2, 3].into_iter().collect();
assert_eq!(set.contains(&1), true);
assert_eq!(set.contains(&4), false);

pub fn get<Q>(&self, value: &Q) -> Option<&T>

where Q: Hash + Equivalent<T> + ?Sized,

Returns a reference to the value in the set, if any, that is equal to the given value.

The value may be any borrowed form of the set’s value type, but Hash and Eq on the borrowed form must match those for the value type.

Examples

use hashbrown::HashSet;

let set: HashSet<_> = [1, 2, 3].into_iter().collect();
assert_eq!(set.get(&2), Some(&2));
assert_eq!(set.get(&4), None);

pub fn get_or_insert(&mut self, value: T) -> &T

Inserts the given value into the set if it is not present, then returns a reference to the value in the set.

Examples

use hashbrown::HashSet;

let mut set: HashSet<_> = [1, 2, 3].into_iter().collect();
assert_eq!(set.len(), 3);
assert_eq!(set.get_or_insert(2), &2);
assert_eq!(set.get_or_insert(100), &100);
assert_eq!(set.len(), 4); // 100 was inserted

pub fn get_or_insert_with<Q, F>(&mut self, value: &Q, f: F) -> &T

where Q: Hash + Equivalent<T> + ?Sized, F: FnOnce(&Q) -> T,

Inserts a value computed from f into the set if the given value is not present, then returns a reference to the value in the set.

Examples

use hashbrown::HashSet;

let mut set: HashSet<String> = ["cat", "dog", "horse"]
    .iter().map(|&pet| pet.to_owned()).collect();

assert_eq!(set.len(), 3);
for &pet in &["cat", "dog", "fish"] {
    let value = set.get_or_insert_with(pet, str::to_owned);
    assert_eq!(value, pet);
}
assert_eq!(set.len(), 4); // a new "fish" was inserted

The following example will panic because the new value doesn’t match.

let mut set = hashbrown::HashSet::new();
set.get_or_insert_with("rust", |_| String::new());

pub fn entry(&mut self, value: T) -> Entry<'_, T, S, A>

Gets the given value’s corresponding entry in the set for in-place manipulation.

Examples

use hashbrown::HashSet;
use hashbrown::hash_set::Entry::*;

let mut singles = HashSet::new();
let mut dupes = HashSet::new();

for ch in "a short treatise on fungi".chars() {
    if let Vacant(dupe_entry) = dupes.entry(ch) {
        // We haven't already seen a duplicate, so
        // check if we've at least seen it once.
        match singles.entry(ch) {
            Vacant(single_entry) => {
                // We found a new character for the first time.
                single_entry.insert();
            }
            Occupied(single_entry) => {
                // We've already seen this once, "move" it to dupes.
                single_entry.remove();
                dupe_entry.insert();
            }
        }
    }
}

assert!(!singles.contains(&'t') && dupes.contains(&'t'));
assert!(singles.contains(&'u') && !dupes.contains(&'u'));
assert!(!singles.contains(&'v') && !dupes.contains(&'v'));

pub fn is_disjoint(&self, other: &HashSet<T, S, A>) -> bool

Returns true if self has no elements in common with other. This is equivalent to checking for an empty intersection.

Examples

use hashbrown::HashSet;

let a: HashSet<_> = [1, 2, 3].into_iter().collect();
let mut b = HashSet::new();

assert_eq!(a.is_disjoint(&b), true);
b.insert(4);
assert_eq!(a.is_disjoint(&b), true);
b.insert(1);
assert_eq!(a.is_disjoint(&b), false);

pub fn is_subset(&self, other: &HashSet<T, S, A>) -> bool

Returns true if the set is a subset of another, i.e., other contains at least all the values in self.

Examples

use hashbrown::HashSet;

let sup: HashSet<_> = [1, 2, 3].into_iter().collect();
let mut set = HashSet::new();

assert_eq!(set.is_subset(&sup), true);
set.insert(2);
assert_eq!(set.is_subset(&sup), true);
set.insert(4);
assert_eq!(set.is_subset(&sup), false);

pub fn is_superset(&self, other: &HashSet<T, S, A>) -> bool

Returns true if the set is a superset of another, i.e., self contains at least all the values in other.

Examples

use hashbrown::HashSet;

let sub: HashSet<_> = [1, 2].into_iter().collect();
let mut set = HashSet::new();

assert_eq!(set.is_superset(&sub), false);

set.insert(0);
set.insert(1);
assert_eq!(set.is_superset(&sub), false);

set.insert(2);
assert_eq!(set.is_superset(&sub), true);

pub fn insert(&mut self, value: T) -> bool

Adds a value to the set.

If the set did not have this value present, true is returned.

If the set did have this value present, false is returned.

Examples

use hashbrown::HashSet;

let mut set = HashSet::new();

assert_eq!(set.insert(2), true);
assert_eq!(set.insert(2), false);
assert_eq!(set.len(), 1);

pub unsafe fn insert_unique_unchecked(&mut self, value: T) -> &T

Insert a value the set without checking if the value already exists in the set.

This operation is faster than regular insert, because it does not perform lookup before insertion.

This operation is useful during initial population of the set. For example, when constructing a set from another set, we know that values are unique.

Safety

This operation is safe if a value does not exist in the set.

However, if a value exists in the set already, the behavior is unspecified: this operation may panic, loop forever, or any following operation with the set may panic, loop forever or return arbitrary result.

That said, this operation (and following operations) are guaranteed to not violate memory safety.

However this operation is still unsafe because the resulting HashSet may be passed to unsafe code which does expect the set to behave correctly, and would cause unsoundness as a result.

pub fn replace(&mut self, value: T) -> Option<T>

Adds a value to the set, replacing the existing value, if any, that is equal to the given one. Returns the replaced value.

Examples

use hashbrown::HashSet;

let mut set = HashSet::new();
set.insert(Vec::<i32>::new());

assert_eq!(set.get(&[][..]).unwrap().capacity(), 0);
set.replace(Vec::with_capacity(10));
assert_eq!(set.get(&[][..]).unwrap().capacity(), 10);

pub fn remove<Q>(&mut self, value: &Q) -> bool

where Q: Hash + Equivalent<T> + ?Sized,

Removes a value from the set. Returns whether the value was present in the set.

The value may be any borrowed form of the set’s value type, but Hash and Eq on the borrowed form must match those for the value type.

Examples

use hashbrown::HashSet;

let mut set = HashSet::new();

set.insert(2);
assert_eq!(set.remove(&2), true);
assert_eq!(set.remove(&2), false);

pub fn take<Q>(&mut self, value: &Q) -> Option<T>

where Q: Hash + Equivalent<T> + ?Sized,

Removes and returns the value in the set, if any, that is equal to the given one.

The value may be any borrowed form of the set’s value type, but Hash and Eq on the borrowed form must match those for the value type.

Examples

use hashbrown::HashSet;

let mut set: HashSet<_> = [1, 2, 3].into_iter().collect();
assert_eq!(set.take(&2), Some(2));
assert_eq!(set.take(&2), None);

pub fn allocation_size(&self) -> usize

Returns the total amount of memory allocated internally by the hash set, in bytes.

The returned number is informational only. It is intended to be primarily used for memory profiling.

Trait Implementations

impl<T> AsRef<T> for StopWords

where T: ?Sized, <StopWords as Deref>::Target: AsRef<T>,

fn as_ref(&self) -> &T

Converts this type into a shared reference of the (usually inferred) input type.

impl Clone for StopWords

fn clone(&self) -> StopWords

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for StopWords

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for StopWords

fn default() -> StopWords

Returns the “default value” for a type.

impl Deref for StopWords

type Target = HashSet<String>

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl DerefMut for StopWords

fn deref_mut(&mut self) -> &mut Self::Target

Mutably dereferences the value.

impl From<HashSet<String>> for StopWords

fn from(lowercased: HashSet<String>) -> Self

Converts to this type from the input type.

impl From<HashSet<String>> for StopWords

fn from(lowercased: HashSet<String>) -> Self

Converts to this type from the input type.

impl PartialEq for StopWords

fn eq(&self, other: &StopWords) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for StopWords

impl StructuralPartialEq for StopWords