pub struct AlignedCorpus {
pub source: Vec<Vec<String>>,
pub target: Vec<Vec<String>>,
pub t_table: HashMap<(String, String), f64>,
}Expand description
A sentence-aligned bilingual corpus together with its IBM Model 1 translation table.
Fields§
§source: Vec<Vec<String>>Source sentences (tokenized)
target: Vec<Vec<String>>Target sentences (tokenized)
t_table: HashMap<(String, String), f64>Trained translation probabilities p(target | source)
Implementations§
Source§impl AlignedCorpus
impl AlignedCorpus
Sourcepub fn train(
sentence_pairs: Vec<(Vec<String>, Vec<String>)>,
n_iter: usize,
) -> Result<Self>
pub fn train( sentence_pairs: Vec<(Vec<String>, Vec<String>)>, n_iter: usize, ) -> Result<Self>
Build an AlignedCorpus by training IBM Model 1 on sentence_pairs
for n_iter EM iterations.
§Errors
Propagates errors from ibm_model1.
Sourcepub fn viterbi_align(&self, idx: usize) -> Result<Vec<AlignmentPair>>
pub fn viterbi_align(&self, idx: usize) -> Result<Vec<AlignmentPair>>
Viterbi-decode the best source→target alignment for sentence pair idx.
For each target token the source token with the highest t(tgt | src) is
chosen (including a virtual NULL source token, which produces no output pair).
§Errors
Returns TextError::InvalidInput when idx is out of range.
Trait Implementations§
Auto Trait Implementations§
impl Freeze for AlignedCorpus
impl RefUnwindSafe for AlignedCorpus
impl Send for AlignedCorpus
impl Sync for AlignedCorpus
impl Unpin for AlignedCorpus
impl UnsafeUnpin for AlignedCorpus
impl UnwindSafe for AlignedCorpus
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> Pointable for T
impl<T> Pointable for T
Source§impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
Source§fn to_subset(&self) -> Option<SS>
fn to_subset(&self) -> Option<SS>
The inverse inclusion map: attempts to construct
self from the equivalent element of its
superset. Read moreSource§fn is_in_subset(&self) -> bool
fn is_in_subset(&self) -> bool
Checks if
self is actually part of its subset T (and can be converted to it).Source§fn to_subset_unchecked(&self) -> SS
fn to_subset_unchecked(&self) -> SS
Use with care! Same as
self.to_subset but without any property checks. Always succeeds.Source§fn from_subset(element: &SS) -> SP
fn from_subset(element: &SS) -> SP
The inclusion map: converts
self to the equivalent element of its superset.