Module opencv::text

Scene Text Detection and Recognition

The opencv_text module provides different algorithms for text detection and recognition in natural scene images.

Scene Text Detection

Class-specific Extremal Regions for Scene Text Detection

The scene text detection algorithm described below has been initially proposed by Lukás Neumann & Jiri Matas Neumann11. The main idea behind Class-specific Extremal Regions is similar to the MSER in that suitable Extremal Regions (ERs) are selected from the whole component tree of the image. However, this technique differs from MSER in that selection of suitable ERs is done by a sequential classifier trained for character detection, i.e. dropping the stability requirement of MSERs and selecting class-specific (not necessarily stable) regions.

The component tree of an image is constructed by thresholding by an increasing value step-by-step from 0 to 255 and then linking the obtained connected components from successive levels in a hierarchy by their inclusion relation:

The component tree may contain a huge number of regions even for a very simple image as shown in the previous image. This number can easily reach the order of 1 x 10^6 regions for an average 1 Megapixel image. In order to efficiently select suitable regions among all the ERs the algorithm make use of a sequential classifier with two differentiated stages.

In the first stage incrementally computable descriptors (area, perimeter, bounding box, and Euler’s number) are computed (in O(1)) for each region r and used as features for a classifier which estimates the class-conditional probability p(r|character). Only the ERs which correspond to local maximum of the probability p(r|character) are selected (if their probability is above a global limit p_min and the difference between local maximum and local minimum is greater than a delta_min value).

In the second stage, the ERs that passed the first stage are classified into character and non-character classes using more informative but also more computationally expensive features. (Hole area ratio, convex hull ratio, and the number of outer boundary inflexion points).

This ER filtering process is done in different single-channel projections of the input image in order to increase the character localization recall.

After the ER filtering is done on each input channel, character candidates must be grouped in high-level text blocks (i.e. words, text lines, paragraphs, …). The opencv_text module implements two different grouping algorithms: the Exhaustive Search algorithm proposed in Neumann12 for grouping horizontally aligned text, and the method proposed by Lluis Gomez and Dimosthenis Karatzas in Gomez13 Gomez14 for grouping arbitrary oriented text (see erGrouping).

To see the text detector at work, have a look at the textdetection demo: https://github.com/opencv/opencv_contrib/blob/master/modules/text/samples/textdetection.cpp

Scene Text Recognition

Modules

• prelude

Structs

• ERStat
  The ERStat structure represents a class-specific Extremal Region (ER).
• OCRBeamSearchDecoder
  OCRBeamSearchDecoder class provides an interface for OCR using Beam Search algorithm.
• OCRBeamSearchDecoder_ClassifierCallback
  Callback with the character classifier is made a class.
• OCRHMMDecoder
  OCRHMMDecoder class provides an interface for OCR using Hidden Markov Models.
• OCRHMMDecoder_ClassifierCallback
  Callback with the character classifier is made a class.

Enums

• classifier_type
• decoder_mode
• erGrouping_Modes
  text::erGrouping operation modes
• ocr_engine_mode
  Tesseract.OcrEngineMode Enumeration
• page_seg_mode
  Tesseract.PageSegMode Enumeration

Constants

• ERFILTER_NM_IHSGrad
• ERFILTER_NM_RGBLGrad
• ERGROUPING_ORIENTATION_ANY
  Text grouping method proposed in Gomez13 Gomez14 for grouping arbitrary oriented text. Regions are agglomerated by Single Linkage Clustering in a weighted feature space that combines proximity (x,y coordinates) and similarity measures (color, size, gradient magnitude, stroke width, etc.). SLC provides a dendrogram where each node represents a text group hypothesis. Then the algorithm finds the branches corresponding to text groups by traversing this dendrogram with a stopping rule that combines the output of a rotation invariant text group classifier and a probabilistic measure for hierarchical clustering validity assessment.
• ERGROUPING_ORIENTATION_HORIZ
  Exhaustive Search algorithm proposed in Neumann11 for grouping horizontally aligned text. The algorithm models a verification function for all the possible ER sequences. The verification fuction for ER pairs consists in a set of threshold-based pairwise rules which compare measurements of two regions (height ratio, centroid angle, and region distance). The verification function for ER triplets creates a word text line estimate using Least Median-Squares fitting for a given triplet and then verifies that the estimate is valid (based on thresholds created during training). Verification functions for sequences larger than 3 are approximated by verifying that the text line parameters of all (sub)sequences of length 3 are consistent.
• OCR_CNN_CLASSIFIER
• OCR_DECODER_VITERBI
• OCR_KNN_CLASSIFIER
• OCR_LEVEL_TEXTLINE
• OCR_LEVEL_WORD
• OEM_CUBE_ONLY
• OEM_DEFAULT
• OEM_TESSERACT_CUBE_COMBINED
• OEM_TESSERACT_ONLY
• PSM_AUTO
• PSM_AUTO_ONLY
• PSM_AUTO_OSD
• PSM_CIRCLE_WORD
• PSM_OSD_ONLY
• PSM_SINGLE_BLOCK
• PSM_SINGLE_BLOCK_VERT_TEXT
• PSM_SINGLE_CHAR
• PSM_SINGLE_COLUMN
• PSM_SINGLE_LINE
• PSM_SINGLE_WORD

Traits

• BaseOCR
• BaseOCRConst
  Constant methods for crate::text::BaseOCR
• ERFilter
  Base class for 1st and 2nd stages of Neumann and Matas scene text detection algorithm Neumann12. :
• ERFilterConst
  Constant methods for crate::text::ERFilter
• ERFilter_Callback
  Callback with the classifier is made a class.
• ERFilter_CallbackConst
  Constant methods for crate::text::ERFilter_Callback
• ERStatTrait
  Mutable methods for crate::text::ERStat
• ERStatTraitConst
  Constant methods for crate::text::ERStat
• OCRBeamSearchDecoderTrait
  Mutable methods for crate::text::OCRBeamSearchDecoder
• OCRBeamSearchDecoderTraitConst
  Constant methods for crate::text::OCRBeamSearchDecoder
• OCRBeamSearchDecoder_ClassifierCallbackTrait
  Mutable methods for crate::text::OCRBeamSearchDecoder_ClassifierCallback
• OCRBeamSearchDecoder_ClassifierCallbackTraitConst
  Constant methods for crate::text::OCRBeamSearchDecoder_ClassifierCallback
• OCRHMMDecoderTrait
  Mutable methods for crate::text::OCRHMMDecoder
• OCRHMMDecoderTraitConst
  Constant methods for crate::text::OCRHMMDecoder
• OCRHMMDecoder_ClassifierCallbackTrait
  Mutable methods for crate::text::OCRHMMDecoder_ClassifierCallback
• OCRHMMDecoder_ClassifierCallbackTraitConst
  Constant methods for crate::text::OCRHMMDecoder_ClassifierCallback
• OCRHolisticWordRecognizer
  OCRHolisticWordRecognizer class provides the functionallity of segmented wordspotting. Given a predefined vocabulary , a DictNet is employed to select the most probable word given an input image.
• OCRHolisticWordRecognizerConst
  Constant methods for crate::text::OCRHolisticWordRecognizer
• OCRTesseract
  OCRTesseract class provides an interface with the tesseract-ocr API (v3.02.02) in C++.
• OCRTesseractConst
  Constant methods for crate::text::OCRTesseract
• TextDetector
  An abstract class providing interface for text detection algorithms
• TextDetectorCNN
  TextDetectorCNN class provides the functionallity of text bounding box detection. This class is representing to find bounding boxes of text words given an input image. This class uses OpenCV dnn module to load pre-trained model described in LiaoSBWL17. The original repository with the modified SSD Caffe version: https://github.com/MhLiao/TextBoxes. Model can be downloaded from DropBox. Modified .prototxt file with the model description can be found in opencv_contrib/modules/text/samples/textbox.prototxt.
• TextDetectorCNNConst
  Constant methods for crate::text::TextDetectorCNN
• TextDetectorConst
  Constant methods for crate::text::TextDetector

Functions

• compute_nm_channels
  Compute the different channels to be processed independently in the N&M algorithm Neumann12.
• create_er_filter_nm1
  Create an Extremal Region Filter for the 1st stage classifier of N&M algorithm Neumann12.
• create_er_filter_nm1_from_file
  Reads an Extremal Region Filter for the 1st stage classifier of N&M algorithm from the provided path e.g. /path/to/cpp/trained_classifierNM1.xml
• create_er_filter_nm2
  Create an Extremal Region Filter for the 2nd stage classifier of N&M algorithm Neumann12.
• create_er_filter_nm2_from_file
  Reads an Extremal Region Filter for the 2nd stage classifier of N&M algorithm from the provided path e.g. /path/to/cpp/trained_classifierNM2.xml
• create_ocrhmm_transitions_table
  Utility function to create a tailored language model transitions table from a given list of words (lexicon).
• create_ocrhmm_transitions_table_1
• detect_regions
• detect_regions_from_file
  Extracts text regions from image.
• detect_text_swt
  Applies the Stroke Width Transform operator followed by filtering of connected components of similar Stroke Widths to return letter candidates. It also chain them by proximity and size, saving the result in chainBBs.
• er_grouping
  Find groups of Extremal Regions that are organized as text blocks.
• er_grouping_1
  C++ default parameters
• load_classifier_nm1
  Allow to implicitly load the default classifier when creating an ERFilter object.
• load_classifier_nm2
  Allow to implicitly load the default classifier when creating an ERFilter object.
• load_ocr_beam_search_classifier_cnn
  Allow to implicitly load the default character classifier when creating an OCRBeamSearchDecoder object.
• load_ocrhmm_classifier
  Allow to implicitly load the default character classifier when creating an OCRHMMDecoder object.
• load_ocrhmm_classifier_cnnDeprecated
  Allow to implicitly load the default character classifier when creating an OCRHMMDecoder object.
• load_ocrhmm_classifier_nmDeprecated
  Allow to implicitly load the default character classifier when creating an OCRHMMDecoder object.
• mse_rs_to_er_stats
  Converts MSER contours (vector<Point>) to ERStat regions.