Reading cursive handwriting by alignment of letter prototypes

We describe a new approach to the visual recognition of cursive handwriting. An effort is made to attain human- like performance by using a method based on pictorial alignment and on a model of the process of handwriting. The alignment approach permits recognition of character instances that appear embedded in connected strings. A system embodying this approach has been implemented and tested on five different word sets. The performance was stable both across words and across writers. The system exhibited a substantial ability to interpret cursive connected strings without recourse to lexical knowledge. The interpretation of cursive connected handwriting is considerably more difficult than the reading of printed text. This difficulty may be the reason for the relative lack of attention to the problem of reading cursive script within the field of computational vision. The present article describes progress made toward understanding and solving this problem. We identify and discuss two main causes of the diffi- culties associated with handwriting recognition: uncer- tainty of segmentation of words into characters and var- iability of character shapes. We then extend a method that has been recently proposed for general object rec- ognition, the alignment of pictorial descriptions, to handwriting recognition. A system based on the align- ment of letter prototypes has been implemented and tested. Our results may indicate that the achievement of human-like performance in reading cursive hand- writing is within the reach of the state of the art in com- puter vision.