What happens between the image and the text
Classical OCR — including Tesseract, the open-source engine behind a large share of the world's scanning software — runs a pipeline, and each stage can fail in ways that show up much later.
First the image is binarised: every pixel is decided to be either ink or background. This single step causes most catastrophic failures, because it is where uneven lighting does its damage. A photograph with a shadow across one half will threshold that half to solid black, and no amount of cleverness downstream recovers text from a black rectangle.
Then the page is deskewed and analysed for layout — finding blocks, columns, lines and finally individual character shapes. Each shape is classified, and the results are checked against a language model that knows which letter sequences are plausible. That final stage is why OCR output often contains real words that are the wrong words: the model confidently repaired an ambiguous shape into something that fits the language.
Why screenshots are easy and photographs are hard
A screenshot is the ideal input. The text is perfectly sharp, perfectly aligned, evenly lit, high contrast, and rendered rather than captured. Accuracy on clean screenshots is routinely near-perfect, and if you have a choice between screenshotting something and photographing it, that choice matters more than any setting.
A photograph of a page introduces every problem at once: perspective distortion from not being exactly parallel, uneven lighting from the room, a shadow from your own head or phone, focus that is slightly off, motion blur, and page curvature if the document is a book. Each is individually survivable and they compound.
Resolution is the most commonly misunderstood factor. OCR wants roughly 300 DPI for body text, meaning a lowercase letter should be around 20-30 pixels tall. Below about 10 pixels tall, accuracy collapses no matter how clean the image is — there is simply not enough information to distinguish an 'e' from a 'c'. This is why zooming in before screenshotting small text beats any post-processing.
- Lowercase letters should be at least 20 px tall; below 10 px is hopeless.
- Even lighting matters more than bright lighting.
- Shoot parallel to the page, not at an angle.
- Sharp focus beats high megapixels.
The things that destroy accuracy
Some inputs are simply outside what classical OCR does well, and knowing them saves time spent assuming you configured something wrong.
Handwriting is the big one. Tesseract and its relatives are trained on printed type; cursive handwriting is a fundamentally different recognition problem that needs models built for it. Expect poor results and do not conclude the tool is broken.
Text over images or textured backgrounds breaks binarisation, because there is no clean ink-versus-background split to find. Decorative and script fonts are frequently misread. Very low contrast — grey text on a slightly lighter grey, common in modern UI design — thresholds unpredictably. Tables and multi-column layouts often produce correct words in the wrong reading order, because layout analysis decided the structure incorrectly. And rotation past a few degrees defeats line detection, though deskewing handles small angles automatically.
Choosing the language, and why it matters more than expected
OCR engines use a language model to resolve ambiguous shapes, so telling the engine the wrong language actively harms accuracy rather than merely failing to help. Running Portuguese text through an English model produces English-looking words that were never on the page, because every ambiguous character is repaired towards English.
This is also why accented characters are frequently dropped when the language is set wrong: the model has no expectation of them and treats the accent as noise.
For documents mixing languages, most engines accept several language packs at once, at some cost to accuracy in each. If one language dominates, choosing that single language usually beats a combined model.
Preparing an image so OCR can succeed
The fixes are almost all upstream. In rough order of effect: get more resolution, get the lighting even, get the page square to the camera, and raise contrast to a genuine black-on-white.
Cropping to just the text region helps twice — it removes background that could confuse layout analysis, and it removes anything that would drag automatic thresholding in the wrong direction. Converting to greyscale before OCR is essentially free and occasionally helps. Straightening a photographed page matters more than people expect, because line detection assumes horizontal text.
One counterintuitive point: do not sharpen aggressively. Sharpening creates haloes around strokes, and binarisation reads haloes as ink, which thickens characters and merges adjacent ones. Mild is fine; heavy sharpening makes things worse.
- Crop tightly to the text before recognising.
- Straighten the page; small skew is handled, large skew is not.
- Increase contrast towards true black on true white.
- Avoid heavy sharpening — it merges characters.
Where the recognition runs
OCR is applied disproportionately to documents people would not want to hand over: contracts, payslips, medical letters, ID documents, and screenshots of private conversations. That makes the location of the processing a real question rather than a technicality.
Tesseract compiles to WebAssembly and runs entirely in a browser tab, downloading the language pack once and caching it. That is how our image-to-text tool works, which is why it functions offline after first use — and why the documents never leave the device.
The honest trade-off: cloud OCR services, particularly the ones using modern transformer-based models, are meaningfully more accurate on hard inputs like handwriting and photographed receipts. If you have a difficult image and it contains nothing sensitive, they will do better. For anything private, local recognition on a well-prepared image is usually more than good enough, and the preparation matters more than the engine.