"RAG Core Study (4/26) — OCR & Layout Analysis"

Part 2 covered well-behaved digital PDFs. This part covers everything else — scans, text trapped inside images, hard tables, multi-column layouts.

Corpora almost always contain abnormal PDFs: scanned paper, broken-font Korean PDFs, tables that only humans can read, multi-column reports. Part 4 compares five tools — PaddleOCR · Tesseract · AWS Textract · Azure Document Intelligence · LayoutLMv3 — on five axes: character accuracy · layout preservation · table extraction · Korean (CJK) support · operational cost.

0. Prerequisites

  • Part 2's markdown conversion flow.
  • The distinction between images and text objects inside a PDF.

You do not need OCR internals (CTC, attention OCR). The article focuses on the tool decision table.

1. Learning Objectives

  1. State the difference between OCR and layout analysis in one line — OCR recognises characters; layout analysis recognises blocks, order, and roles.
  2. Compare the five tools on strength, weakness, license, cost.
  3. Identify the most dangerous failure for Korean (CJK) corpora.

2. 핵심 요약

OCR recognises characters; layout analysis recognises blocks of characters. Both are separate stages, and a good RAG preprocessor passes through both. PaddleOCR is the open / multilingual / free default. Tesseract is the oldest open OCR but weaker on Korean. AWS Textract and Azure DI are commercial APIs with strong table, form, and signature support. LayoutLMv3 is not an OCR — it is a model that consumes (text + bbox + image patches) for document understanding. For Korean, start with PaddleOCR and mix in Azure DI when tables matter.

3. Intuition — What the Five Tools See on the Same Page

A scanned Korean meeting-minutes page: top logo, Korean title, two-column body, a table in the middle, handwritten signatures at the bottom.

  • PaddleOCR: clean CJK character recognition; table cells identified by coordinates but markdown table conversion is weak.
  • Tesseract: has a kor model but struggles with handwriting and unusual fonts. Free and fast.
  • AWS Textract: strong tables and forms. Korean officially supported (2023+). Per-page cost.
  • Azure Document Intelligence: top-tier tables, signatures, checkboxes. Excellent Korean support. Per-page cost.
  • LayoutLMv3: not an OCR. Takes (text + bbox + image patches) and performs document understanding — classification, QA, entity extraction.
diagram-1

4. Definition — Two Separate Stages

Stage Output Example Tools
OCR Pixels → Unicode characters with per-character bbox. Tesseract, PaddleOCR
Layout Analysis bbox set → block classification (Title / Paragraph / Table / Image) + reading order LayoutLMv3, DocTR, Donut
End-to-end Document AI Pixels → (text + structure + extracted fields) in one pass AWS Textract, Azure DI

Side-by-side:

Tool OCR Layout Table Form/Sign License Korean
Tesseract 3 1 1 — Apache 2.0 2
PaddleOCR 4 2 2 — Apache 2.0 4
AWS Textract 4 3 4 3 Commercial 3
Azure DI 4 4 4 4 Commercial 4
LayoutLMv3 — 4 2 2 MIT depends on OCR

LayoutLMv3 is a model, not an OCR. Korean performance depends on whatever OCR feeds it.

5. Standard Calls

from paddleocr import PaddleOCR
ocr = PaddleOCR(lang="korean")
result = ocr.ocr("page.png")  # [[bbox, (text, score)], ...]

import pytesseract
text = pytesseract.image_to_string("page.png", lang="kor")

import boto3
client = boto3.client("textract")
resp = client.detect_document_text(Document={"Bytes": open("page.png","rb").read()})

from azure.ai.documentintelligence import DocumentIntelligenceClient
client = DocumentIntelligenceClient(endpoint, key)
poller = client.begin_analyze_document("prebuilt-layout", body=open("page.pdf","rb"))
result = poller.result()

LayoutLMv3 loads via Hugging Face Transformers and operates on OCR output, not raw pixels.

6. Walkthrough — From Pixels to Markdown

  1. Image normalisation — rotation, deskew, denoise.
  2. Text detection — bounding boxes for character regions.
  3. Text recognition — characters per box (CRNN, attention OCR).
  4. Layout classification — group bboxes into blocks (title, paragraph, table, image).
  5. Table recognition — split table regions into row/column cells.
  6. Reading-order sort — humans-style order across columns.
  7. Markdown serialisation — emit headings, paragraphs, tables, image refs.

Tools differ in how far down this pipeline they take you.

7. Variants and Cases

7.1 PaddleOCR

  • What changes: Apache 2.0 open OCR for 80+ languages; v4 Korean is the standard accuracy baseline.
  • Why it appeared: Baidu open-sourced its internal OCR, enabling multilingual processing without paying for APIs.
  • What becomes possible: free local OCR for Korean. GPU acceleration available.
  • Where it fits: Korean-primary corpora; budget-sensitive deployments.
  • Limits: weak on tables, signatures, checkboxes; pair with another tool.

7.2 Tesseract

  • What changes: the oldest open OCR with kor language pack.
  • Why it appeared: HP/Google released it as public infrastructure; easy install everywhere.
  • What becomes possible: a free OCR baseline on any OS.
  • Where it fits: simple Latin-text corpora.
  • Limits: CJK accuracy lags PaddleOCR; weak on handwriting and unusual fonts.

7.3 AWS Textract

  • What changes: AWS Document AI API — text + tables + forms + handwriting + natural-language Queries for field extraction.
  • Why it appeared: enterprise documents (invoices, contracts, forms) needed automated extraction. Korean official 2023+.
  • What becomes possible: business forms and contracts to cell-level markdown with SLA / HIPAA options.
  • Where it fits: AWS-native enterprise document workflows.
  • Limits: per-page cost; data leaves to AWS; not a fit for strict on-prem requirements.

7.4 Azure Document Intelligence (formerly Form Recognizer)

  • What changes: Microsoft Document AI with prebuilt models (layout / receipts / IDs / business cards) and custom fine-tunable models.
  • Why it appeared: Azure's OCR stack matured; Korean support is top-tier.
  • What becomes possible: tables, signatures, checkboxes, key-value pairs in one pass. prebuilt-layout is a strong RAG preprocessor baseline.
  • Where it fits: Azure environments with both Korean and table requirements.
  • Limits: per-page cost; data egress.

7.5 LayoutLMv3 — Post-OCR Understanding

  • What changes: takes (text + bbox + image patches) and learns a unified embedding for document understanding.
  • Why it appeared: text alone misses reading order and block roles. Microsoft Research (2022).
  • What becomes possible: extract total fields on receipts, signature blocks in contracts, titles vs body in reports.
  • Where it fits: post-processing OCR output for classification and field extraction in RAG KB enrichment.
  • Limits: not an OCR; Korean LayoutLMv3 support is limited and depends on the upstream OCR.

8. Limits and Failure Modes

8.1 Korean Output Shows Question Marks

  • Why intrinsic: PDF font-mapping errors plus no OCR pass. Digital PDFs without embedded fonts break text extraction.
  • Diagnosis: ? or □ in the output.
  • Mitigation: rasterise to image and OCR with PaddleOCR(lang="korean").
  • Later part: this part §7.1.

8.2 Tables Collapse to a Single Line

  • Why intrinsic: OCR alone does not know cell boundaries; cells collapse into a whitespace-delimited row.
  • Diagnosis: few | pipes; cell values appear as whitespace blobs.
  • Mitigation: use Textract or Azure DI table modes; or post-process OCR bboxes into cells.
  • Later part: §7.3, §7.4.

8.3 Multi-column Reading-order Errors

  • Why intrinsic: OCR recognises characters, not order. Sorting purely top-down, left-right scrambles two-column text.
  • Diagnosis: sentences cross columns mid-way.
  • Mitigation: LayoutLMv3, or reading-order algorithms inside Textract / Azure DI.
  • Later part: §7.4, §7.5.

8.4 Handwriting / Signatures Lost

  • Why intrinsic: most OCRs target printed text; handwriting accuracy drops below 50% on the same image.
  • Diagnosis: handwritten regions return blank or noisy strings.
  • Mitigation: Textract FORMS mode or Azure DI prebuilt-handwritten.
  • Later part: §7.3, §7.4.

8.5 Cost and Latency Stack-up

  • Why intrinsic: 10K-page OCR with PaddleOCR is tens of hours locally; Textract/Azure DI costs tens to hundreds of dollars; LayoutLMv3 post-processing adds GPU time.
  • Diagnosis: estimate one full KB build's time and dollars.
  • Mitigation: tier the corpus and use incremental OCR.
  • Later part: Part 25.

8.5 Common Pitfalls

  • "Digital PDFs don't need OCR." — Korean font-mapping fails often. §8.1.
  • "OCR captures tables automatically." — Often not; tables need a dedicated step. §8.2.
  • "One tool fits all documents." — Cost and accuracy both lose; tier the corpus.
  • "One OCR pass and we are done." — Tables, handwriting, multi-column may need separate models.
  • "LayoutLMv3 is an OCR." — It is not.

9. Settled Conclusions

Q1. What is the difference between OCR and Layout Analysis?

OCR recognises characters. Layout analysis recognises blocks of characters and their reading order. Two separate stages. Chapter: §4.

Q2. Default OCR for Korean corpora?

PaddleOCR (kor). Mix in Azure DI when tables or signatures matter. Chapter: §7.1, §7.4.

Q3. Which two tools lead on table extraction?

AWS Textract and Azure DI. Both produce cell-level tables and key-value pairs. Chapter: §7.3, §7.4.

Q4. When is LayoutLMv3 the right choice?

After OCR provides (text + bbox), when you need classification or field extraction on the document — typically RAG KB enrichment. Chapter: §7.5.

Q5. Why does OCR fix the "question marks" failure on Korean digital PDFs?

OCR re-recognises characters from pixels, bypassing the PDF's broken font map. Chapter: §8.1.

10. Further Reading

Primary

  • Du, Y. et al. PP-OCRv4. arXiv:2304.10833
  • Smith, R. An Overview of the Tesseract OCR Engine. ICDAR 2007.
  • Huang, Y. et al. LayoutLMv3. ACM MM 2022. arXiv:2204.08387
  • AWS Textract / Azure Document Intelligence official model cards (2024 updates).

Official docs

  • PaddleOCR: https://github.com/PaddlePaddle/PaddleOCR
  • Tesseract: https://tesseract-ocr.github.io/
  • AWS Textract: https://docs.aws.amazon.com/textract/
  • Azure Document Intelligence: https://learn.microsoft.com/azure/ai-services/document-intelligence/
  • LayoutLMv3 model card: https://huggingface.co/microsoft/layoutlmv3-base

Supporting

  • Author note §3 — OCR / Layout priority.
  • Author note §35-6 — Filter-first's preprocessing value.

Cheat Sheet

Tool Korean Tables License Cost Fits
Tesseract 2 1 Apache Free Simple English
PaddleOCR 4 2 Apache Free Korean default
AWS Textract 3 4 Commercial Per-page Forms, tables
Azure DI 4 4 Commercial Per-page Korean + tables
LayoutLMv3 OCR-dep 2 MIT GPU time OCR post-processing

Bridge — What's Next

Next — RAG Core Study (5/26) — Five Paths of Chunking.

The documents are markdown, the search space is partitioned. Now: in what unit do we split? Fixed / Paragraph / Heading / Semantic / Sliding Window — and the Whole-document vs Chunk-level decision table left over from Part 3.

Series overview: Series index

댓글

댓글 쓰기

이 블로그의 인기 게시물

Agent Memory Engine (2/10) — Building an AI Agent Memory System with SQLite Alone

← 1/10 SQLite Memory Engine f… 📚 Series Index 3/10 memcore Completed → A memory engine that runs without daemons, without dependencies, anywhere 핵심 요약 This post covers how to design an AI agent memory engine (memcore) on a single-file SQLite backend. - Structure: 25 modules, ~6,464 lines, 22 DB tables - Principles: no daemon, single-file portability, no required external dependencies, host-agnostic - Core techniques: 4-layer hybrid search (topic · vector · FTS5 · LIKE), U-tag dialectic-based confidence evolution, ingestion-layer bias rejection rules What This Post Covers One approach to building AI agent memory without a server or cloud vector DB. Specifically: (1) the limitations of text-file-based memory, (2) vector DB alternatives compared and selection criteria, (3) how to layer search tiers, (4) how to evolve confidence scores over time, and (5) how to block biased memory accumulation at the ingestion stage. Design Principles: Why Single-File SQLite Text-file-based ...
Read more »

"ML Foundations (9/9) — PyTorch vs TensorFlow, and the Road to Local LLMs"

이미지
← 8/9 Deep Learning Architec… 📚 Series Index (series end) The final part. So far we've focused on models. Now we focus on the tools that actually run them . We'll lay out the philosophical difference between PyTorch and TensorFlow, and trace how Hugging Face transformers , llama.cpp , MLX , and Ollama built the bridge to running large language models on your own machine. By the end you should have the full mental model of "download a pretrained LLM and serve it locally." 0. Learning Objectives Explain the eager-vs-graph difference between PyTorch and TensorFlow. Explain, in graph terms, how autograd automates the backward pass. Use Hugging Face transformers ' AutoModel , AutoTokenizer , and pipeline abstractions. Describe llama.cpp's GGUF quantization, INT4 inference, and CPU-first flow. Describe Apple MLX's use of unified memory and how it differs from PyTorch. Run a local LLM with Ollama and call it through an OpenAI-compatible API. ...
Read more »

"RAG Core Study (14/26) — Evaluation Sets with RAGAS & DeepEval"

이미지
Series overview: Series index 📚 Series Index Without an evaluation set, every RAG improvement is a story, not evidence. RAG systems fail in more than one place: retrieval, context selection, answer generation, and grounding. That means "looks better" is not a metric. Part 14 explains how to build a golden dataset , how RAGAS and DeepEval fit into the workflow, and why teams must separate retrieval quality from answer quality if they want tuning decisions to hold up. 0. Prerequisites Part 13 reranking — system quality now depends on multi-stage ranking. Part 1 grounding — a correct-looking answer is not enough. Part 12 Hybrid — multiple retrieval settings need comparison on fixed data. 1. Learning Objectives Build a small but useful golden dataset for RAG. Distinguish retrieval metrics from answer metrics . Use RAGAS and DeepEval in the right roles. Avoid the common traps in judge-model-based evaluation. 2. 핵심 요약 An evaluation set for RAG is ...
Read more »

"ML Foundations (8/9) — Deep Learning Architectures: CNN, RNN, Attention"

이미지
← 7/9 Deep Learning Training 📚 Series Index 9/9 PyTorch vs TensorFlow → What did we keep adding on top of an MLP? Vision went CNN, sequences went RNN/LSTM, and eventually both converged on Attention and the Transformer. Each architecture is easier to remember if you read it as what it gave up to gain something else . This part is the one-line summary of the decisive inflection points: LeNet → AlexNet → ResNet → LSTM → Attention → Transformer. 0. Learning Objectives Explain why CNN's convolution, pooling, and weight sharing match images so well. Trace what got unblocked from LeNet → AlexNet → VGG → ResNet as depth grew. State BPTT for RNNs and the vanishing/exploding gradient problem. Write the LSTM gate equations (forget, input, output) with intuition. Write the attention formula in query/key/value form and the scaled dot-product variant. State the precise reasons Transformers replaced RNNs (parallelism + long-range dependencies). 1. 핵심 요약 CNN : weight sharin...
Read more »

"ML Foundations (7/9) — Deep Learning Training: Optimizers, Regularization, Initialization"

← 6/9 Neural Networks 📚 Series Index 8/9 Deep Learning Architec… → Building an MLP, as we did in Part 6, is not the same as getting it to train well . The right optimizer, regularization, initialization, and learning rate — when these line up, deep networks converge. When they don't, the network refuses to learn at all. This part is the catalog of those crafts, with formulas, paper citations, and code in one place. 0. Learning Objectives Compare and write the update rules for SGD, Momentum, Nesterov, and Adam. Explain how Dropout, BatchNorm, and LayerNorm work and where they belong in a model. Derive the variance formulas for Xavier and He initialization and match them to activations. Implement step, cosine, and warmup learning-rate schedules in PyTorch. Explain why gradient clipping is effectively required for RNNs and Transformers. Diagnose the most common training failures (NaN, plateau, overfitting) and apply first-line fixes. 1. 핵심 요약 SGD : \(w \leftarro...
Read more »

OpenClaw to Hermes Migration (2/13) — What to Preserve, Partially Port, or Discard

← 1/13 Current Structure Inve… 📚 Series Index 3/13 What Moves to Hermes → A code review record classifying operational scripts into three tiers: preserve, partial-preserve, and discard. What This Post Covers An asset classification framework (preserve / partial-preserve / discard) for migrating a legacy agent system to a successor A concrete case applying the Strangler Fig + Subset Migration strategy to a memory pipeline Classification results evaluated across LOC, external dependencies, and replaceability axes A method for slimming down a bloated single file (1,310 LOC) using the migration as a natural entry point Problem Definition: Port Everything or Cut Deep? The OpenClaw inventory comprises multiple agents, multiple scheduler/daemon processes, and multiple operational scripts. When moving to the successor system, Hermes, the choice is not binary. A full port carries accumulated technical debt along with it; a minimal port discards hard-won domain logic. Code-revie...
Read more »

AI Agents I Built (5/7) — Building an Automated Blogger API Publishing System

← 4/7 My Life Organizer Agent 📚 Series Index 6/7 Dual → Designing a Markdown-Based Auto-Publish Pipeline with Blogger API v3 + OAuth 2.0 Key Summary Blogger API v3 + OAuth 2.0 enables a pipeline that converts Markdown files into publishable HTML posts blogger_publish.py handles the full flow — frontmatter parsing → body cleanup → HTML conversion → CSS injection → API post — in a single script For bulk publishing, quota management (delay, retry, TOC suppression) is the critical design concern Platform Selection Criteria API access is mandatory for AI agent-driven content publishing. A comparison of platforms: Platform API Cost AdSense Decision WordPress.com REST API Paid plan required Paid only Cost overhead Ghost REST API Self-hosted or paid Manual setup Ops overhead Blogger v3 API Free Native integration Adopted Blogger is free, exposes a REST API, and integrates natively with AdSense. For personal blog automation, this combination i...
Read more »