How to Parse Price Lists from PDF: Data Extraction Tools and Methods

The Problem with PDF Price Lists

The PDF format was originally created for document presentation, not data exchange:

• Lack of structure: the data does not have a clear tabular structure; information can be located in arbitrary positions
• Complex layout: Multi-column layouts, merged cells, table breaks between pages
• Different types: Text-based PDFs and scanned images require different approaches to processing
• Non-standard formats: Each supplier uses its own price list template

To extract data effectively, it is necessary to correctly identify the document type and select the appropriate tools.

Types of PDF documents

PDF price lists can be divided into two main categories:

Text PDFs

Documents created programmatically (from Excel, Word, 1C) with the text layer preserved. Characteristics:

• Ability to select and copy text
• Font and positioning information is preserved
• Relatively simple data extraction
• Good quality of table recognition

Scanned PDFs

Scanned or photographed documents that are images. Characteristics:

• Text cannot be selected or copied.
• OCR (Optical Character Recognition) required
• Quality depends on the scan resolution and the condition of the original.
• More complex and slower processing

Methods for parsing text PDFs

For text-based PDFs, there are several approaches to data extraction:

1. Extraction by coordinates

This method is based on precise positioning of elements on the page. It's suitable for standardized price lists with a fixed structure:

• The coordinates of the areas with data are determined
• Extracts text from specific areas of a document
• Requires customization for each price list template
• High precision with stable format

2. Table recognition

Automatic detection of table structures in a document:

• Analysis of separator lines and spaces
• Defining cell and row boundaries
• Extracting data in a structured form
• Adapts to various layouts

3. Text analysis

Extracting all text and then parsing it using templates:

• Obtaining unstructured text
• Search for patterns (articles, prices, names)
• Using regular expressions
• Suitable for non-standard formats

OCR for scanned PDFs

Optical character recognition (OCR) converts images of text into an editable format.

OCR technologies

Tesseract OCR — a free open source library from Google:

• Support for over 100 languages, including Russian
• Ability to learn from your own data
• Integration with various programming languages
• Requires image pre-processing

ABBYY FineReader — a commercial solution with high accuracy:

• Advanced recognition algorithms
• Excellent quality for the Russian language
• Automatic distortion correction
• Recognizing complex tables

Improving OCR quality

Image pre-processing significantly improves recognition results:

• Resolution increase: at least 300 DPI for high-quality recognition
• Binarization: convert to black and white for clearer text
• Tilt correction: aligning rotated scans
• Noise Removal: filtering artifacts and noise
• Contrast enhancement: improving the readability of faded text

PDF parsing tools

Tabula

Free GUI tool to extract tables from PDF:

• Simple visual selection of areas to extract
• Export to CSV, JSON, and Excel
• Batch processing of multiple files
• Versions available for Windows, Mac, and Linux
• Library for automation via Python/R

Camelot

Python library for precise table extraction:

• Two modes: Stream (without borders) and Lattice (with lines)
• Fine-tuning recognition parameters
• Evaluation of the quality of extracted data
• Export to various formats
• Integration into automated processes

PDFTables

Online service for converting PDF tables:

• Web interface without installing programs
• API for automatic processing
• High accuracy of structure recognition
• Paid processing rates

Adobe Acrobat Pro

A professional solution with wide capabilities:

• Export PDF to Excel with structure preserved
• Built-in OCR for scanned documents
• Editing and optimizing PDFs
• Batch file processing
• Paid license

Python libraries

Specialized libraries are available for software processing of PDF:

• PyPDF2: basic text and metadata extraction
• pdfplumber: detailed analysis of the document structure
• PDFMiner: low-level parsing with control over all elements
• pdf2image: Convert pages to images for OCR

Automation of PDF price list parsing

To process price lists regularly, the entire process must be automated.

Stages of automation

1. Receiving files: Monitoring email, FTP, and cloud storage for new pricing
2. Type definition: Automatically check for the presence of a text layer in a PDF
3. Selecting a method: using OCR for scans or direct parsing for text files
4. Data Extraction: applying configured rules to a specific provider
5. Normalization: bringing data to a unified format
6. Validation: checking the correctness of the extracted data
7. Loading: import into the target system (ERP, online store)

Error handling

The system must correctly handle problematic situations:

• Logging files with low recognition quality
• Critical parsing error notifications
• Manual verification of questionable data
• Saving backup copies of source files

Checking the quality of extracted data

After parsing, you need to make sure that the received data is correct.

Validation methods

• Completeness control: comparison of the number of items with previous versions of the price list
• Checking formats: validation of articles, prices, units of measurement
• Value ranges: identifying abnormal prices or quantities
• Required fields: checking for the presence of critical data
• Comparison with the standard: testing manually labeled data on a test sample

Quality metrics

Evaluation of parsing efficiency:

• Accuracy: the proportion of correctly extracted data among all extracted data
• Completeness (Recall): the proportion of extracted data from all data available in the document
• F1 measure: harmonic mean of precision and recall
• Processing speed: parsing time for one document

International specifics

When working with price lists from foreign suppliers, take regional characteristics into account.

Multilingual recognition

• Using OCR with support for multiple languages simultaneously
• Document processing in English, German, French, Chinese
• Correct recognition of special characters and diacritics
• Setting up dictionaries for industry terminology

Data formats

• Numbers: comma or period as a separator (10.50 or 10.50)
• Dates: DD.MM.YYYY (Europe) or MM/DD/YYYY (USA)
• Currencies: various symbols and codes (EUR, USD, GBP, CNY)
• Units of measurement: metric system vs. imperial/US system

Conclusion

Parsing PDF price lists requires a comprehensive approach, taking into account the document type and its structure. For text-based PDFs, specialized table recognition tools are effective, while scanned documents require preliminary OCR processing.

Properly automating the parsing process saves dozens of hours of manual work and minimizes errors during data transfer. The choice of tools depends on the volume of data to be processed, accuracy requirements, and available budget.

For large-scale processing of price lists from multiple suppliers, we recommend using ready-made automation platforms that integrate various parsing methods and ensure the stable processing of documents of any complexity.