From 4 Weeks to 45 Minutes: Designing a Document Extraction System for 4,700+ PDFs

A hybrid system was developed to extract revision numbers from 4,700 engineering drawing PDFs by combining deterministic and AI methods. The pipeline first uses the PyMuPDF library for fast, rule-based extraction on text-based documents, successfully processing about 70-80% of the corpus at no cost. For the remaining image-based or ambiguous PDFs, the system falls back to using GPT-4 Vision for optical character recognition and extraction. This two-stage approach proved significantly more efficient, reducing processing time from over 1.5 hours to 45 minutes and cutting API costs by over 70% compared to a pure AI solution. The project highlights the value of balancing cost, speed, and accuracy by using expensive AI models only when necessary.