One Vendor, Four Spellings: How Deterministic Stages Beat Similarity Scores

A multi-stage pipeline effectively deduplicates a large supplier list by first applying simple, deterministic rules. Normalizing hostnames and collapsing subdomains to their registered domains successfully removed 76% of the duplicate entries without complex models. The remaining records were compared using fuzzy string matching, which revealed that similarity scores for true duplicates and distinct vendors heavily overlapped. This overlap demonstrates that no single threshold can safely automate merges, making the matcher's primary value the creation of a prioritized queue for human review.