Using Splink to identify duplicate transactions

[mrbdo]

Hi, I wonder whether Splink would be a good library for identifying duplicate transactions in a dataset. For example duplicate payments based on the amount, date, vendor name, and invoice reference. The expected number of matches would be several orders of magnitude lower than when using Splink for classic deduplication of a person dataset, therefore I am not sure the probabilistic model would work as well. Any thoughts on that ?

[RobinL]

https://moj-analytical-services.github.io/splink/demos/examples/duckdb/transactions.html

[mrbdo]

Thanks for the prompt reply. I saw this example where we know we have 100% duplicate between the datasets. I ll take that even if there is on the contrary very few duplicates in the whole dataset, the library should still work then ! thanks

[RobinL]

Yeah, finding links and finding dupes are equivalent problems so you can use Splink for either problem