Blocking on coordinates

[lamaeldo]

Hello, i'm trying to use splink for record linking accross two datasets of individuals which have coordinates (CRS is British National Grid). I've been trying to use coordinates as a blocking tool finding all records within 20 meters, using SQRT(POWER(l.x_coordinate - r.x_coordinate, 2) + POWER(l.y_coordinate - r.y_coordinate, 2)) <= 20, but this is very slow and doesn't seem to be making use of all cores (isn't constrained by memory either). I was wondering if this was inherent to the fact that blocking rules are SQL conditions, meaning that a geographic blocking can never be efficient, or if there is a way to improve this

[RobinL]

Blocking rules need to be equality conditions for good performance see here:

https://moj-analytical-services.github.io/splink/topic_guides/blocking/performance.html?h=equality#equi-join-conditions

With a continuous numeric variable, probably you want to block on the rounded numbers

Thinking about it, you actually need two blocking rules on the rounded values e.g. 0.9 and 1.1 are close, but if rounding up will round to different values. So you want to block on (say) rounded up, and also the value +0.5 rounded up

[lamaeldo]

Thanks for this, i've been thinking that another way of checking this would be to create two grids of squares of x meters, one starting at coordinates 0;0, and one another one starting at x/2;x/2. In pre processiong, each of the records can be assigned identifier of the square they are located in for both grids. I'm thinking that by then blocking on both of these identifiers, this would achieve roughly the same effect. Would that be correct or am I missing something?

[RobinL]

Yes - exactly, that should also work fine