How to cluster based on filtered predictions?

[nickloganfarmer]

After I run the linker.inference.predict function on my dataset, I manually run the predictions through a custom filter process to cut down on incorrect predictions and end up with a Pandas dataframe. However, the clustering seems to need to use the original SplinkDataframe produced from the predict function. Is there a way for me to use the cluster_pairwise_predictions_at_threshold function with my filtered Pandas dataframe and not the entire original SplinkDataframe? Below is an example of my code with my custom "replace_mismatches" function used to filter and return a clean Dataframe:

pairwise_predictions = linker.inference.predict(threshold_match_probability = 0.92)

predictPandaDf = pairwise_predictions.as_pandas_dataframe().drop_duplicates(subset=["id_l", "id_r"], inplace=True)

predictPandaDf = predictPandaDf[predictPandaDf.apply(replace_mismatches, axis=1)].sort_values("match_weight", ascending=False)

[ADBond]

You can register a new predictions table with the linker by using linker.table_management.register_table_predict():

predict_splink_df = linker.table_management.register_table_predict(predictPandaDf)
clusters = linker.clustering.cluster_pairwise_predictions_at_threshold(predict_splink_df, threshold_match_probability=...)

In general you can use linker.table_management.register_table() to turn a table into a SplinkDataFrame - using register_table_predict just means that it is also cached, so that in some circumstances Splink can avoid redoing calculations.