Use indexes in squashed tables

[erizocosmico]

Related to: src-d/go-mysql-server#187

[erizocosmico]

After thinking a bit about this, I think the way to go forward with this can be divided in two steps:

First iteration: just make the indexing work here by also taking into account for the squashing plan.IndexableTable and getting the indexes from there. Only use the indexes of the topmost table (the first step of the iterator) as the others cannot be used.

Because the index release now works per-query there is no way to just release a few indexes and not others, so the downside of doing this is that all indexes involved in this query would be locked until it finishes, even though they are currently not being used because they're not for the topmost node.

Second iteration: after having expression removal for indexes in go-mysql-server we will need a way to retrieve those expressions here. We cannot just insert a rule and do this with some analyzer magic because squashing tables rule needs to go after Pushdown, which needs to be after index assign. So, if expressions are removed from those steps, it means they're gone and there's no way around that.
The only way I could see this working with that logic is if plan.IndexableTable kept not only the index but the expressions it replaces. The plan.IndexableTable will not use them itself but it may be useful for things like this.

Another possible way forward is to just decide to keep all expressions, even if they are part of an index. The worst case would be and UDF that might be expensive, which in this case is only enry and is not that expensive.

WDYT @ajnavarro?

[ajnavarro]

The first iteration was the same idea I came with, and the easiest one to make possible use the power of squash joins and indexes at the same time.

I'm thinking in a way to push down filters from any comparison expression that is indexed, to make indexes usable for joins, per example. Maybe can be related to the second iteration.

We can start with the first iteration that and then think about second steps. WDYT?

[erizocosmico]

Yep, I think it's better to do this first step now and tackle the other thing after we're in a more mature step with the indexes.

[erizocosmico]

In the middle of this I found a problem trying to fit together this two things:

• Indexes do not work per-repo, they just return stuff however they want
• The squash iterators are RowRepoIters, which means they do this iteration per repository

This means we can't mix together this as it is. There would be two options for this:

• Make indexes work per repo (which we can't do).
• Make squash iterators not be RowRepoIters (which we can do but it's a huge task). The downsides to this is that pool repo iteration will be repeated in every iterator that is a starting point (can't have parents) and all iterators would need to be modified to pass down the instance of the repository as well as be refactored to take into account the fact that they will not work per-repo.

WDYT? @ajnavarro