Upgrade path for Gramps 6.0/database schema version 21

[DavidMStraub]

@Nick-Hall, @dsblank - I am worried about how to best ensure a smooth upgrade path for Gramps Web (API) users when we switch to JSON data.

So far, the only schema change that needed to be handled during the existence of Gramps Web API was the 5.1 → 5.2 migration, for which we added

• the python -m gramps_webapi grampsdb --tree <sometreeid> migrate CLI command
• the /api/trees/<sometreeid>/migrate endpoint

Under the hood, these two commands simply call the database load method with force_schema_upgrade set to True.

However, that schema change was only a change to the pickled data, not to the table schema, so it also played nicely with the SharedPostgreSQL addon, which uses a single table to host many trees.

As far as I can see, for SQLite this should work as well for the schema 21 migration. Can you confirm?

For the "normal" PostgreSQL addon, are you planning a JSON compatible version @dsblank?

For SharedPostgreSQL, it seems I will have to implement the changes analogous to the ones made in PR gramps-project/gramps#1786 to dbapi.py, e.g. write JSON instead of blobs and implement the upgrade_table_for_json_data method.

What is the timescale for 6.0 release, so we can plan the migrations for Gramps Web (API) accordingly?

[Nick-Hall]

What is the timescale for 6.0 release

We haven't decided yet, but I have time available in the next couple of months.

[dsblank]

For the "normal" PostgreSQL addon, are you planning a JSON compatible version @dsblank?

Looking over the code, I don't see anything that needs to be updated. We might need to make some changes if we use JSON extract (and other functions) for direct access to json_data but nothing in master requires that.

[dsblank]

Oh, we'll have to add a postgresql version for column_exists():

https://github.com/gramps-project/gramps/pull/1786/files#diff-cfacb207e7d622fcd6347dd49344d3542912b9524f4da789fdfcb16bfbf3be51R194

[DavidMStraub]

Oh right, and the rest is handled by the DBAPI class. In SharedPostgreSQL, more changes will be needed because it has its own DBAPI class (https://github.com/gramps-project/addons-source/blob/maintenance/gramps52/SharedPostgreSQL/shareddbapi.py).

[DavidMStraub]

Oh, we'll have to add a postgresql version for column_exists():

https://github.com/gramps-project/gramps/pull/1786/files#diff-cfacb207e7d622fcd6347dd49344d3542912b9524f4da789fdfcb16bfbf3be51R194

@dsblank are you planning to implement that? It would be good if we have the Postgres addon updated in time for the 6.0 release.

[dsblank]

@DavidMStraub I don't really have a postgresql dev environment, and don't know postgresql. Probably better left into the hands of someone else. But I'd be glad to review.

[DavidMStraub]

I thought because you're the original author of the plugin 😆 but I can give it a try as well.