Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use vtbackup in scheduled backups #658

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

frouioui
Copy link
Member

@frouioui frouioui commented Jan 28, 2025

This Pull Request modifies the scheduled backup feature that was introduced in #553. The implementation in the original PR is rather simple and was a "v1". The CRD VitessBackupSchedule was introduced, giving the option to end-users to configure different Strategy - which tells which keyspace/shard we want to benchmark. Per VitessBackupSchedule all these strategies were merged together to form a single vtctldclient command that would be executed in a K8S's Job following the schedule set in the VitessBackupSchedule.

While this approach works, it has limitation: it uses vtctldclient which will temporarily take out a serving tablet from the serving tablet pool and use it to take the backup. While this limitation is documented in the release notes, we want to encourage people to use vtbackup as an alternative.

The goal of this Pull Request is to modify the VitessBackupSchedule controller to create K8S's Job with a vtbackup pod instead of a dummy pod running a vtctldclient command. This new goal introduces a new complexity inherited by vtbackup: a vtbackup pod can only backup one shard. Which is significantly different than the previous implementation where we created a dummy pod that would contain a shell command executing vtctldclient as many times as we had a Strategy in the VitessBackupSchedule CRD. To fit this new requirement, the VitessBackupSchedule controller's logic was changed to create as many Job as we have Strategy in the CRD. The Jobs are named after the CRD's name + the keyspace + the shard range. For every Job a new PVC is also created, which is used by the vtbackup pod. Both the Job and PVC are owned by the VitessBackupSchedule CRD and will be removed following the retention policy set by the end-users.

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant