JSON Schema Validation for Helm

[igooch]

Note

Milestone of #3766

Introduction

As part of Agones in-place upgrades we identified adding JSON schema validation as a step in validating an Agones upgrade #3766. The goal of this validation is to make any breaking changes in the Helm chart immediately apparent to the user.

JSON schema validation will validate the Agones Helm chart .Values object (note that this is not the values.yaml file itself. Details are in the Helm documentation on JSON schema files). Validation only applies to the helm install or helm upgrade, and not a direct kubectl install of the values.yaml file.

Critical User Journey

1. An Agones release has a breaking change. For example foo.bar moves to foo.controller.bar.
2. User does not read the documentation about the breaking change, and applies helm upgrade --install agones --version=$NewRelease with their current chart configuration of --set foo.bar=baz.
3. Before the new release is installed by Helm, the validation of the Helm .Values object fails with error similar to Error: validation failed foo.controller.bar is required.
   • Note that Helm currently uses https://github.com/xeipuuv/gojsonschema as its JSON validation, so all error messages come from this repo, and are not modified by Helm or Agones.
4. User makes changes suggested by the Helm error, and successfully re-applies helm upgrade with --set foo.controller.bar=baz.

Design Overview

Goals

• Validation only applies to Agones Helm installations or upgrades.
• Validate that all values required for an Agones installation are present.
• Validate that all values are of the correct type.
• Validate that there are no unknown given values. (Attempting to set a value at a path that does not exist.)
• The validation only applies to a specific chart version. I.e. the JSON schema validation for chart 1.46 is only applied to 1.46 and not 1.45 or 1.47.

Non-Goals

• Validating configurations. JSON schema does basic validation of Helm chart values (required field is present, value is of the correct type, etc.), but is not sophisticated enough to handle configuration versions.
• Non-Helm Agones installations or upgrades.

Future Goals

• Warning on deprecated fields. This ability is present in newer versions of JSON schema, but is not yet supported by Helm.

Core Changes Required

Create a new values.schema.json in agones/helm/install/agones.

Testing

• Add a new CI / CD test using helm lint to validate the standard Helm installation.
  • Possible alternatives: helm template --debug or helm upgrade --install --dry-run --debug

Documentation

• User-facing documentation
  • Adding info on chart validation to https://agones.dev/site/docs/installation/install-agones/helm/ and possibly https://agones.dev/site/docs/installation/upgrading/
  • Include details on how to test installation first (helm upgrade --dry-run --debug).
  • If unable to resolve validation error and it is a valid installation, [Break Glass and Don’t Panic] use --skip-schema-validation flag, and file an issue or PR on Agones GitHub with the details on what part of the valid validation failed.
• Developer documentation
  • Adding or changing a Helm value must update the values.schema.json file.
    • There is no way of accurately generating the schema – this must be done manually.
    • Add instructions to agones/install/helm/agones/README.md