Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Epic] implement Testing Framework for Docs Code Snippets #13928

Open
5 of 7 tasks
interurban opened this issue Jan 31, 2025 · 0 comments
Open
5 of 7 tasks

[Epic] implement Testing Framework for Docs Code Snippets #13928

interurban opened this issue Jan 31, 2025 · 0 comments
Assignees
Labels
area/docs Improvements or additions to documentation kind/epic Large new features or investments needs-triage Needs attention from the triage team

Comments

@interurban
Copy link
Collaborator

interurban commented Jan 31, 2025

Objective:
To ensure the accuracy, reliability, and up-to-date status of code examples in the Pulumi documentation by implementing a robust testing framework that integrates with our GitHub PR workflow and includes scheduled cron jobs.

Background:
Our documentation is a critical resource for users to understand and effectively implement Pulumi solutions. Maintaining high-quality, executable code examples is essential for providing clear and accurate guidance. To achieve this, we've requre a testing framework that automatically tests code snippets both before merging PRs and on a regular schedule to catch out of date code examples and flag them for remediation..

Goals:

Pre-Merge Testing: Integrate the testing framework into the PR workflow to automatically test code snippets in the static/programs directory. This ensures that all examples are functional and correct before they are merged into the main documentation.
Scheduled Testing: Implement a cron job that regularly tests all code snippets to catch and rectify any issues that arise from changes in Pulumi’s APIs or dependencies.
Makefile Support: Allow example programs to include a Makefile with a test target, enabling more complex testing scenarios beyond the default pulumi preview.
Enhanced Shortcode Flexibility: Update Hugo shortcodes to import any arbitrary file from an example program, enhancing the documentation’s ability to showcase diverse and complex examples.
Documentation and Guidelines: Provide comprehensive documentation on how to use the testing system and contribute code examples, ensuring consistency and ease of use for all contributors.

Tasks:

  • Integration of testing framework into GitHub PR workflow.
  • Setup and configuration of cron job for regular testing.
  • Documentation of testing procedures and shortcode usage in our public readmes
  • Implementation of Makefile support for complex testing scenarios.
  • Update and deploy Hugo shortcode enhancements.
  • Alert doc-ops for code example test cron job failures #13931
  • K8 testing solution (requires standing up a cluster for ongoing testing as to not delay PR process time)
@pulumi-bot pulumi-bot added the needs-triage Needs attention from the triage team label Jan 31, 2025
@github-project-automation github-project-automation bot moved this to 🤔 Triage in Docs 📚 Jan 31, 2025
@interurban interurban added area/docs Improvements or additions to documentation kind/epic Large new features or investments labels Jan 31, 2025
@interurban interurban self-assigned this Jan 31, 2025
@interurban interurban moved this from 🤔 Triage to 🤯 Epics in Docs 📚 Jan 31, 2025
@interurban interurban changed the title [Epic] [Epic] implement Testing Framework for Docs Code Snippets Jan 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/docs Improvements or additions to documentation kind/epic Large new features or investments needs-triage Needs attention from the triage team
Projects
Status: 🤯 Epics
Development

No branches or pull requests

2 participants