Merge branch 'aws:main' into bhaoz-READMEupdate-withSigned

.github/pull_request_template.md

@@ -0,0 +1,32 @@
+# PR Approval Steps
+
+## For Requester
+
+1. Description
+    - [ ] Check the PR title and description for clarity. It should describe the changes made and the reason behind them.
+    - [ ] Ensure that the PR follows the contribution guidelines, if applicable.
+2. Security requirements
+    - [ ] Ensure that a Pull Request (PR) does not expose passwords and other sensitive information by using git-secrets and upload relevant evidence: https://github.com/awslabs/git-secrets
+    - [ ] Ensure commit has GitHub Commit Signature
+3. Manual review
+    1. Click on the Files changed tab to see the code changes. Review the changes thoroughly:
+        - [ ] Code Quality: Check for coding standards, naming conventions, and readability.
+        - [ ] Functionality: Ensure that the changes meet the requirements and that all necessary code paths are tested.
+        - [ ] Security: Check for any security issues or vulnerabilities.
+        - [ ] Documentation: Confirm that any necessary documentation (code comments, README updates, etc.) has been updated.
+4. Check for Merge Conflicts:
+    - [ ] Verify if there are any merge conflicts with the base branch. GitHub will usually highlight this. If there are conflicts, you should resolve them.
+
+## For Reviewer
+
+1. Go through `For Requester` section to double check each item.
+2. Request Changes or Approve the PR:
+    1. If the PR is ready to be merged, click Review changes and select Approve.
+    2. If changes are required, select Request changes and provide feedback. Be constructive and clear in your feedback.
+3. Merging the PR
+    1. Check the Merge Method:
+        1. Decide on the appropriate merge method based on your repository's guidelines (e.g., Squash and merge, Rebase and merge, or Merge).
+    2. Merge the PR:
+        1. Click the Merge pull request button.
+        2. Confirm the merge by clicking Confirm merge.
+

.github/workflows/security-monitoring.yml

@@ -67,7 +67,7 @@ jobs:
 
   put-metric-data:
     runs-on: ubuntu-latest
-    needs: [check-dependabot-alerts]
+    needs: [check-dependabot-alerts, check-code-scanning-alerts]
     steps:
       - name: Configure AWS Credentials
         uses: aws-actions/configure-aws-credentials@12e3392609eaaceb7ae6191b3f54bbcb85b5002b

helm_chart/readme.md

@@ -60,7 +60,7 @@ Notes:
 * Update helm chart dependencies. It ensures that all the sub-charts required by the main chart are fetched and properly set up before deploying the main chart. It doesn’t actually deploy the chart itself but prepares it by ensuring all dependencies are resolved.
 
   ```
-  helm dependencies update src/hyperpod_cli/helm_chart/HyperPodHelmChart
+  helm dependencies update helm_chart/HyperPodHelmChart
   ```
 
 ### Step Two:
@@ -84,7 +84,7 @@ Notes:
 
 * Deploy a Helm Chart to Your Kubernetes Cluster. This command deploys the Helm chart to your cluster with custom configurations applied, as specified in the values.yaml file. Please note that only certain versions of dependencies will be deployed based on the configuration specified in the values.yaml.
   ```
-  helm install dependencies src/hyperpod_cli/helm_chart/HyperPodHelmChart --namespace kube-system
+  helm install dependencies helm_chart/HyperPodHelmChart --namespace kube-system
   ```
 
 ### Step Four (only required for installing Kueue):

test/integration_tests/abstract_integration_tests.py

@@ -223,36 +223,16 @@ def describe_vpc_stack_and_set_values(self, cfn_client):
                         "OutputValue"
                     )
 
-    def update_cluster_auth(self):
-        with open(
-            "test/integration_tests/charts/hp-node-auth.yaml",
-            "r",
-        ) as hyperpod_current_context:
-            template = hyperpod_current_context.read()
-
-        template = re.sub(
-            "SAGEMAKER_EXECUTION_ROLE",
-            self.cfn_output_map.get("ExecutionRole"),
-            template,
-        )
-        template = re.sub(
-            "SAGEMAKER_SERVICE_ROLE",
-            self.cfn_output_map.get("ServiceRole"),
-            template,
-        )
-
-        with open("/tmp/hp-node-auth.yaml", "w") as hyperpod_current_context:
-            hyperpod_current_context.write(template)
-
+    def apply_helm_charts(self):
         command = [
-            "kubectl",
-            "apply",
-            "-f",
-            "/tmp/hp-node-auth.yaml",
+            "helm",
+            "dependencies",
+            "update",
+            "helm_chart/HyperPodHelmChart"
         ]
 
         try:
-            # Execute the command to update kubeconfig
+            # Execute the command to update helm charts
             logger.info(
                 subprocess.run(
                     command,
@@ -262,36 +242,37 @@ def update_cluster_auth(self):
                 )
             )
         except subprocess.CalledProcessError as e:
-            raise RuntimeError(f"Failed to apply auth charts: {e}")
+            raise RuntimeError(f"Failed to update helm charts: {e}")
 
-    def install_training_operator(self):
-        command = [
-            "kubectl",
-            "apply",
-            "-k",
-            "github.com/kubeflow/training-operator/manifests/overlays/standalone?ref=v1.7.0",
+        apply_commanmd = [
+            "helm",
+            "install",
+            "dependencies",
+            "helm_chart/HyperPodHelmChart",
+            "--namespace",
+            "kube-system"
         ]
 
         try:
-            # Execute the command to update kubeconfig
+            # Execute the command to apply helm charts
             logger.info(
                 subprocess.run(
-                    command,
+                    apply_commanmd,
                     check=True,
                     capture_output=True,
                     text=True,
                 )
             )
         except subprocess.CalledProcessError as e:
-            raise RuntimeError(f"Failed to install training operator: {e}")
+            raise RuntimeError(f"Failed to apply helm charts: {e}")
+
 
     def setup(self):
         self.new_session = self._create_session()
         self.create_test_resorces(self.new_session)
         self.create_kube_context()
-        self.update_cluster_auth()
+        self.apply_helm_charts()
         self.create_hyperpod_cluster(self.new_session)
-        self.install_training_operator()
 
     def tearDown(self):
         self.delete_hyperpod_cluster(self.new_session)