Gitleaks - Secret Scanning

.dockerignore

@@ -1 +1,2 @@
-checkpoints/
+checkpoints/
+.github

.github/workflows/gitleaks.yml

@@ -0,0 +1,15 @@
+name: gitleaks
+on: [pull_request, push, workflow_dispatch]
+jobs:
+  scan:
+    name: gitleaks
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+      - uses: gitleaks/gitleaks-action@v2
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE}}
+          GITLEAKS_NOTIFY_USER_LIST: '@sandergi'

.gitignore

@@ -206,5 +206,7 @@ cython_debug/
 # Poetry local configuration file - https://python-poetry.org/docs/configuration/#local-configuration
 poetry.toml
 
+# Gitleaks
+gitleaks-baseline.json
 
 # End of https://www.toptal.com/developers/gitignore/api/python,macOS

.pre-commit-config.yaml

@@ -0,0 +1,12 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.6.0
+    hooks:
+    -   id: end-of-file-fixer
+    -   id: check-yaml
+-   repo: https://github.com/gitleaks/gitleaks
+    rev: v8.18.4
+    hooks:
+    -   id: gitleaks

README.md

@@ -273,6 +273,8 @@ The recommended way to turn a set of research scripts like [Wav2Lip](https://git
 
 4. Once you have these written, you can follow the same steps as the common models to deploy the model.
 
+### 💣 Secret Scanning
 
+Gitleaks will automatically run pre-commit (see `pre-commit-config.yaml` for details) to prevent commits with secrets in the first place. To test this without committing, run `pre-commit` from the terminal. To skip this check, use `SKIP=gitleaks git commit -m "message"` to commit changes. Preferably, label false positives with the `#gitleaks:allow` comment instead of skipping the check.
 
-
+Gitleaks will also run in the CI pipeline as a GitHub action on push and pull request (can also be manually triggered in the actions tab on GitHub). To update the baseline of ignored secrets, run `python ./scripts/create_gitleaks_baseline.py` from the venv and commit the changes to `.gitleaksignore`.

scripts/create_gitleaks_baseline.py

@@ -0,0 +1,18 @@
+#!/usr/bin/env python3
+
+import subprocess
+import json
+
+# create a baseline file
+subprocess.run(
+    ["gitleaks", "detect", "--report-path", "gitleaks-baseline.json"],
+)
+
+# parse the baseline file
+with open("gitleaks-baseline.json") as f:
+    baseline = json.load(f)
+
+# output list of "Fingerprint"s to .gitleaksignore
+with open(".gitleaksignore", "w") as f:
+    for leak in baseline:
+        f.write(leak["Fingerprint"] + "\n")