update docs

docs/mkdocs.yml

@@ -38,7 +38,8 @@ plugins:
   - macros
 markdown_extensions:
 - admonition
-- pymdownx.tabbed # https://facelessuser.github.io/pymdown-extensions/
+- pymdownx.tabbed: # https://facelessuser.github.io/pymdown-extensions/
+    alternate_style: true
 - pymdownx.keys
 - pymdownx.details
 - pymdownx.inlinehilite

docs/src/admin.md

@@ -7,9 +7,9 @@
 
 The admin panel is used to do the following:
 
-- Manually override the model outputs during events and advisories that would adversely effect the river quality.
+- Manually override the model outputs during events and advisories that would adversely affect the river quality.
 - Set a custom message that shows on the home page alongside the flags.
-- Manually update the database.
+- Manually update the database (e.g. add or remove Boathouses)
 - Download the data as a CSV, including downloading up to 90 days worth of data at a time.
 
 You can reach the admin panel by going to `/admin` after the URL for the flagging website homepage.

docs/src/cloud/twitter_bot.md

@@ -1,5 +1,8 @@
 # Twitter Bot
 
+???+ warning
+    The X/Twitter bot is no longer being used or maintained because of significant changes to the API causing the code to break.
+
 Every time the website updates, it sends out a tweet. In order for it to do that though, you need to set up a Twitter account.
 
 ## First Time Setup

docs/src/development/index.md

@@ -1,5 +1,8 @@
 # Development - Overview
 
+??? note
+    **(Nov 2024 update)** We cannot guarantee that anything in the Development section is fully up-to-date. This should be considered a historic document. The best explanation for how the code works will ultimately be the code itself.
+
 The Development guide is aimed at users who wish to understand the code base and make changes to it if need be.
 
 This overview page describes at a high-level what the website's infrastructure is, how it all relates, and why those things are in the app.
@@ -39,17 +42,17 @@ Jinja2 <.. blueprints
 class Admin
 Admin : /admin.py
 class SQLAlchemy
-PostgreSQL : (Heroku service)
+PostgreSQL : "(Heroku service)"
 SQLAlchemy <.. Admin
 SQLAlchemy <.. PostgreSQL
 Flask <.. Admin
 class BasicAuth
 BasicAuth : /admin.py
 BasicAuth ..> Admin
 class HerokuTaskScheduler
-HerokuTaskScheduler : (Heroku service)
+HerokuTaskScheduler : "(Heroku service)"
 class Redis
-Redis : (Heroku service)
+Redis : "(Heroku service)"
 Heroku .. Heroku Task Scheduler
 FlaskCaching ..> Redis
 class FlaskCaching
@@ -64,11 +67,12 @@ Predictive Models : /data/predictive_models.py
 
 ## Overview of repo
 
-Sometimes it can be a little confusing and overwhelming seeing all the files strewn about in the root directory of the repo. I get it! Here are all the files (as of writing) and why they're there.
+Sometimes it can be a little confusing and overwhelming seeing all the files strewn about in the root directory of the repo. Here are all the files (as of writing) and why they're there.
 
 ```
 .
 ├── .flaskenv
+├── .python_version
 ├── .github
 │   └── workflows
 │       └── tests.yml
@@ -85,16 +89,17 @@ Sometimes it can be a little confusing and overwhelming seeing all the files str
 ├── Procfile
 ├── pytest.ini
 ├── README.md
-├── requirements
+├── Dockerfile
+├── docker-compose.yml
+├── requirements.in
 │   └── ...
 ├── requirements.txt
-├── run_unix_dev.sh
-├── run_windows_dev.bat
 ├── runtime.txt
 └── tests
      └── ...
 ```
 
+- `.python_version`: Both Heroku and `uv` use this to determine the Python version.
 - `.flaskenv`: Helper file for Flask local deployment. ([more info](https://flask.palletsprojects.com/en/1.1.x/cli/#environment-variables-from-dotenv))
 - `.github/workflows/tests.yml`: This file is handled by Github Actions. It runs the unit-tests. ([more info](https://docs.github.com/en/actions/learn-github-actions))
 - `.gitignore`: Tells git what files to ignore. ([more info](https://git-scm.com/docs/gitignore))
@@ -105,8 +110,7 @@ Sometimes it can be a little confusing and overwhelming seeing all the files str
 - `Procfile`: Heroku uses this to know what to run on the deployed instance. ([more info](https://devcenter.heroku.com/articles/procfile))
 - `pytest.ini`: Unit-testing configuration. When you run `python -m pytest ./tests`, this file is read in. We need it for some Pytest extensions, and to define a label we use to skip tests that require credentials. ([more info](https://docs.pytest.org/en/stable/customize.html))
 - `README.md`: Self-explanatory.
-- `requirements`: This folder contains the various environment configurations, e.g. local development needs different things than production, Windows needs different things than OS X, etc. ([more info](https://github.com/jazzband/pip-tools) and [more info](https://docs.python.org/3/tutorial/venv.html))
-- `requirements.txt`: Heroku requires a `requirements.txt` in the root of the repo.
-- `run_unix_dev.sh`: On OS X and Linux, you should run this to fire up the website locally.
-- `run_windows_dev.sh`: On Windows, you should run this to fire up the website locally. This one is less maintained than the unix dev script, and may require minor manual editing to work.
-- `runtime.txt`: Heroku requires this to define the runtime. ([more info](https://devcenter.heroku.com/articles/python-runtimes))
+- `Dockerfile`: Used to build a Docker image.
+- `docker-compose.yml`: Determines how the code runs with the `docker compose` commands.
+- `requirements.in`: This is the pre-compiled version of our dependencies. Use `uv pip compile requirements.in -o requirements.txt` to generate compiled dependencies.
+- `requirements.txt`: The frozen dependencies used by the webserver. Heroku also requires a `requirements.txt` in the root of the repo.

docs/src/development_resources/index.md

@@ -1,5 +1,15 @@
 # Stack
 
+??? info
+    **(Nov 2024 update)** This document is more of a historic document to authoritatively resolve questions regarding the stack decisions. It is not an ongoing, active document. Consider this document archived.
+
+    A small handful of the things in this document are no longer true or have not fully panned out, notably:
+
+    - Heroku no longer has a free tier. We continue to use Heroku because it is affordable _enough_, and making changes at this stage would be onerous given the stability of the project.
+    - In practice, the project has been maintained not by academics or inexperienced volunteers, but by the original project lead for the website, Daniel Reeves (me), who has also developed significantly more expertise in web development and data engineering since working on this in 2020 (this project was one of the ways I gained experience!). The decision to use simple cron and CLI calls plus Pandas to process and stored in an ephemeral way is not exactly how I'd do things these days, especially given that (a) the code is not being used by others and therefore no concessions really needed to be made, (b) I have a lot more comfort spinning up infrastructure, including on cloud services that have generous free tiers for relevant services like BigQuery on GCP.
+
+    Despite that I would do a few things differently if I were to rebuild this today, I cannot say I have any regrets with our stack decisions. Even with the benefit of much gained experience over the years, I do believe our stack decisions made a lot of sense at the time, and also continue to work well enough today.
+
 ## Project History
 
 Traditionally, the CRWA Flagging Program was hosted on a PHP-built website that hosted a predictive model and ran it. However, that website was out of commission due to some bugs and the CRWA's lack of PHP development resources.

docs/src/development_resources/shell.md

@@ -1,38 +1,28 @@
 # Flask Shell Documentation
 
-The shell is used to access app functions and data, such as Hobolink and USGS
-data and access to the database.
+The shell is used to access app functions and data, such as Hobolink and USGS  data and access to the database.
 
-The reason why the shell is useful is because there may be cases where you want to play around with the app's functions. For example, maybe you see something that seems fishy in the data, so you want to have direct access to the function the website is running. You may also want to
+The reason why the shell is useful is because there may be cases where you want to play around with the app's functions. For example, maybe you see something that seems fishy in the data, so you want to have direct access to the function the website is running.
 
 The way Flask works makes it impossible to run the website's functions outside the Flask app context, which means importing the functions into a naked shell doesn't work as intended. The `flask shell` provides all the tools needed to let coders access the functions the exact same way the website does, except in a shell environment.
 
 ## Run the Shell
 
-1. Open up a terminal at the `flagging` folder.
+You can run the shell locally, but it is strongly recommended to do it in Docker Compose instead.
 
-2. Activate a Python virtual environment:
+If the Docker Compose is not running, you can spin it up and run it like so:
 
 ```shell
-python3 -m venv venv
-source venv/bin/activate
-python3 -m pip install -r requirements.txt
+docker compose run web flask shell
 ```
 
-3. Set up the `FLASK_ENV` environment variable:
+If you'd like to leave the website running in the background, or you already have it running in another terminal, use `exec` instead of `run`:
 
 ```shell
-export FLASK_ENV=development
+docker compose up -d
+docker compose exec web flask shell
 ```
 
-4. Run the shell:
-
-```shell
-flask shell
-```
-
-And you should be good to go! The functions listed below should be available for use, and the section below contains some example use cases for the shell.
-
 ???+ tip
     To exit from the shell, type `exit()` then ++enter++.
 

docs/src/setup.md

@@ -17,16 +17,13 @@ Install all of the following programs onto your computer:
 - [uv](https://docs.astral.sh/uv/#getting-started)
 - [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) (first time setup guide [here](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup))
 - [Docker Desktop](https://www.docker.com/products/docker-desktop/)
+- [A Github account](https://github.com/). The GitHub account should have the same email as the one registered to your `git config --global user.email` that you set in the first time git setup. You also need to set up SSH with Github ([instructions here](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account)).
 
 **Recommended:**
 
 - A good text editor or IDE, such as [PyCharm](https://www.jetbrains.com/pycharm/) (which is powerful but bulky and geared toward advanced users).
 - [Heroku CLI](https://devcenter.heroku.com/articles/heroku-cli) _(required for remote deployment to Heroku.)_
 
-**Other:**
-
-- It is strongly recommend that you create a [GitHub account](https://github.com/) if you haven't done so already. The GitHub account should have the same email as the one registered to your `git config --global user.email` that you set in the first time git setup.
-
 ## Download and Setup the Code Base
 
 The flagging website is open source; the whole website's source code is available on GitHub. This section of the setup guide shows you the preferred way to install it and set up the code on a local computer.
@@ -40,9 +37,9 @@ The flagging website is open source; the whole website's source code is availabl
 3. Run the following to download your fork and setup the connection to the upstream remote. Replace `YOUR_USERNAME_HERE` (in the first line) with your actual GitHub username.
 
 ```shell
-git clone https://github.com/YOUR_USERNAME_HERE/flagging/
+git clone git@github.com:YOUR_USERNAME_HERE/flagging.git
 cd flagging
-git remote add upstream https://github.com/codeforboston/flagging.git
+git remote add upstream git@github.com:codeforboston/flagging.git
 git fetch upstream
 ```
 
@@ -61,13 +58,17 @@ cp -n .env.example .env
 
     If you do not have HOBOlink credentials, please turn on demo mode by setting `USE_MOCK_DATA=true`.
 
-    **(Optional)** If you'd like, create a Mapbox access token and add it to your `.env`: https://www.mapbox.com/ If you don't do this, the map will not fully render.
+The steps below marked **Optional** are not required to get the website to run, but will improve the experience.
+
+6. **(Optional)** If you'd like, create a Mapbox access token and add it to your `.env`: [https://www.mapbox.com/](https://www.mapbox.com/) If you don't do this, the map will not fully render.
+
+The steps below marked **Very optional** are generally _not_ recommended unless you have a good reason you want to do them.
 
-    **(Very optional)** If you'd like, connect to Sentry via the `SENTRY_DSN` and `SENTRY_ENVIRONMENT` env vars: https://sentry.io/
+7. **(Very optional)** If you'd like, connect to Sentry via the `SENTRY_DSN` and `SENTRY_ENVIRONMENT` env vars: [https://sentry.io/](https://sentry.io/)
 
-    **(Very optional)** You can also set up `https` and run that way. Create a certfile and key via the command `./run ssl-cert`, and add `CERTFILE=server.crt`, `KEYFILE=server.key`, and `PORT=443` to your `.env`. However this will require some additional finagling as your browser will not by default trust self-signed certs, so it's not recommended for most users.
+8. **(Very optional)** You can also set up `https` and run that way. Create a certfile and key via the command `./run ssl-cert`, and add `CERTFILE=server.crt`, `KEYFILE=server.key`, and `PORT=443` to your `.env`. However this will require some additional finagling as your browser will not by default trust self-signed certs, so it's not recommended for most users.
 
-    **(Very optional)** You can also set up Twitter/X credentials and send tweets. However, right now we do not use Twitter/X; this functionality is effectively deprecated.
+9. **(Very optional)** You can also set up Twitter/X credentials and send tweets. However, right now we do not use Twitter/X; this functionality is effectively deprecated.
 
 ## Run the Website Locally
 

requirements.in

@@ -1,3 +1,7 @@
+# Use this command:
+# >>> uv pip compile requirements.in -o requirements.txt
+# To compile the below dependencies.
+
 # ==========================================================
 # Core
 # ==========================================================