workshop.qmd

---
jupyter: python3
---
# Union.ai Serverless Workshop

Welcome to the Union.ai Serverless Workshop! In this workshop, we will cover:

1. Setup workspace and connect to Union.ai Serverless
2. Launch a simple ML workflow & get familiar with the Python SDK
3. LLM tracking workflow using HuggingFace and Weights & Bias.
4. Explore models using Union hosted VSCode

## Setup Workspace

1. If you are not signed into Google, sign in by clicking the "Sign in" on the upper right of this page.
2. If you have not already, sign up for Union Serverless at: https://signup.union.ai/?group=workshop
3. Navigate to https://serverless-1.us-east-2.s.union.ai/

```{python}
try:
    import google.colab
    IN_COLAB = True
except ImportError:
    IN_COLAB = False

if IN_COLAB:
    !git clone --depth 1 https://github.com/unionai-oss/llm-tracking-workshop.git
    %cd llm-tracking-workshop
    %pip install -r dev/requirements.txt
```

## Authenticate on Codespaces or Colab

To authenticate with Severless run, and go to the link provided to authenticate your CLI.

```{python}
!unionai create login device-flow
```

To make sure everything is working, run this sample workflow:

```{python}
!unionai run --remote workflows/starter.py main
```

Go to the link provided to see your execution on Union!

## ML Workflow

In this section, we will launch a simple ML workflow and get familiar with the Python SDK, `flytekit`. If you are on Codespaces, you can see the `workflows/ml_workflow.py` in the IDE. On Colab, go [to this link](https://github.com/unionai-oss/llm-tracking-workshop/blob/main/workflows/ml_workflow.py) to see the code.

One benefit of `flytekit` is that you can run the workflow locally as well:

```{python}
!unionai run workflows/ml_workflow.py main
```

And to run [the workflow](https://github.com/unionai-oss/llm-tracking-workshop/blob/main/workflows/ml_workflow.py) on Union:

```{python}
!unionai run --remote workflows/ml_workflow.py main
```

In this section, we highlight the following features in the SDK and UI:

- `@task`, `@workflow` in SDK
- `ImageSpec`
- Declarative infrastructure
- Caching
- `FlyteDeck`

## LLM Workflow

In this section, we run a LLM workflow using Hugging Face and Weights and Biases. If you are on Codespaces, you can see the `workflows/llm_tracking.py` in the IDE. On Colab, go [to this link](https://github.com/unionai-oss/llm-tracking-workshop/blob/main/workflows/llm_tracking.py) to see the code.

First, go to [https://wandb.ai/authorize](https://wandb.ai/authorize) to log into your Weights and Biases account and get an API key. If you are creating a new Weights and Biases account, then it's simplier to sign up with a **Personal** account.

Run the following command to create a secret on Union and paste your wandb API key:

```{python}
!unionai create secret wandb_api_key
```

To see all your secrets run:

```{python}
!unionai get secret
```

If you are running into an issue with the secrets, uncomment the following code to delete the secret and try again:

```{python}
# !unionai delete secret wandb_api_key
```

With the secret define, you can now run [the workflow](https://github.com/unionai-oss/llm-tracking-workshop/blob/main/workflows/llm_tracking.py) on Union:

```{python}
!unionai run --remote workflows/llm_tracking.py main \
    --wandb_project unionai-serverless-demo \
    --model distilbert-base-uncased
```

Run the following to get the execution id of the workflow we just launched.

```{python}
from utils import get_recent_execution_id
from unionai.remote import UnionRemote

remote = UnionRemote()
EXECUTION_ID = get_recent_execution_id("llm_tracking.main", remote)

# Make sure the execution ID matches the one above
print(EXECUTION_ID)
```

Models you can use are:
- `bert-base-cased`
- `distilbert-base-uncased`
- `roberta-base`

In this section, we highlight the following code and UI features:

- Secrets API
- UI to trigger workflow
- Task level metrics for GPUs
- Error handling

## VSCode Integration

In this section, we will launch a VSCode instance hosted by Union to interact with the LLM trained by the previous workflow. To get the URI for our model, copy and paste the execution id into the following cell:

```{python}
execution = remote.fetch_execution(name=EXECUTION_ID)
llm_uri = execution.outputs["o1"].remote_source

print(llm_uri)
```

Launch the vscode environment by running:

```{python}
from utils import launch_vscode, stop_vscode

vscode_execution_id = launch_vscode(llm_uri, remote)
```

In the VSCode environment:
1. Open the `interact_with_model.py` file and click on `Run Cell` button at the top of file.
2. There are reviews in `interact_with_model.py` that you can pass to your model.

We will highlight how to get the model in Union's VSCode enviornment and how to use Jupyter to interact query the model.

To stop the VSCode environment, uncomment the following and run it:

```{python}
# stop_vscode(vscode_execution_id, remote)
```

Thank you for attending this workshop!