Skip to content

Commit

Permalink
GITBOOK-196: No subject
Browse files Browse the repository at this point in the history
  • Loading branch information
Ambika Joshi authored and gitbook-bot committed Jan 2, 2025
1 parent 17a94c5 commit 22d5d8e
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 0 deletions.
7 changes: 7 additions & 0 deletions .gitbook/includes/bulk-run-credit-warning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
title: Bulk Run Credit Warning
---

{% hint style="warning" %}
NOTE: If your "Saved Workflow" costs 4 credits per run, in the Bulk Runner and Evaluation Tools each of your CSV rows. So, if you have 100 CSV rows with data, your credit consumption will be `100 row * 4 Credits = 400 Credits.` 
{% endhint %}
4 changes: 4 additions & 0 deletions guides/understanding-bulk-runner-and-evaluation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ Here are the **quickstart guides** for Bulk Runner and Evaluation:

<table data-view="cards"><thead><tr><th></th><th data-hidden data-card-target data-type="content-ref"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td>How to set up Bulk Runner?</td><td><a href="how-to-set-up-bulk-runner.md">how-to-set-up-bulk-runner.md</a></td><td><a href="../../.gitbook/assets/Bulk Runner.png_400x400.png">Bulk Runner.png_400x400.png</a></td></tr><tr><td>How to set up Evaluation?</td><td><a href="how-to-set-up-evaluations.md">how-to-set-up-evaluations.md</a></td><td><a href="../../.gitbook/assets/W.I.9.png_400x400.png">W.I.9.png_400x400.png</a></td></tr></tbody></table>

{% include "../../.gitbook/includes/bulk-run-credit-warning.md" %}

## How Does Bulk Runner Work? <a href="#r7so22ymyyn2" id="r7so22ymyyn2"></a>

### Bulk Run Overview <a href="#id-2v61ngoeupi4" id="id-2v61ngoeupi4"></a>
Expand All @@ -41,6 +43,8 @@ This diagram details the process of generating AI-driven answers to a set of tes

![](<../../.gitbook/assets/Understanding Bulk run and Evaluations.jpg>)



**1. Test Question Set**

The process begins with a curated set of questions. Examples of such questions include:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ In this example scenario, we are setting up a simple bulk run to check regressio

<table data-view="cards"><thead><tr><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Check out the example run here: BULK RUNNER (Regression Only)</strong></td><td><a href="https://gooey.ai/bulk/farmerchat-bulk-evaluator-regression-only-ggzy9gld1eae/">https://gooey.ai/bulk/farmerchat-bulk-evaluator-regression-only-ggzy9gld1eae/</a></td></tr><tr><td><strong>Check out the example run here: BULK RUNNER (Bulk and Evaluation)</strong></td><td><a href="https://gooey.ai/bulk/farmerchat-bulk-evaluator-gpt-4o-mixtral-claude-vs-gemini-pro-15-b0o8aos3rj8y/">https://gooey.ai/bulk/farmerchat-bulk-evaluator-gpt-4o-mixtral-claude-vs-gemini-pro-15-b0o8aos3rj8y/</a></td></tr></tbody></table>

{% include "../../.gitbook/includes/bulk-run-credit-warning.md" %}

### Step 1: Select Gooey Workflows <a href="#jmvc9vjmbif9" id="jmvc9vjmbif9"></a>

Choose the “SAVED” run from Gooey.AI Workflows that you would like to use.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ In this example scenario, we are comparing and evaluating the quality of the ans

<table data-view="cards"><thead><tr><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Check out the example run here: Evaluation only</strong> </td><td><a href="https://gooey.ai/eval/copilot-evaluator-artpuhzwvily/">https://gooey.ai/eval/copilot-evaluator-artpuhzwvily/</a></td></tr></tbody></table>

{% include "../../.gitbook/includes/bulk-run-credit-warning.md" %}

### Step 1: Select Gooey Workflows to evaluate <a href="#mj1hmvoaayxg" id="mj1hmvoaayxg"></a>

Choose the “SAVED” run from Gooey.AI Workflows that you would like to use.
Expand Down

0 comments on commit 22d5d8e

Please sign in to comment.