diff --git "a/.gitbook/assets/Screenshot 2024-10-30 at 1.27.20\342\200\257PM (1).png" "b/.gitbook/assets/Screenshot 2024-10-30 at 1.27.20\342\200\257PM (1).png" new file mode 100644 index 0000000..b68fdc8 Binary files /dev/null and "b/.gitbook/assets/Screenshot 2024-10-30 at 1.27.20\342\200\257PM (1).png" differ diff --git "a/.gitbook/assets/Screenshot 2024-10-30 at 1.27.20\342\200\257PM.png" "b/.gitbook/assets/Screenshot 2024-10-30 at 1.27.20\342\200\257PM.png" new file mode 100644 index 0000000..b68fdc8 Binary files /dev/null and "b/.gitbook/assets/Screenshot 2024-10-30 at 1.27.20\342\200\257PM.png" differ diff --git "a/.gitbook/assets/Screenshot 2024-10-30 at 1.28.00\342\200\257PM.png" "b/.gitbook/assets/Screenshot 2024-10-30 at 1.28.00\342\200\257PM.png" new file mode 100644 index 0000000..336cf85 Binary files /dev/null and "b/.gitbook/assets/Screenshot 2024-10-30 at 1.28.00\342\200\257PM.png" differ diff --git "a/.gitbook/assets/Screenshot 2024-10-30 at 1.35.22\342\200\257PM.png" "b/.gitbook/assets/Screenshot 2024-10-30 at 1.35.22\342\200\257PM.png" new file mode 100644 index 0000000..8b0205d Binary files /dev/null and "b/.gitbook/assets/Screenshot 2024-10-30 at 1.35.22\342\200\257PM.png" differ diff --git a/SUMMARY.md b/SUMMARY.md index 08e9e5e..447765e 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -19,30 +19,31 @@ * [Building a Multi-Modal Copilot](guides/copilot/building-a-multi-modal-copilot.md) * [πŸ“’ Broadcast Messages (via web or API)](guides/copilot/broadcast-messages-via-web-or-api.md) * [Frequently Asked Questions about AI Copilot](guides/copilot/frequently-asked-questions-about-ai-copilot.md) -* [🎞️ How to create AI Animations?](guides/how-to-create-ai-animations.md) -* [πŸ–ΌοΈ Create an AI Image with text](guides/create-an-ai-image-with-text/README.md) - * [AI Image Prompting](https://docs.google.com/presentation/d/1RaoMP0l7FnBZovDAR42zVmrUND9W5DW6eWet-pi6kiE/edit#slide=id.p) - * [API Tips for AI Image Generator](guides/create-an-ai-image-with-text/api-tips-for-ai-image-generator.md) * [πŸ‘„ How to use AI Lip Sync Generator?](guides/how-to-use-ai-lip-sync-generator/README.md) * [Lip Sync Animation Generator (WITH AUDIO FILES)](guides/how-to-use-ai-lip-sync-generator/lip-sync-animation-generator-with-audio-files.md) * [LipSync videos with Custom Voices](guides/how-to-use-ai-lip-sync-generator/lipsync-videos-with-custom-voices.md) * [Set up your API for Lipsync with Local Folders](guides/how-to-use-ai-lip-sync-generator/set-up-your-api-for-lipsync-with-local-folders.md) * [Tips to create great HD lipsync output](guides/how-to-use-ai-lip-sync-generator/tips-to-create-great-hd-lipsync-output.md) -* [🀳 How to make amazing AI Art QR Codes?](guides/how-to-make-amazing-ai-art-qr-codes/README.md) - * [API tips on AI Art QR Codes](guides/how-to-make-amazing-ai-art-qr-codes/api-tips-on-ai-art-qr-codes.md) -* [πŸ“Έ AI Photo Editor](guides/ai-photo-editor/README.md) - * [Build your avatar with AI](guides/ai-photo-editor/build-your-avatar-with-ai.md) -* [🌐 How to create SEO-Optimized content with AI?](guides/how-to-create-seo-optimized-content-with-ai.md) -* [πŸ” Generate β€œPeople Also Ask” SEO Content](guides/generate-people-also-ask-seo-content.md) -* [πŸ“Š How to create language evaluation for ASR?](guides/how-to-create-language-evaluation-for-asr.md) +* [πŸ—£οΈ How to use ASR?](guides/how-to-use-asr/README.md) + * [πŸ“Š How to create language evaluation for ASR?](guides/how-to-use-asr/how-to-create-language-evaluation-for-asr.md) +* [How to use Compare AI Translations?](guides/how-to-use-compare-ai-translations/README.md) + * [Google Translate Glossary](guides/how-to-use-compare-ai-translations/google-translate-glossary.md) * [How does RAG-based document search work?](guides/how-does-rag-based-document-search-work.md) * [🧩 How to use Gooey Functions?](guides/how-to-use-gooey-functions/README.md) * [✨ LLM-enabled Functions](guides/how-to-use-gooey-functions/llm-enabled-functions.md) -* [How to use Compare AI Translations?](guides/how-to-use-compare-ai-translations/README.md) - * [Google Translate Glossary](guides/how-to-use-compare-ai-translations/google-translate-glossary.md) * [βš–οΈ Understanding Bulk Runner and Evaluation](guides/understanding-bulk-runner-and-evaluation/README.md) * [πŸ’ͺ How to set up Bulk Runner?](guides/understanding-bulk-runner-and-evaluation/how-to-set-up-bulk-runner.md) * [πŸ•΅οΈβ€β™€οΈ How to set up Evaluations?](guides/understanding-bulk-runner-and-evaluation/how-to-set-up-evaluations.md) +* [🎞️ How to create AI Animations?](guides/how-to-create-ai-animations.md) +* [🀳 How to make amazing AI Art QR Codes?](guides/how-to-make-amazing-ai-art-qr-codes/README.md) + * [API tips on AI Art QR Codes](guides/how-to-make-amazing-ai-art-qr-codes/api-tips-on-ai-art-qr-codes.md) +* [πŸ–ΌοΈ Create an AI Image with text](guides/create-an-ai-image-with-text/README.md) + * [AI Image Prompting](https://docs.google.com/presentation/d/1RaoMP0l7FnBZovDAR42zVmrUND9W5DW6eWet-pi6kiE/edit#slide=id.p) + * [API Tips for AI Image Generator](guides/create-an-ai-image-with-text/api-tips-for-ai-image-generator.md) +* [πŸ“Έ AI Photo Editor](guides/ai-photo-editor/README.md) + * [Build your avatar with AI](guides/ai-photo-editor/build-your-avatar-with-ai.md) +* [πŸ” Generate β€œPeople Also Ask” SEO Content](guides/generate-people-also-ask-seo-content.md) +* [🌐 How to create SEO-Optimized content with AI?](guides/how-to-create-seo-optimized-content-with-ai.md) ## πŸ˜‡ CONTRIBUTING diff --git a/guides/how-to-use-asr/README.md b/guides/how-to-use-asr/README.md new file mode 100644 index 0000000..cdba7cc --- /dev/null +++ b/guides/how-to-use-asr/README.md @@ -0,0 +1,85 @@ +--- +description: A simple guide on how to use Automatic Speech Recognition +--- + +# πŸ—£οΈ How to use ASR? + +We find speech recognition and translations important when we create AI workflows for frontline workers, traders, and impact organizations. + +In AI copilot scenarios, we found that users prefer to send queries by voice rather than text. This could mean: + +* voice notes to Whatsapp and web copilots +* Voice-based IVR in low internet coverage areas + +So as part of our [AI Workflow Standards](https://blog.gooey.ai/workflow-standards), we now host over 15 ASR models, that can be chained into the AI Copilot workflows. + +Here is a simple guide to use the ASR Workflow + +### Step 1: Add your audio samples + +* Head to the Gooey.AI Speech Recognition and Translation Workflow: [https://gooey.ai/speech/](https://gooey.ai/speech/) +* Click on the "Clear all" Button and upload your audio file via a local folder or a link + +{% hint style="info" %} +If you are using a link you can use: + +* a hosted media link +* a google drive link of the audio file +* youtube video link +{% endhint %} + + + +
+ +
+ + + +
+ +
+ +### Step 2: Select the Speech to Text provider + +* Select the language in "Speech-to-Text Provider" from the dropdown provided. + +{% hint style="info" %} +Use the "Filter by Language" dropdown, if you are unsure which models will work with your source language +{% endhint %} + +
+ +### Step 3: Select your translation model + +* Click on the "Translate" checkbox +* Select the translation model of your choice + +### Step 4: Hit "Run" + +* Click on the "Run" button + +### FAQS + +Q: How do I test the ASR models that can transcribe Swahili? + +A: Use the "Filter by Language" dropdown, if you are unsure which models will work with your source language. + +Q: In the translate section I can see "Google Translate" and "GhanaNLP", which model should I use? + +A: If you are translating an African Language you can test if GhanaNLP is a better choice. GhanaNLP Machine Translation supports: Twi, Ewe, Ga, Fanti, Yoruba, Dagbani, Kikuyu, Fra fra, Luo (Kenya, Tanzania), Meru, Kusaal + +Q: I have tested a few models, but I want to evaluate a larger dataset without using the API, is that possible? + +A: Yes! It is very easy to set up large-scale ASR evaluations in Gooey! Here is the guide for: + +
How to create language evaluation for ASR?how-to-create-language-evaluation-for-asr.md
+ + + +Q: What is the "Prompt" section when I choose GPT4o-Audio? + +A: GPT4o-Audio is an LLM based transcription model, the prompt section will allow you to output the transcribed audio in more specific ways. For example, if you input a Hindi audio sample, you can say "Translate the Hindi recording as accurately as possible". This will use the LLM directly to make the translation. You could also use it in other innovative ways like "Summarize the Hindi recording to English in bullet points" which could give you the salient points of the recording directly. Like the example here: + +{% embed url="https://gooey.ai/speech/?run_id=1rnoo9r71o3q&uid=fm165fOmucZlpa5YHupPBdcvDR02" %} + diff --git a/guides/how-to-create-language-evaluation-for-asr.md b/guides/how-to-use-asr/how-to-create-language-evaluation-for-asr.md similarity index 65% rename from guides/how-to-create-language-evaluation-for-asr.md rename to guides/how-to-use-asr/how-to-create-language-evaluation-for-asr.md index f48324b..126001a 100644 --- a/guides/how-to-create-language-evaluation-for-asr.md +++ b/guides/how-to-use-asr/how-to-create-language-evaluation-for-asr.md @@ -28,7 +28,7 @@ There are several components to test: #### Also see: -
🏎️ Global Language Understanding for AIsGlobal Language Understanding for AIsgooey.ai - cute robots racing vintage painted magazine advertisement muted colorful illustration.png
πŸ—£οΈ Check out our Hindi ASR Evaluationhttps://gooey.ai/bulk/compare-hindi-speech-recognition-hkgs8120p11t/gooey.ai - cute vintage poster style illustration of a small group of indian kids learning hindi (1).png
+
🏎️ Global Language Understanding for AIsGlobal Language Understanding for AIsgooey.ai - cute robots racing vintage painted magazine advertisement muted colorful illustration.png
πŸ—£οΈ Check out our Hindi ASR Evaluationhttps://gooey.ai/bulk/compare-hindi-speech-recognition-hkgs8120p11t/gooey.ai - cute vintage poster style illustration of a small group of indian kids learning hindi (1).png
## Getting Started @@ -39,7 +39,7 @@ There are several components to test: 3. Add the human-created transcription for each sample 4. Add the English translation for each sample -
+
### Step 1 - Select the ASR Models @@ -49,37 +49,37 @@ Head to our bulk and eval workflow. In the example, we have already pre-filled the various models that can be tested. You can choose the ones you want to run by selecting it in the dropdown. -

Select the models you want to evaluate your audio samples on

+

Select the models you want to evaluate your audio samples on

### Step 2 - Add your CSV/Google Sheets Upload your CSV/Google Sheet from [Step 0](how-to-create-language-evaluation-for-asr.md#step-0-prepare-your-data). In this example, we have used a Google Sheet of 10 Audio Samples with transcripts and translations. A preview of your sheet will appear once it is correctly uploaded. -
+
### Step 3 - Select the input column Select the column in the input from the dropdown box. The outputs will appear as various columns. In this example, it will be the "audios" column -
+
### Step 4 - Select the pre-built evaluator In the "Evaluation Workflows" section select the "Speech Recognition Model Evaluator". -
+
### Step 5 - Hit Submit Once you hit submit, the selected ASR model workflows (see [Step 1](how-to-create-language-evaluation-for-asr.md#step-1-select-the-asr-models)) will run for each audio file in the sheet (see [Step 2](how-to-create-language-evaluation-for-asr.md#step-2-add-your-csv-google-sheets)). An output CSV will be generated on the right-hand side of the page. -

The outputs will appear in a table format on the right of the page, the output will appear in new columns after your originally populated columns

+

The outputs will appear in a table format on the right of the page, the output will appear in new columns after your originally populated columns

After the runs are complete, the selected Evaluator (see [Step 4](how-to-create-language-evaluation-for-asr.md#step-4-select-the-pre-built-evaluator)), will compare the ASR model outputs to the human-generated translations. It will assess and rate how accurately each model has translated the audio sample. A bar graph with the performance will appear once the entire evaluation is complete. -

After the evaluation is complete, table and a bar graph will show the evalution scores

+

After the evaluation is complete, table and a bar graph will show the evalution scores

@@ -89,7 +89,7 @@ A bar graph with the performance will appear once the entire evaluation is compl A: Arrange the audio sample link in the first column, for each audio link add transcriptions and translations in the respective row. -

Screenshot of audio sample links with trasncriptions and english translation

+

Screenshot of audio sample links with trasncriptions and english translation

#### Q: What is the ideal length of the recording?