From 12f63e7bfad1a824f7aef42e90c5b9ffd0b26b6b Mon Sep 17 00:00:00 2001
From: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>
Date: Mon, 30 Oct 2023 14:13:46 -0700
Subject: [PATCH 1/7] Adding call automation ai sample to th repo

---
 callautomation-openai-sample/.gitignore    |  30 ++++
 callautomation-openai-sample/README.md     |  56 ++++++++
 callautomation-openai-sample/package.json  |  29 ++++
 callautomation-openai-sample/src/app.ts    | 154 +++++++++++++++++++++
 callautomation-openai-sample/tsconfig.json |  13 ++
 5 files changed, 282 insertions(+)
 create mode 100644 callautomation-openai-sample/.gitignore
 create mode 100644 callautomation-openai-sample/README.md
 create mode 100644 callautomation-openai-sample/package.json
 create mode 100644 callautomation-openai-sample/src/app.ts
 create mode 100644 callautomation-openai-sample/tsconfig.json

diff --git a/callautomation-openai-sample/.gitignore b/callautomation-openai-sample/.gitignore
new file mode 100644
index 00000000..1f45ab17
--- /dev/null
+++ b/callautomation-openai-sample/.gitignore
@@ -0,0 +1,30 @@
+# Ignore node_modules directory
+node_modules/
+
+# Ignore environment variables file
+.env
+
+# Ignore build output directory
+dist/
+build/
+public/assets/
+
+# Ignore IDE/Editor-specific files
+.vscode/
+.vs
+.idea/
+
+# Ignore user-specific configuration files
+.npmrc
+.gitconfig
+
+# Ignore log files
+*.log
+
+# Ignore OS-generated files
+.DS_Store
+Thumbs.db
+
+# Ignore package lock files
+package-lock.json
+yarn.lock
\ No newline at end of file
diff --git a/callautomation-openai-sample/README.md b/callautomation-openai-sample/README.md
new file mode 100644
index 00000000..6905354f
--- /dev/null
+++ b/callautomation-openai-sample/README.md
@@ -0,0 +1,56 @@
+|page_type|languages|products
+|---|---|---|
+|sample|<table><tr><td>Typescript</tr></td></table>|<table><tr><td>azure</td><td>azure-communication-services</td></tr></table>|
+
+# Call Automation - Quick Start Sample
+
+This sample application shows how the Azure Communication Services - Call Automation SDK can be used with Azure OpenAI Service to enable intelligent conversational agents. It answers an inbound call, does a speech recognition with the recognize API and Cognitive Services, uses OpenAi Services with the input speech and responds to the caller through Cognitive Services' Text to Speech. This sample application configured for accepting input speech until the caller terminates the call or a long silence is detected
+
+## Prerequisites
+
+- Create an Azure account with an active subscription. For details, see [Create an account for free](https://azure.microsoft.com/free/)
+- [Visual Studio Code](https://code.visualstudio.com/download) installed
+- [Node.js](https://nodejs.org/en/download) installed
+- Create an Azure Communication Services resource. For details, see [Create an Azure Communication Resource](https://docs.microsoft.com/azure/communication-services/quickstarts/create-communication-resource). You will need to record your resource **connection string** for this sample.
+- Get a phone number for your new Azure Communication Services resource. For details, see [Get a phone number](https://learn.microsoft.com/en-us/azure/communication-services/quickstarts/telephony/get-phone-number?tabs=windows&pivots=programming-language-csharp)
+- Create an Azure Cognitive Services resource. For details, see Create an Azure Cognitive Services Resource.
+- An Azure OpenAI Resource and Deployed Model. See [instructions](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal).
+
+## Before running the sample for the first time
+
+1. Open an instance of PowerShell, Windows Terminal, Command Prompt or equivalent and navigate to the directory that you would like to clone the sample to.
+2. git clone `https://github.com/Azure-Samples/communication-services-javascript-quickstarts.git`.
+3. cd into the `callautomation-openai-sample` folder.
+4. From the root of the above folder, and with node installed, run `npm install`
+
+### Setup and host your Azure DevTunnel
+
+[Azure DevTunnels](https://learn.microsoft.com/en-us/azure/developer/dev-tunnels/get-started?tabs=windows) is an Azure service that enables you to share local web services hosted on the internet. Use the commands below to connect your local development environment to the public internet. This creates a tunnel with a persistent endpoint URL and which allows anonymous access. We will then use this endpoint to notify your application of calling events from the ACS Call Automation service.
+
+```bash
+devtunnel create --allow-anonymous
+devtunnel port create -p 8080
+devtunnel host
+```
+### Add a Managed Identity to the ACS Resource that connects to the Cognitive Services Resource
+
+Follow the instructions in the [documentation](https://learn.microsoft.com/en-us/azure/communication-services/concepts/call-automation/azure-communication-services-azure-cognitive-services-integration).
+
+### Configuring application
+
+Open the `.env` file to configure the following settings
+
+1. `CONNECTION_STRING`: Azure Communication Service resource's connection string.
+2. `CALLBACK_URI`: Base url of the app. (For local development replace the dev tunnel url)
+3. `COGNITIVE_SERVICE_ENDPOINT`: Azure Cognitive Service endpoint
+4. `AZURE_OPENAI_SERVICE_KEY`: Azure Open AI service key
+5. `AZURE_OPENAI_SERVICE_ENDPOINT`: Azure Open AI endpoint
+6. `AZURE_OPENAI_DEPLOYMENT_MODEL_NAME`: Azure Open AI deployment name
+
+### Run app locally
+
+1. Open a new Powershell window, cd into the `callautomation-openai-sample` folder and run `npm run dev`
+2. Browser should pop up with the below page. If not navigate it to `http://localhost:8080/`
+3. Register an EventGrid Webhook for the IncomingCall Event that points to your DevTunnel URI. Instructions [here](https://learn.microsoft.com/en-us/azure/communication-services/concepts/call-automation/incoming-call-notification).
+
+Once that's completed you should have a running application. The best way to test this is to place a call to your ACS phone number and talk to your intelligent agent.
\ No newline at end of file
diff --git a/callautomation-openai-sample/package.json b/callautomation-openai-sample/package.json
new file mode 100644
index 00000000..bdf9777f
--- /dev/null
+++ b/callautomation-openai-sample/package.json
@@ -0,0 +1,29 @@
+{
+  "name": "callautomation_openai",
+  "version": "1.0.0",
+  "description": "",
+  "main": "index.js",
+  "scripts": {
+    "build": "tsc",
+    "dev": "nodemon ./src/app.ts"
+  },
+  "keywords": [],
+  "author": "",
+  "license": "ISC",
+  "dependencies": {
+    "@azure/communication-call-automation": "^1.1.0-beta.2",
+    "@azure/communication-common": "^2.2.0",
+    "@azure/eventgrid": "^4.12.0",
+    "@azure/openai": "^1.0.0-beta.7",
+    "@types/express": "^4.17.17",
+    "@types/node": "^20.2.1",
+    "dotenv": "^16.3.1",
+    "express": "^4.18.2",
+    "uuid": "^9.0.1"
+  },
+  "devDependencies": {
+    "nodemon": "^2.0.22",
+    "ts-node": "^10.9.1",
+    "typescript": "^5.0.4"
+  }
+}
diff --git a/callautomation-openai-sample/src/app.ts b/callautomation-openai-sample/src/app.ts
new file mode 100644
index 00000000..93d2866b
--- /dev/null
+++ b/callautomation-openai-sample/src/app.ts
@@ -0,0 +1,154 @@
+import { config } from 'dotenv';
+import express, { Application } from 'express';
+import { createIdentifierFromRawId } from "@azure/communication-common";
+import { CallAutomationClient, CallConnection, AnswerCallOptions, CallMedia, TextSource, AnswerCallResult, CallMediaRecognizeSpeechOptions } from "@azure/communication-call-automation";
+import { v4 as uuidv4 } from 'uuid';
+import { AzureKeyCredential, OpenAIClient } from '@azure/openai';
+config();
+
+const PORT = process.env.PORT;
+const app: Application = express();
+app.use(express.json());
+
+let callConnectionId: string;
+let callConnection: CallConnection;
+let acsClient: CallAutomationClient;
+let openAiClient : OpenAIClient;
+let answerCallResult: AnswerCallResult;
+let callerId: string;
+let callMedia: CallMedia;
+
+async function createAcsClient() {
+	const connectionString = process.env.CONNECTION_STRING || "";
+	acsClient = new CallAutomationClient(connectionString);
+	console.log("Initialized ACS Client.");
+}
+
+async function createOpenAiClient() {
+	const openAiServiceEndpoint = process.env.AZURE_OPENAI_SERVICE_ENDPOINT || "";
+	const openAiKey = process.env.AZURE_OPENAI_SERVICE_KEY || "";
+	openAiClient = new OpenAIClient(
+		openAiServiceEndpoint, 
+		new AzureKeyCredential(openAiKey)
+	  );
+	console.log("Initialized Open Ai Client.");
+}
+
+async function hangUpCall() {
+	callConnection.hangUp(true);
+}
+
+async function startRecognizing(callMedia: CallMedia, callerId: string, message: string, context: string){
+	const play : TextSource = { text: message, voiceName: "en-US-NancyNeural", kind: "textSource"}
+	const recognizeOptions: CallMediaRecognizeSpeechOptions = { 
+		endSilenceTimeoutInSeconds: 1, 
+		playPrompt: play, 
+		initialSilenceTimeoutInSeconds: 15, 
+		interruptPrompt: false, 
+		operationContext: context, 
+		kind: "callMediaRecognizeSpeechOptions",
+	}; 
+
+	const targetParticipant = createIdentifierFromRawId(callerId);
+	await callMedia.startRecognizing(targetParticipant, recognizeOptions)
+}
+
+async function handlePlay(callConnectionMedia:CallMedia){
+	const textContent = 'Goodbye';
+	const play : TextSource = { text: textContent, voiceName: "en-US-NancyNeural", kind: "textSource"}
+	await callConnectionMedia.playToAll([play]);
+}
+
+async function getChatGptResponse(speechInput: string){
+	const deploymentName = process.env.AZURE_OPENAI_DEPLOYMENT_MODEL_NAME;
+	const messages = [
+		{ role: "system", content: "You are a helpful assistant." },
+		{ role: "user", content: `In less than 200 characters: respond to this question: ${speechInput}` },
+	];
+
+	const response = await openAiClient.getChatCompletions(deploymentName, messages);
+	const responseContent = response.choices[0].message.content;
+	return responseContent;
+}
+
+app.post("/api/incomingCall", async (req: any, res:any)=>{
+	console.log(`Received incoming call event - data --> ${JSON.stringify(req.body)} `);
+	const event = req.body[0];
+	try{
+		const eventData = event.data;
+		if (event.eventType === "Microsoft.EventGrid.SubscriptionValidationEvent") {
+			console.log("Received SubscriptionValidation event");
+			res.status(200).json({
+				validationResponse: eventData.validationCode,
+			});
+		}
+
+		callerId = eventData.from.rawId;
+		const uuid = uuidv4();
+		const callbackUri = `${process.env.CALLBACK_URI}/api/callbacks/${uuid}?callerId=${callerId}`;
+		const incomingCallContext = eventData.incomingCallContext;
+		console.log(`Cognitive service endpoint:  ${process.env.COGNITIVE_SERVICE_ENDPOINT.trim()}`)
+		let cognitiveServiceEndpoint = process.env.COGNITIVE_SERVICE_ENDPOINT;
+		const answerCallOptions: AnswerCallOptions = { cognitiveServicesEndpoint: cognitiveServiceEndpoint }; 
+		answerCallResult = await acsClient.answerCall(incomingCallContext, callbackUri, answerCallOptions);
+		console.log(`Incoming call answered. Cognitive Service Url :${cognitiveServiceEndpoint}, 
+		Callback Uri: ${callbackUri},  CallConnection Id: ${answerCallResult.callConnectionProperties.callConnectionId} `);
+		callConnection = answerCallResult.callConnection;
+		callMedia = callConnection.getCallMedia();
+	}
+	catch(error){
+		console.error("Error during the incoming call event.", error);
+	}
+});
+
+app.post('/api/callbacks/:contextId', async (req:any, res:any) => {
+	const contextId = req.params.contextId;
+  	const event = req.body[0];
+	const eventData = event.data;
+	console.log(`Received event type ${event.type}`);
+	console.log(`event type match ${event.type === "Microsoft.Communication.CallConnected"}`);
+	if(event.type === "Microsoft.Communication.CallConnected"){
+		console.log("Received CallConnected event");
+		const text = 'Hello. How can I help?';
+		startRecognizing(callMedia, callerId, text, 'GetFreeFormText');
+	}
+	else if(event.type === "Microsoft.Communication.PlayCompleted" || event.type === "Microsoft.Communication.playFailed"){
+		console.log("Received PlayCompleted event");
+		hangUpCall();
+	}
+	else if(event.type === "Microsoft.Communication.RecognizeCompleted"){
+		if(eventData.recognitionType === "speech"){
+			const speechText = eventData.speechResult.speech;
+			if(speechText !== ''){
+				console.log(`Recognized speech ${speechText}`);
+				const chatGptResponse = await getChatGptResponse(speechText);
+				startRecognizing(callMedia, callerId, chatGptResponse, 'OpenAISample')
+			}
+		}
+	}
+	else if(event.type === "Microsoft.Communication.RecognizeFailed"){
+		const resultInformation = eventData.resultInformation
+		var code = resultInformation.subCode;
+		if(code === 8510){
+			const text = 'I ve noticed that you have been silent. Are you still there?';
+			startRecognizing(callMedia, callerId, text, 'GetFreeFormText');
+		}
+		else{
+			handlePlay(callMedia);
+		}
+	}
+	else if(event.type === "Microsoft.Communication.CallDisconnected"){
+		console.log("Received CallDisconnected event");
+	}
+  });
+
+app.get('/', (req, res) => {
+	res.send('Hello ACS CallAutomation!');
+  });
+
+// Start the server
+app.listen(PORT, async () => {
+	console.log(`Server is listening on port ${PORT}`);
+	await createAcsClient();
+	await createOpenAiClient();
+});
diff --git a/callautomation-openai-sample/tsconfig.json b/callautomation-openai-sample/tsconfig.json
new file mode 100644
index 00000000..c67252ac
--- /dev/null
+++ b/callautomation-openai-sample/tsconfig.json
@@ -0,0 +1,13 @@
+{
+  "compilerOptions": {
+      "target": "ES2015",
+      "module": "commonjs",
+      "outDir": "./dist",
+      "rootDir": "./src",
+      "moduleResolution": "node",
+      "esModuleInterop": true,
+      "skipLibCheck": true,
+      "forceConsistentCasingInFileNames": true
+  },
+  "include": ["./src"]
+}
\ No newline at end of file

From d5a2663b842363a633220d5257f9bb36f0773480 Mon Sep 17 00:00:00 2001
From: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>
Date: Mon, 30 Oct 2023 14:21:02 -0700
Subject: [PATCH 2/7] removing the env file from gitignore

---
 callautomation-openai-sample/.gitignore | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/callautomation-openai-sample/.gitignore b/callautomation-openai-sample/.gitignore
index 1f45ab17..ff7e8212 100644
--- a/callautomation-openai-sample/.gitignore
+++ b/callautomation-openai-sample/.gitignore
@@ -1,9 +1,6 @@
 # Ignore node_modules directory
 node_modules/
 
-# Ignore environment variables file
-.env
-
 # Ignore build output directory
 dist/
 build/

From 286e847214dc0e08a7fd5da8da4941ab6df782ea Mon Sep 17 00:00:00 2001
From: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>
Date: Mon, 30 Oct 2023 14:28:24 -0700
Subject: [PATCH 3/7] Adding env file

---
 callautomation-openai-sample/.env | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 callautomation-openai-sample/.env

diff --git a/callautomation-openai-sample/.env b/callautomation-openai-sample/.env
new file mode 100644
index 00000000..6e5737b8
--- /dev/null
+++ b/callautomation-openai-sample/.env
@@ -0,0 +1,7 @@
+PORT=8080
+CONNECTION_STRING="<YOUR-ACS-CONNECTION_STRING>"
+CALLBACK_URI="<YOUR-DEVTUNNEL-CALLBACK_URI>"
+COGNITIVE_SERVICE_ENDPOINT="<YOUR-COGNITIVE_SERVICE_ENDPOINT>"
+AZURE_OPENAI_SERVICE_KEY = "<YOUR-AZURE_OPENAI_SERVICE_KEY>"
+AZURE_OPENAI_SERVICE_ENDPOINT="<YOUR-AZURE_OPENAI_SERVICE_ENDPOINT>"
+AZURE_OPENAI_DEPLOYMENT_MODEL_NAME="<YOUR-AZURE_OPENAI_DEPLOYMENT_MODEL_NAME>"
\ No newline at end of file

From 477dbfc67854f57d9fba73e46685f95a005169a8 Mon Sep 17 00:00:00 2001
From: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>
Date: Mon, 30 Oct 2023 14:28:46 -0700
Subject: [PATCH 4/7] reverting gitignore for env file

---
 callautomation-openai-sample/.gitignore | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/callautomation-openai-sample/.gitignore b/callautomation-openai-sample/.gitignore
index ff7e8212..1f45ab17 100644
--- a/callautomation-openai-sample/.gitignore
+++ b/callautomation-openai-sample/.gitignore
@@ -1,6 +1,9 @@
 # Ignore node_modules directory
 node_modules/
 
+# Ignore environment variables file
+.env
+
 # Ignore build output directory
 dist/
 build/

From b8d889652cf6bf3b4c548f8bfe9567f5d54dd8c7 Mon Sep 17 00:00:00 2001
From: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>
Date: Mon, 20 Nov 2023 12:49:13 -0800
Subject: [PATCH 5/7] Updated prompt and call transfer for GA version

---
 callautomation-openai-sample/.env         |   3 +-
 callautomation-openai-sample/package.json |   2 +-
 callautomation-openai-sample/src/app.ts   | 139 ++++++++++++++++++----
 3 files changed, 119 insertions(+), 25 deletions(-)

diff --git a/callautomation-openai-sample/.env b/callautomation-openai-sample/.env
index 6e5737b8..06cbea9b 100644
--- a/callautomation-openai-sample/.env
+++ b/callautomation-openai-sample/.env
@@ -4,4 +4,5 @@ CALLBACK_URI="<YOUR-DEVTUNNEL-CALLBACK_URI>"
 COGNITIVE_SERVICE_ENDPOINT="<YOUR-COGNITIVE_SERVICE_ENDPOINT>"
 AZURE_OPENAI_SERVICE_KEY = "<YOUR-AZURE_OPENAI_SERVICE_KEY>"
 AZURE_OPENAI_SERVICE_ENDPOINT="<YOUR-AZURE_OPENAI_SERVICE_ENDPOINT>"
-AZURE_OPENAI_DEPLOYMENT_MODEL_NAME="<YOUR-AZURE_OPENAI_DEPLOYMENT_MODEL_NAME>"
\ No newline at end of file
+AZURE_OPENAI_DEPLOYMENT_MODEL_NAME="<YOUR-AZURE_OPENAI_DEPLOYMENT_MODEL_NAME>"
+AGENT_PHONE_NUMBER="<YOUR-AGENT_PHONE_NUMBER>"
\ No newline at end of file
diff --git a/callautomation-openai-sample/package.json b/callautomation-openai-sample/package.json
index bdf9777f..6345f378 100644
--- a/callautomation-openai-sample/package.json
+++ b/callautomation-openai-sample/package.json
@@ -11,7 +11,7 @@
   "author": "",
   "license": "ISC",
   "dependencies": {
-    "@azure/communication-call-automation": "^1.1.0-beta.2",
+    "@azure/communication-call-automation": "^1.1.0-alpha.20231109.1",
     "@azure/communication-common": "^2.2.0",
     "@azure/eventgrid": "^4.12.0",
     "@azure/openai": "^1.0.0-beta.7",
diff --git a/callautomation-openai-sample/src/app.ts b/callautomation-openai-sample/src/app.ts
index 93d2866b..015f3f15 100644
--- a/callautomation-openai-sample/src/app.ts
+++ b/callautomation-openai-sample/src/app.ts
@@ -1,7 +1,7 @@
 import { config } from 'dotenv';
 import express, { Application } from 'express';
-import { createIdentifierFromRawId } from "@azure/communication-common";
-import { CallAutomationClient, CallConnection, AnswerCallOptions, CallMedia, TextSource, AnswerCallResult, CallMediaRecognizeSpeechOptions } from "@azure/communication-call-automation";
+import { PhoneNumberIdentifier, createIdentifierFromRawId } from "@azure/communication-common";
+import { CallAutomationClient, CallConnection, AnswerCallOptions, CallMedia, TextSource, AnswerCallResult, CallMediaRecognizeSpeechOptions, CallIntelligenceOptions, PlayOptions } from "@azure/communication-call-automation";
 import { v4 as uuidv4 } from 'uuid';
 import { AzureKeyCredential, OpenAIClient } from '@azure/openai';
 config();
@@ -18,6 +18,29 @@ let answerCallResult: AnswerCallResult;
 let callerId: string;
 let callMedia: CallMedia;
 
+const answerPromptSystemTemplate = `You are an assisant designed to answer the customer query and analyze the sentiment score from the customer tone. 
+    You also need to determine the intent of the customer query and classify it into categories such as sales, marketing, shopping, etc.
+    Use a scale of 1-10 (10 being highest) to rate the sentiment score. 
+    Use the below format, replacing the text in brackets with the result. Do not include the brackets in the output: 
+    Content:[Answer the customer query briefly and clearly in two lines and ask if there is anything else you can help with] 
+    Score:[Sentiment score of the customer tone] 
+    Intent:[Determine the intent of the customer query] 
+    Category:[Classify the intent into one of the categories]`;
+
+const helloPrompt = "Hello, thank you for calling! How can I help you today?";
+const timeoutSilencePrompt = "I’m sorry, I didn’t hear anything. If you need assistance please let me know how I can help you.";
+const goodbyePrompt = "Thank you for calling! I hope I was able to assist you. Have a great day!";
+const connectAgentPrompt = "I'm sorry, I was not able to assist you with your request. Let me transfer you to an agent who can help you further. Please hold the line and I'll connect you shortly.";
+const callTransferFailurePrompt = "It looks like all I can’t connect you to an agent right now, but we will get the next available agent to call you back as soon as possible.";
+const agentPhoneNumberEmptyPrompt = "I’m sorry, we're currently experiencing high call volumes and all of our agents are currently busy. Our next available agent will call you back as soon as possible.";
+const EndCallPhraseToConnectAgent = "Sure, please stay on the line. I’m going to transfer you to an agent.";
+
+const transferFailedContext = "TransferFailed";
+const connectAgentContext = "ConnectAgent";
+
+const agentPhonenumber = process.env.AGENT_PHONE_NUMBER;
+const chatResponseExtractPattern = /(?<=: ).*/g;
+
 async function createAcsClient() {
 	const connectionString = process.env.CONNECTION_STRING || "";
 	acsClient = new CallAutomationClient(connectionString);
@@ -53,24 +76,48 @@ async function startRecognizing(callMedia: CallMedia, callerId: string, message:
 	await callMedia.startRecognizing(targetParticipant, recognizeOptions)
 }
 
-async function handlePlay(callConnectionMedia:CallMedia){
-	const textContent = 'Goodbye';
-	const play : TextSource = { text: textContent, voiceName: "en-US-NancyNeural", kind: "textSource"}
-	await callConnectionMedia.playToAll([play]);
+function getSentimentScore(sentimentScore: string){
+	const pattern = /(\d)+/g;
+	const match = sentimentScore.match(pattern);
+	return match ? parseInt(match[0]): -1;
 }
 
-async function getChatGptResponse(speechInput: string){
+async function handlePlay(callConnectionMedia:CallMedia, textToPlay: string, context: string){
+	const play : TextSource = { text: textToPlay, voiceName: "en-US-NancyNeural", kind: "textSource"}
+	const playOptions : PlayOptions = { operationContext: context };
+	await callConnectionMedia.playToAll([play], playOptions);
+}
+
+async function detectEscalateToAgentIntent(speechInput:string) {
+	return hasIntent(speechInput, "talk to agent");
+}
+
+async function hasIntent(userQuery: string, intentDescription: string){
+	const systemPrompt = "You are a helpful assistant";
+	const userPrompt = `In 1 word: does ${userQuery} have similar meaning as ${intentDescription}?`;
+	const result = await getChatCompletions(systemPrompt, userPrompt);
+	var isMatch = result.toLowerCase().startsWith("yes");
+	console.log("OpenAI results: isMatch=%s, customerQuery=%s, intentDescription=%s", isMatch, userQuery, intentDescription);
+	return isMatch;
+}
+
+async function getChatCompletions(systemPrompt: string, userPrompt: string){
 	const deploymentName = process.env.AZURE_OPENAI_DEPLOYMENT_MODEL_NAME;
 	const messages = [
-		{ role: "system", content: "You are a helpful assistant." },
-		{ role: "user", content: `In less than 200 characters: respond to this question: ${speechInput}` },
+		{ role: "system", content: systemPrompt },
+		{ role: "user", content: userPrompt },
 	];
 
 	const response = await openAiClient.getChatCompletions(deploymentName, messages);
 	const responseContent = response.choices[0].message.content;
+	console.log(responseContent);
 	return responseContent;
 }
 
+async function getChatGptResponse(speechInput: string){
+	return await getChatCompletions(answerPromptSystemTemplate, speechInput);
+}
+
 app.post("/api/incomingCall", async (req: any, res:any)=>{
 	console.log(`Received incoming call event - data --> ${JSON.stringify(req.body)} `);
 	const event = req.body[0];
@@ -81,18 +128,18 @@ app.post("/api/incomingCall", async (req: any, res:any)=>{
 			res.status(200).json({
 				validationResponse: eventData.validationCode,
 			});
+
+			return;
 		}
 
 		callerId = eventData.from.rawId;
 		const uuid = uuidv4();
 		const callbackUri = `${process.env.CALLBACK_URI}/api/callbacks/${uuid}?callerId=${callerId}`;
 		const incomingCallContext = eventData.incomingCallContext;
-		console.log(`Cognitive service endpoint:  ${process.env.COGNITIVE_SERVICE_ENDPOINT.trim()}`)
-		let cognitiveServiceEndpoint = process.env.COGNITIVE_SERVICE_ENDPOINT;
-		const answerCallOptions: AnswerCallOptions = { cognitiveServicesEndpoint: cognitiveServiceEndpoint }; 
+		console.log(`Cognitive service endpoint:  ${process.env.COGNITIVE_SERVICE_ENDPOINT.trim()}`);
+		const callIntelligenceOptions: CallIntelligenceOptions = { cognitiveServicesEndpoint: process.env.COGNITIVE_SERVICE_ENDPOINT };
+		const answerCallOptions: AnswerCallOptions = { callIntelligenceOptions: callIntelligenceOptions }; 
 		answerCallResult = await acsClient.answerCall(incomingCallContext, callbackUri, answerCallOptions);
-		console.log(`Incoming call answered. Cognitive Service Url :${cognitiveServiceEndpoint}, 
-		Callback Uri: ${callbackUri},  CallConnection Id: ${answerCallResult.callConnectionProperties.callConnectionId} `);
 		callConnection = answerCallResult.callConnection;
 		callMedia = callConnection.getCallMedia();
 	}
@@ -105,24 +152,71 @@ app.post('/api/callbacks/:contextId', async (req:any, res:any) => {
 	const contextId = req.params.contextId;
   	const event = req.body[0];
 	const eventData = event.data;
-	console.log(`Received event type ${event.type}`);
+	console.log(`Received callback event - data --> ${JSON.stringify(req.body)} `);
 	console.log(`event type match ${event.type === "Microsoft.Communication.CallConnected"}`);
 	if(event.type === "Microsoft.Communication.CallConnected"){
 		console.log("Received CallConnected event");
-		const text = 'Hello. How can I help?';
-		startRecognizing(callMedia, callerId, text, 'GetFreeFormText');
+		startRecognizing(callMedia, callerId, helloPrompt, 'GetFreeFormText');
 	}
-	else if(event.type === "Microsoft.Communication.PlayCompleted" || event.type === "Microsoft.Communication.playFailed"){
+	else if(event.type === "Microsoft.Communication.PlayCompleted"){
 		console.log("Received PlayCompleted event");
+		
+		if(eventData.operationContext && eventData.operationContext === transferFailedContext) {
+			hangUpCall();
+		}
+		else if(eventData.operationContext === connectAgentContext) {
+			if(!agentPhonenumber){
+				handlePlay(callMedia, agentPhoneNumberEmptyPrompt, transferFailedContext);
+			}
+			else{
+				const phoneNumberIdentifier: PhoneNumberIdentifier = { phoneNumber: agentPhonenumber };
+				const result = await callConnection.transferCallToParticipant(phoneNumberIdentifier);
+				console.log("Transfer call initiated");
+			}
+		}
+	}
+	else if(event.type === "Microsoft.Communication.playFailed"){
+		console.log("Received PlayFailed event");
 		hangUpCall();
 	}
+	else if(event.type === "Microsoft.Communication.callTransferAccepted"){
+		console.log("Call transfer accepted event received");
+	}
+	else if(event.type === "Microsoft.Communication.callTransferFailed"){
+		console.log("Call transfer failed event received");
+		var resultInformation = eventData.resultInformation;
+		console.log("Encountered error during call transfer, message=%s, code=%s, subCode=%s", resultInformation?.message, resultInformation?.code, resultInformation?.subCode);
+		handlePlay(callMedia, callTransferFailurePrompt, transferFailedContext);
+	}
 	else if(event.type === "Microsoft.Communication.RecognizeCompleted"){
 		if(eventData.recognitionType === "speech"){
 			const speechText = eventData.speechResult.speech;
 			if(speechText !== ''){
 				console.log(`Recognized speech ${speechText}`);
-				const chatGptResponse = await getChatGptResponse(speechText);
-				startRecognizing(callMedia, callerId, chatGptResponse, 'OpenAISample')
+				if(await detectEscalateToAgentIntent(speechText)){
+					handlePlay(callMedia, EndCallPhraseToConnectAgent, connectAgentContext);
+				}
+				else{
+					const chatGptResponse = await getChatGptResponse(speechText);
+					const match = chatGptResponse.match(chatResponseExtractPattern);
+					console.log(match);
+					if(match){
+						console.log("Chat GPT Answer=%s, Sentiment Rating=%s, Intent=%s, Category=%s",
+						match[0], match[1], match[2], match[3]);
+						const score = getSentimentScore(match[1].trim());
+						console.log("score=%s", score)
+						if(score > -1 && score < 5){
+							handlePlay(callMedia, connectAgentPrompt, connectAgentContext);
+						}
+						else{
+							startRecognizing(callMedia, callerId, match[0], 'OpenAISample')
+						}
+					}
+					else{
+						console.log("No match found");
+						startRecognizing(callMedia, callerId, chatGptResponse, 'OpenAISample')
+					}
+				}
 			}
 		}
 	}
@@ -130,11 +224,10 @@ app.post('/api/callbacks/:contextId', async (req:any, res:any) => {
 		const resultInformation = eventData.resultInformation
 		var code = resultInformation.subCode;
 		if(code === 8510){
-			const text = 'I ve noticed that you have been silent. Are you still there?';
-			startRecognizing(callMedia, callerId, text, 'GetFreeFormText');
+			startRecognizing(callMedia, callerId, timeoutSilencePrompt, 'GetFreeFormText');
 		}
 		else{
-			handlePlay(callMedia);
+			handlePlay(callMedia, goodbyePrompt, "goodbye");
 		}
 	}
 	else if(event.type === "Microsoft.Communication.CallDisconnected"){

From 88406cff3a2362576535a92818e8922b33552871 Mon Sep 17 00:00:00 2001
From: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>
Date: Tue, 28 Nov 2023 13:36:21 -0800
Subject: [PATCH 6/7] addressing the PR comments and updating the package

---
 callautomation-openai-sample/README.md    |  3 ++-
 callautomation-openai-sample/package.json |  2 +-
 callautomation-openai-sample/src/app.ts   | 15 +++++++++++----
 3 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/callautomation-openai-sample/README.md b/callautomation-openai-sample/README.md
index 6905354f..d9b8c8a9 100644
--- a/callautomation-openai-sample/README.md
+++ b/callautomation-openai-sample/README.md
@@ -13,7 +13,7 @@ This sample application shows how the Azure Communication Services - Call Automa
 - [Node.js](https://nodejs.org/en/download) installed
 - Create an Azure Communication Services resource. For details, see [Create an Azure Communication Resource](https://docs.microsoft.com/azure/communication-services/quickstarts/create-communication-resource). You will need to record your resource **connection string** for this sample.
 - Get a phone number for your new Azure Communication Services resource. For details, see [Get a phone number](https://learn.microsoft.com/en-us/azure/communication-services/quickstarts/telephony/get-phone-number?tabs=windows&pivots=programming-language-csharp)
-- Create an Azure Cognitive Services resource. For details, see Create an Azure Cognitive Services Resource.
+- Create an Azure AI Multi Services resource. For details, see [Create an Azure Cognitive Services Resource](https://ms.portal.azure.com/#create/Microsoft.CognitiveServicesAllInOne).
 - An Azure OpenAI Resource and Deployed Model. See [instructions](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal).
 
 ## Before running the sample for the first time
@@ -46,6 +46,7 @@ Open the `.env` file to configure the following settings
 4. `AZURE_OPENAI_SERVICE_KEY`: Azure Open AI service key
 5. `AZURE_OPENAI_SERVICE_ENDPOINT`: Azure Open AI endpoint
 6. `AZURE_OPENAI_DEPLOYMENT_MODEL_NAME`: Azure Open AI deployment name
+6. `AGENT_PHONE_NUMBER`: Agent phone number to transfer the call to resolve queries
 
 ### Run app locally
 
diff --git a/callautomation-openai-sample/package.json b/callautomation-openai-sample/package.json
index 6345f378..c97b1c7a 100644
--- a/callautomation-openai-sample/package.json
+++ b/callautomation-openai-sample/package.json
@@ -11,7 +11,7 @@
   "author": "",
   "license": "ISC",
   "dependencies": {
-    "@azure/communication-call-automation": "^1.1.0-alpha.20231109.1",
+    "@azure/communication-call-automation": "^1.1.0",
     "@azure/communication-common": "^2.2.0",
     "@azure/eventgrid": "^4.12.0",
     "@azure/openai": "^1.0.0-beta.7",
diff --git a/callautomation-openai-sample/src/app.ts b/callautomation-openai-sample/src/app.ts
index 015f3f15..c54275d6 100644
--- a/callautomation-openai-sample/src/app.ts
+++ b/callautomation-openai-sample/src/app.ts
@@ -17,6 +17,7 @@ let openAiClient : OpenAIClient;
 let answerCallResult: AnswerCallResult;
 let callerId: string;
 let callMedia: CallMedia;
+let maxTimeout = 2;
 
 const answerPromptSystemTemplate = `You are an assisant designed to answer the customer query and analyze the sentiment score from the customer tone. 
     You also need to determine the intent of the customer query and classify it into categories such as sales, marketing, shopping, etc.
@@ -37,6 +38,7 @@ const EndCallPhraseToConnectAgent = "Sure, please stay on the line. I’m going
 
 const transferFailedContext = "TransferFailed";
 const connectAgentContext = "ConnectAgent";
+const goodbyeContext = "Goodbye";
 
 const agentPhonenumber = process.env.AGENT_PHONE_NUMBER;
 const chatResponseExtractPattern = /(?<=: ).*/g;
@@ -161,14 +163,18 @@ app.post('/api/callbacks/:contextId', async (req:any, res:any) => {
 	else if(event.type === "Microsoft.Communication.PlayCompleted"){
 		console.log("Received PlayCompleted event");
 		
-		if(eventData.operationContext && eventData.operationContext === transferFailedContext) {
-			hangUpCall();
+		if(eventData.operationContext && ( eventData.operationContext === transferFailedContext 
+			 || eventData.operationContext === goodbyeContext )) {
+				console.log("Disconnecting the call");
+				hangUpCall();
 		}
 		else if(eventData.operationContext === connectAgentContext) {
 			if(!agentPhonenumber){
+				console.log("Agent phone number is empty.");
 				handlePlay(callMedia, agentPhoneNumberEmptyPrompt, transferFailedContext);
 			}
 			else{
+				console.log("Initiating the call transfer.");
 				const phoneNumberIdentifier: PhoneNumberIdentifier = { phoneNumber: agentPhonenumber };
 				const result = await callConnection.transferCallToParticipant(phoneNumberIdentifier);
 				console.log("Transfer call initiated");
@@ -223,11 +229,12 @@ app.post('/api/callbacks/:contextId', async (req:any, res:any) => {
 	else if(event.type === "Microsoft.Communication.RecognizeFailed"){
 		const resultInformation = eventData.resultInformation
 		var code = resultInformation.subCode;
-		if(code === 8510){
+		if(code === 8510 && maxTimeout > 0){
+			maxTimeout--;
 			startRecognizing(callMedia, callerId, timeoutSilencePrompt, 'GetFreeFormText');
 		}
 		else{
-			handlePlay(callMedia, goodbyePrompt, "goodbye");
+			handlePlay(callMedia, goodbyePrompt, goodbyeContext);
 		}
 	}
 	else if(event.type === "Microsoft.Communication.CallDisconnected"){

From b7263df958d3b4d5f907ef45b2e757d350b1df8b Mon Sep 17 00:00:00 2001
From: Vinothini Dharmaraj <146493756+v-vdharmaraj@users.noreply.github.com>
Date: Tue, 28 Nov 2023 15:12:09 -0800
Subject: [PATCH 7/7] Update README.md

---
 callautomation-openai-sample/README.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/callautomation-openai-sample/README.md b/callautomation-openai-sample/README.md
index d9b8c8a9..5eb20c39 100644
--- a/callautomation-openai-sample/README.md
+++ b/callautomation-openai-sample/README.md
@@ -13,7 +13,7 @@ This sample application shows how the Azure Communication Services - Call Automa
 - [Node.js](https://nodejs.org/en/download) installed
 - Create an Azure Communication Services resource. For details, see [Create an Azure Communication Resource](https://docs.microsoft.com/azure/communication-services/quickstarts/create-communication-resource). You will need to record your resource **connection string** for this sample.
 - Get a phone number for your new Azure Communication Services resource. For details, see [Get a phone number](https://learn.microsoft.com/en-us/azure/communication-services/quickstarts/telephony/get-phone-number?tabs=windows&pivots=programming-language-csharp)
-- Create an Azure AI Multi Services resource. For details, see [Create an Azure Cognitive Services Resource](https://ms.portal.azure.com/#create/Microsoft.CognitiveServicesAllInOne).
+- Create Azure AI Multi Service resource. For details, see [Create an Azure AI Multi service](https://learn.microsoft.com/en-us/azure/cognitive-services/cognitive-services-apis-create-account).
 - An Azure OpenAI Resource and Deployed Model. See [instructions](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal).
 
 ## Before running the sample for the first time
@@ -54,4 +54,4 @@ Open the `.env` file to configure the following settings
 2. Browser should pop up with the below page. If not navigate it to `http://localhost:8080/`
 3. Register an EventGrid Webhook for the IncomingCall Event that points to your DevTunnel URI. Instructions [here](https://learn.microsoft.com/en-us/azure/communication-services/concepts/call-automation/incoming-call-notification).
 
-Once that's completed you should have a running application. The best way to test this is to place a call to your ACS phone number and talk to your intelligent agent.
\ No newline at end of file
+Once that's completed you should have a running application. The best way to test this is to place a call to your ACS phone number and talk to your intelligent agent.