Skip to content

Commit 43e4103

Browse files
thangavelethangavele
authored
Merge pull request #4586 from syncfusion-content/EJ2-984607-assist-stt
984607: Added UG content and code snippet for AI AssistView integration with STT and TTS.
2 parents 04a23a2 + 0cb5898 commit 43e4103

File tree

18 files changed

+1333
-16
lines changed

18 files changed

+1333
-16
lines changed

‎ej2-asp-core-mvc/ai-assistview/EJ2_ASP.MVC/speech/speech-to-text.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
layout: post
3+
title: Speech-to-Text With ##Platform_Name## AI AssistView Control | Syncfusion
4+
description: Checkout and learn about configuration of Speech-to-Text with Azure OpenAI in ##Platform_Name## AI AssistView control of Syncfusion Essential JS 2 and more.
5+
platform: ej2-asp-core-mvc
6+
control: Azure Open AI
7+
publishingplatform: ##Platform_Name##
8+
documentation: ug
9+
---
10+
11+
# Speech-to-Text in ASP.NET MVC AI AssistView
12+
13+
The Syncfusion ASP.NET MVC AI AssistView control supports `Speech-to-Text` functionality through the browser's [Web Speech API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API), enabling conversion of spoken words into text using the device's microphone.
14+
15+
## Prerequisites
16+
17+
Before integrating `Speech-to-Text`, ensure the following:
18+
19+
1. The Syncfusion AI AssistView control is properly set up in your ASP.NET MVC application.
20+
- [ASP.NET MVC Getting Started Guide](../getting-started)
21+
22+
2. The AI AssistView control is integrated with [Azure OpenAI](https://microsoft.github.io/PartnerResources/skilling/ai-ml-academy/resources/openai).
23+
- [Integration of Azure OpenAI With ASP.NET MVC AI AssistView control](../ai-integrations/openai-integration)
24+
25+
## Configure Speech-to-Text
26+
27+
To enable Speech-to-Text functionality, modify the `Index.cshtml` file to incorporate the Web Speech API. The [SpeechToText](https://ej2.syncfusion.com/aspnetmvc/documentation/speech-to-text/getting-started) control listens for microphone input, transcribes spoken words, and updates the AI AssistView's editable footer with the transcribed text. The transcribed text is then sent as a prompt to the Azure OpenAI service via the AI AssistView control.
28+
29+
{% tabs %}
30+
{% highlight razor tabtitle="CSHTML" %}
31+
{% include code-snippet/ai-assistview/speech/stt/razor %}
32+
{% endhighlight %}
33+
{% highlight c# tabtitle="SpeechToText.cs" %}
34+
{% include code-snippet/ai-assistview/speech/stt/speechtotextmvc.cs %}
35+
{% endhighlight %}
36+
{% endtabs %}
37+
38+
![Integrating Speech-to-Text with AI AssistView](images/aiassist-stt.png)
39+
40+
## See Also
41+
42+
* [Text-to-Speech](./text-to-speech)

‎ej2-asp-core-mvc/ai-assistview/EJ2_ASP.MVC/speech/text-to-speech.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
layout: post
3+
title: Text-to-Speech With ##Platform_Name## AI AssistView Control | Syncfusion
4+
description: Checkout and learn about configuration of Text-to-Speech with Azure OpenAI in ##Platform_Name## AI AssistView control of Syncfusion Essential JS 2 and more.
5+
platform: ej2-asp-core-mvc
6+
control: Azure Open AI
7+
publishingplatform: ##Platform_Name##
8+
documentation: ug
9+
---
10+
11+
# Text-to-Speech in ASP.NET MVC AI AssistView
12+
13+
The Syncfusion TypeScript AI AssistView component supports `Text-to-Speech` (TTS) functionality using the browser's Web Speech API specifically using the [SpeechSynthesisUtterance](https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesisUtterance) interface to convert AI-generated response into spoken audio.
14+
15+
## Prerequisites
16+
17+
Before integrating `Text-to-Speech`, ensure the following:
18+
19+
1. The Syncfusion AI AssistView control is properly set up in your ASP.NET MVC application.
20+
- [ASP.NET MVC Getting Started Guide](../getting-started)
21+
22+
2. The AI AssistView control is integrated with [Azure OpenAI](https://microsoft.github.io/PartnerResources/skilling/ai-ml-academy/resources/openai).
23+
- [Integration of Azure OpenAI With ASP.NET MVC AI AssistView control](../ai-integrations/openai-integration)
24+
25+
## Configure Text-to-Speech
26+
27+
To enable Text-to-Speech functionality, modify the `Index.cshtml` file to incorporate the Web Speech API. A custom `Read Aloud` button is added to the response toolbar using the [ResponseToolbarSettings](https://help.syncfusion.com/cr/aspnetmvc-js2/Syncfusion.EJ2.InteractiveChat.AIAssistViewResponseToolbarSettings.html) property. When clicked, the [ItemClicked](https://help.syncfusion.com/cr/aspnetmvc-js2/Syncfusion.EJ2.InteractiveChat.AIAssistViewResponseToolbarSettings.html#Syncfusion_EJ2_InteractiveChat_AIAssistViewResponseToolbarSettings_ItemClicked) event extracts plain text from the generated AI response and use the browser SpeechSynthesis API to read it aloud.
28+
29+
{% tabs %}
30+
{% highlight razor tabtitle="CSHTML" %}
31+
{% include code-snippet/ai-assistview/speech/tts/razor %}
32+
{% endhighlight %}
33+
{% highlight c# tabtitle="SpeechToText.cs" %}
34+
{% include code-snippet/ai-assistview/speech/tts/texttospeechmvc.cs %}
35+
{% endhighlight %}
36+
{% endtabs %}
37+
38+
![Integrating Text-to-Speech with AI AssistView](images/aiassist-tts.png)
39+
40+
## See Also
41+
42+
* [Speech-to-Text](./speech-to-text)

‎ej2-asp-core-mvc/ai-assistview/EJ2_ASP.NETCORE/speech/speech-to-text.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
layout: post
3+
title: Speech-to-Text With ##Platform_Name## AI AssistView Control | Syncfusion
4+
description: Checkout and learn about configuration of Speech-to-Text with Azure OpenAI in ##Platform_Name## AI AssistView control of Syncfusion Essential JS 2 and more.
5+
platform: ej2-asp-core-mvc
6+
control: Azure Open AI
7+
publishingplatform: ##Platform_Name##
8+
documentation: ug
9+
---
10+
11+
# Speech-to-Text in ASP.NET Core AI AssistView
12+
13+
The Syncfusion ASP.NET Core AI AssistView control supports `Speech-to-Text` functionality through the browser's [Web Speech API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API), enabling conversion of spoken words into text using the device's microphone.
14+
15+
## Prerequisites
16+
17+
Before integrating `Speech-to-Text`, ensure the following:
18+
19+
1. The Syncfusion AI AssistView control is properly set up in your ASP.NET Core application.
20+
- [ASP.NET Core Getting Started Guide](../getting-started)
21+
22+
2. The AI AssistView control is integrated with [Azure OpenAI](https://microsoft.github.io/PartnerResources/skilling/ai-ml-academy/resources/openai).
23+
- [Integration of Azure OpenAI With ASP.NET Core AI AssistView control](../ai-integrations/openai-integration)
24+
25+
## Configure Speech-to-Text
26+
27+
To enable Speech-to-Text functionality, modify the `Index.cshtml` file to incorporate the Web Speech API. The [SpeechToText](https://ej2.syncfusion.com/aspnetcore/documentation/speech-to-text/getting-started) control listens for microphone input, transcribes spoken words, and updates the AI AssistView's editable footer with the transcribed text. The transcribed text is then sent as a prompt to the Azure OpenAI service via the AI AssistView control.
28+
29+
{% tabs %}
30+
{% highlight razor tabtitle="CSHTML" %}
31+
{% include code-snippet/ai-assistview/speech/stt/tagHelper %}
32+
{% endhighlight %}
33+
{% highlight c# tabtitle="Gemini.cs" %}
34+
{% include code-snippet/ai-assistview/speech/stt/speechtotextcore.cs %}
35+
{% endhighlight %}
36+
{% endtabs %}
37+
38+
![Integrating Speech-to-Text with AI AssistView](images/aiassist-stt.png)
39+
40+
## See Also
41+
42+
* [Text-to-Speech](./text-to-speech)

‎ej2-asp-core-mvc/ai-assistview/EJ2_ASP.NETCORE/speech/text-to-speech.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
layout: post
3+
title: Text-to-Speech With ##Platform_Name## AI AssistView Control | Syncfusion
4+
description: Checkout and learn about configuration of Text-to-Speech with Azure OpenAI in ##Platform_Name## AI AssistView control of Syncfusion Essential JS 2 and more.
5+
platform: ej2-asp-core-mvc
6+
control: Azure Open AI
7+
publishingplatform: ##Platform_Name##
8+
documentation: ug
9+
---
10+
11+
# Text-to-Speech in ASP.NET Core AI AssistView
12+
13+
The Syncfusion TypeScript AI AssistView component supports `Text-to-Speech` (TTS) functionality using the browser's Web Speech API specifically using the [SpeechSynthesisUtterance](https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesisUtterance) interface to convert AI-generated response into spoken audio.
14+
15+
## Prerequisites
16+
17+
Before integrating `Text-to-Speech`, ensure the following:
18+
19+
1. The Syncfusion AI AssistView control is properly set up in your ASP.NET Core application.
20+
- [ASP.NET Core Getting Started Guide](../getting-started)
21+
22+
2. The AI AssistView control is integrated with [Azure OpenAI](https://microsoft.github.io/PartnerResources/skilling/ai-ml-academy/resources/openai).
23+
- [Integration of Azure OpenAI With ASP.NET Core AI AssistView control](../ai-integrations/openai-integration)
24+
25+
## Configure Text-to-Speech
26+
27+
To enable Text-to-Speech functionality, modify the `Index.cshtml` file to incorporate the Web Speech API. A custom `Read Aloud` button is added to the response toolbar using the `e-aiassistview-responsetoolbarsettings` tag helper. When clicked, the [itemClicked](https://help.syncfusion.com/cr/aspnetcore-js2/Syncfusion.EJ2.InteractiveChat.AIAssistViewResponseToolbarSettings.html#Syncfusion_EJ2_InteractiveChat_AIAssistViewResponseToolbarSettings_ItemClicked) event extracts plain text from the generated AI response and use the browser SpeechSynthesis API to read it aloud.
28+
29+
{% tabs %}
30+
{% highlight razor tabtitle="CSHTML" %}
31+
{% include code-snippet/ai-assistview/speech/tts/tagHelper %}
32+
{% endhighlight %}
33+
{% highlight c# tabtitle="Gemini.cs" %}
34+
{% include code-snippet/ai-assistview/speech/tts/texttospeechcore.cs %}
35+
{% endhighlight %}
36+
{% endtabs %}
37+
38+
![Integrating Text-to-Speech with AI AssistView](images/aiassist-tts.png)
39+
40+
## See Also
41+
42+
* [Speech-to-Text](./speech-to-text)

‎ej2-asp-core-mvc/ai-assistview/images/aiassist-stt.png‎

14.8 KB
Loading

‎ej2-asp-core-mvc/ai-assistview/images/aiassist-tts.png‎

23.9 KB
Loading

‎ej2-asp-core-mvc/code-snippet/ai-assistview/ai-integrations/open-ai/openaicore.cs‎

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
1-
using OpenAI;
21
using Azure;
32
using Azure.AI.OpenAI;
3+
using Microsoft.AspNetCore.Mvc;
4+
using Microsoft.AspNetCore.Mvc.RazorPages;
5+
using OpenAI.Chat;
46
namespace WebApplication4.Pages
57
{
68
public class IndexModel : PageModel
@@ -68,22 +70,22 @@ public async Task<IActionResult> OnPostGetAIResponse([FromBody] PromptRequest re
6870
return BadRequest($"Error generating response: {ex.Message}");
6971
}
7072
}
71-
}
73+
}
7274

73-
public class IndexViewModel
74-
{
75-
public List<ToolbarItemModel> Items { get; set; } = new List<ToolbarItemModel>();
76-
public string[] PromptSuggestionData { get; set; }
77-
}
75+
public class IndexViewModel
76+
{
77+
public List<ToolbarItemModel> Items { get; set; } = new List<ToolbarItemModel>();
78+
public string[] PromptSuggestionData { get; set; }
79+
}
7880

79-
public class PromptRequest
80-
{
81-
public string Prompt { get; set; }
82-
}
81+
public class PromptRequest
82+
{
83+
public string Prompt { get; set; }
84+
}
8385

84-
public class ToolbarItemModel
85-
{
86-
public string align { get; set; }
87-
public string iconCss { get; set; }
88-
}
86+
public class ToolbarItemModel
87+
{
88+
public string align { get; set; }
89+
public string iconCss { get; set; }
8990
}
91+
}

‎ej2-asp-core-mvc/code-snippet/ai-assistview/ai-integrations/open-ai/tagHelper‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
@model IndexModel
12
@using Syncfusion.EJ2.InteractiveChat
23
@{
34
ViewData["Title"] = "AI Assistance with Gemini";

0 commit comments

Comments
 (0)