Skip to content

SpeechToText has horrendous performance issues and/or not working properly #1645

@sej69

Description

@sej69

Is there an existing issue for this?

  • I have searched the existing issues

Did you read the "Reporting a bug" section on Contributing file?

Current Behavior

First wrote this on SO: [https://stackoverflow.com/questions/77808446/communitytoolkit-speechtotext-performance-is-extremely-slow?noredirect=1#comment137188380_77808446]

I also found this in your bug reporting which is closed as unverified. https://github.com/CommunityToolkit/Maui/issues/1586 I have verified this now as not operating as your documents state it should using the below code.

The SpeechToText is not detecting silence and only reports back after it appears to timeout. I created a .Net Maui app using the following async method:

     async Task StartListener(CancellationToken cancellationToken)
    {
        var isGranted = await speechToText.RequestPermissions(cancellationToken);
        if (!isGranted)
        {
            await Toast.Make("Permission not granted").Show(CancellationToken.None);
            return;
        }

        do
        {
            Stopwatch sw = Stopwatch.StartNew();

            var recognitionResult = await speechToText.ListenAsync(
                                                CultureInfo.GetCultureInfo(Language),
                                                new Progress<string>(partialText =>
                                                {
                                                    RecognitionText += partialText + " ";
                                                }), cancellationToken);

            sw.Stop();

            if (recognitionResult.IsSuccessful)
            {
                RecognitionText = recognitionResult.Text;
                Debug.WriteLine("Success " + RecognitionText + " Time " + sw.Elapsed);
                RecognitionText = string.Empty;
            }
            else
            {
                Debug.WriteLine("failed - Time " + sw.Elapsed);
            }
        } while (!cancellationToken.IsCancellationRequested);
    }

When I run this on my iPad, it loads and waits. I speak, "This is a test" and then wait for 1 minute when it responds below in the debug window with the success message and time it took. It doesn't seem to matter if I say this phrase quickly after start or waiting for 30 seconds, it still seems to take a minute.

[0:] Success This is a test Time 00:01:00.9459623

This method is unusable in this current state. People won't wait a minute each time they speak for a response. And according to your docs, this method should detect silence.

Expected Behavior

This should detect silence and return text when it does. You probably also want to have a way to adjust the timeout value which doesn't appear to be there now.

Steps To Reproduce

  1. create a new .net maui application
  2. add the communitytoolkit
  3. use the above code to test

Link to public reproduction project repository

https://github.com/sej69/TestSpeechRecognizer

Environment

- .NET MAUI CommunityToolkit:
- OS: ios running on an iPad through my Mac (setup through VS)  iPad is on current version.
- .NET MAUI:

Anything else?

using current versions of all libraries and Maui. (.net 8) Very simple basic project.

Metadata

Metadata

Labels

area/essentialsIssue/Discussion/PR that has to do with EssentialsbugSomething isn't workingunverified

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions