Skip to content

Added speech synthesis bypass engine#2445

Merged
compulim merged 6 commits intomicrosoft:masterfrom
compulim:fix-2435
Oct 5, 2019
Merged

Added speech synthesis bypass engine#2445
compulim merged 6 commits intomicrosoft:masterfrom
compulim:fix-2435

Conversation

@compulim
Copy link
Copy Markdown
Contributor

@compulim compulim commented Oct 4, 2019

Fixes #2435.

Changelog Entry

Fixed

  • Fix #2435. Fix microphone button stuck on voice-triggered expecting input hint without a speech synthesis engine, by @compulim in PR #2445

Description

On a system without speech synthesis engine:

  • Queue a message to be synthesized
  • Open microphone in a programmatic way and not clicking on microphone button

Since speech synthesis is not available, the queued message is pending to be synthesized and never complete. Internally, we will wait until all queued messages are synthesized before turning on the microphone. This is to prevent echo (bot hearing itself talking). Thus, the microphone will get stuck and never turned on.

The speech synthesis engine is configured on the component side. When the saga in the core side tries to perform synthesis, it has no knowledge whether the engine is available or not.

This can be reproduced by:

  • Configure off speech synthesis
  • Click microphone button and trigger the bot to respond with expecting input hint
  • Click microphone button again

Since no speech synthesis engine is configured, the bot message is being queued but never complete. And the microphone is queued to open after all synthesis is completed. Thus, the microphone button is disabled and never open.

Specific Changes

If speech synthesis engine is not configured, we will provide a bypass engine.

The bypass engine will act just like the real engine, except all synthesis operation will be completed instantly. This will unblock any future issues related to queued messages on a configured off system.

In the future, we could consider moving all speech operations to component side to prevent this issue from happening.


  • Testing Added

@compulim compulim changed the title Added no-op speech synthesis engine Added speech synthesis bypass engine Oct 4, 2019
Comment thread CHANGELOG.md Outdated
Co-Authored-By: TJ Durnford <[email protected]>
Comment thread packages/component/src/external/event-target-shim.js Outdated
Comment thread packages/component/src/external/event-target-shim.js
Comment thread packages/component/src/external/event-target-shim.js
Comment thread packages/component/src/external/event-target-shim.js Outdated
Comment thread packages/component/src/BasicTranscript.js
Comment thread packages/component/src/Speech/BypassSpeechSynthesisPonyfill.js
Copy link
Copy Markdown
Contributor

@corinagum corinagum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@compulim compulim mentioned this pull request Oct 4, 2019
13 tasks
@compulim compulim merged commit b2fc36f into microsoft:master Oct 5, 2019
@compulim compulim deleted the fix-2435 branch October 5, 2019 01:43
@compulim compulim mentioned this pull request Oct 25, 2019
55 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Microphone button disabled when speech synthesis is disabled and received an "expecting" input hint

3 participants