Skip to content

Selectable audio input device#3079

Merged
compulim merged 36 commits intomicrosoft:masterfrom
compulim:fix-2481-bluetooth
Apr 22, 2020
Merged

Selectable audio input device#3079
compulim merged 36 commits intomicrosoft:masterfrom
compulim:fix-2481-bluetooth

Conversation

@compulim
Copy link
Copy Markdown
Contributor

@compulim compulim commented Apr 4, 2020

Fixes #2481. Fixes #2987. Fixes #2886. Fixed #2668.

Changelog Entry

Added

  • Resolves #2481. Support selecting different audio input device for Cognitive Services Speech Services, by @compulim in PR #3079

Fixed

Changed

Samples

Description

Support selectable audio input device. From user report, even AirPod is connected, on iOS, it is not selected as default. JavaScript is required to select the device and make it default.

Also, today, we have 3 references of microsoft-cognitiveservices-speech-sdk, which take up significantly amount of size in our bundle (~300 KB each). We are going to converge them into a single copy inside the bundle.

We will temporarily bump to a preview build of web-speech-cognitive-services. When we do production release, we will build a production version of web-speech-cognitive-services and bump the version in Web Chat.

Specific Changes

  • Update webpack.config.js to reference to the same microsoft-cognitiveservices-speech-sdk package to reduce footprint by ~600 KB
  • Add audioContext option to createCognitiveServicesSpeechServicesPonyfillFactory()
    • This is for speech synthesis and supporting our test infrastructure, we will send in a mock audioContext object
  • Add audioInputDeviceId option to createCognitiveServicesSpeechServicesPonyfillFactory()
    • If no audioConfig is set, we will use audioInputDeviceId to select the correct audio input device
  • Add samples to show how to select audio input device, and a comprehensive sample to try out this feature
  • Update HTML test infrastructure to enable testing under IE11
    • Some packages are not transpiled to ES5. We added a postinstall script to transpile them after npm run bootstrap or lerna add. We prefer to use postinstall because build time is substantial and to reduce the tax on our build
    • These packages were not transpiled to ES5:
      • @babel/plugin-proposal-async-generation-functions
        • This package is mainly used under Node.js, hardly in a browser, and rarely on IE11
      • event-iterator
        • This package is using async iterator, which is not natively supported on IE11
      • microsoft-cognitiveservices-speech-sdk
  • Bump react-say to not breaking IE11

External changes


  • Testing Added

@compulim compulim marked this pull request as ready for review April 5, 2020 17:54
@compulim compulim changed the title [DRAFT] Selectable audio input device Selectable audio input device Apr 5, 2020
@compulim compulim force-pushed the fix-2481-bluetooth branch from ad620e4 to 530c1f3 Compare April 5, 2020 17:58
@corinagum corinagum self-assigned this Apr 6, 2020
Comment thread CHANGELOG.md Outdated
Comment thread __tests__/html/offlineUI.fatalError.html
Comment thread samples/README.md Outdated
Comment thread samples/README.md
Comment thread samples/03.speech/h.select-audio-input-device/README.md
Comment thread samples/03.speech/h.select-audio-input-device/comprehensive.html Outdated
Comment thread samples/03.speech/h.select-audio-input-device/comprehensive.html
Comment thread samples/03.speech/h.select-audio-input-device/comprehensive.html Outdated
Comment thread samples/03.speech/h.select-audio-input-device/comprehensive.html Outdated
Comment thread samples/03.speech/h.select-audio-input-device/comprehensive.html
Comment thread samples/03.speech/h.select-audio-input-device/comprehensive.html Outdated
Comment thread samples/03.speech/h.select-audio-input-device/index.html Outdated
Comment thread samples/03.speech/h.select-audio-input-device/comprehensive.html Outdated
Comment thread packages/testharness/src/pageObjects/clickMicrophoneButton.js
Comment thread packages/testharness/src/pageObjects/wait.js
Copy link
Copy Markdown
Contributor

@corinagum corinagum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To discuss:

  • number of packages published for this project that aren't peer reviewed
  • how complicated our single-page samples have gotten. They need to be simpler and split into re-used modules

@compulim
Copy link
Copy Markdown
Contributor Author

compulim commented Apr 8, 2020

Tests for Direct Line Speech added. Test is currently failing due to reliability, will fix it.

Comment thread __tests__/video.js
Comment thread __tests__/html/speechRecognition.simple.js Outdated
@compulim compulim force-pushed the fix-2481-bluetooth branch from b21d54b to 5288298 Compare April 22, 2020 19:05
@compulim compulim merged commit 7425004 into microsoft:master Apr 22, 2020
@compulim compulim deleted the fix-2481-bluetooth branch April 22, 2020 23:42
@compulim compulim mentioned this pull request May 15, 2020
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants