Skip to content

Add Direct Line Speech support#2621

Merged
compulim merged 47 commits intomicrosoft:masterfrom
compulim:feat-dlspeech
Dec 4, 2019
Merged

Add Direct Line Speech support#2621
compulim merged 47 commits intomicrosoft:masterfrom
compulim:feat-dlspeech

Conversation

@compulim
Copy link
Copy Markdown
Contributor

@compulim compulim commented Nov 21, 2019 •
edited
Loading

Fixes #2571. Fixes #2572. Fixes #2574.

Changelog Entry

Added

Description

  • Add Direct Line Speech SDK package
  • Bundle Direct Line Speech SDK with Web Chat CDN bundle and NPM package
  • Add new sample 06.i.directline-speech-sample

Specific Changes

  • Added a new package botframework-directlinespeech-sdk under the folder /packages/directlinespeech-sdk
    • This is the Direct Line Speech SDK adapter set, including both DirectLineJS-alt, and Web Speech API ponyfill
  • Support unabortable speech recognition engine
    • If the speech recognition engine does not allow abort, we will disable microphone button while recognition is ongoing
  • We will start sending detailed speech recognition result to the bot, including confidence and actual transcript
  • We will peacefully ignore OAuth if the DLJS adapter does not support it
  • New 06.i.direct-line-speech sample
  • Add documentation to SPEECH.md
  • Testing Added
    • We need manual testing

stevengum
stevengum reviewed Nov 21, 2019
View reviewed changes
Copy link
Copy Markdown
Member

@stevengum stevengum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@compulim, do we need the commits before adbc6e to get DLS integration, or are those separate changes?

I'd like to take a look through this PR, but I'm not familiar with the codebase and what the other commits are needed for 😔

(I'd like to lessen my cognitive load in this PR if possible 😄)

@compulim compulim changed the title Add Direct Line Speech support [DRAFT] Add Direct Line Speech support Nov 21, 2019
@compulim compulim marked this pull request as ready for review December 3, 2019 10:34
@compulim compulim changed the title [DRAFT] Add Direct Line Speech support Add Direct Line Speech support Dec 3, 2019
@cwhitten
Copy link
Copy Markdown
Member

cwhitten commented Dec 3, 2019

The CI error is coming from https://registry.npmjs.org/botframework-directlinespeech-sdk not being found

@corinagum corinagum added the R7 label Dec 3, 2019
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Copy link
Copy Markdown
Contributor

@corinagum corinagum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

finished the 06.i readme

Comment thread serve.json Outdated
Comment thread samples/06.i.direct-line-speech/README.md Outdated
Comment thread samples/06.i.direct-line-speech/README.md
Comment thread samples/06.i.direct-line-speech/README.md Outdated
Comment thread samples/06.i.direct-line-speech/README.md
Comment thread samples/06.i.direct-line-speech/README.md Outdated
Comment thread samples/06.i.direct-line-speech/README.md Outdated
Comment thread samples/06.i.direct-line-speech/README.md Outdated
Comment thread samples/06.i.direct-line-speech/README.md Outdated
Comment thread samples/06.i.direct-line-speech/README.md Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread samples/06.i.direct-line-speech/README.md Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread samples/06.f.hybrid-speech/README.md Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/src/playWhiteNoise.js
Comment on lines +4 to +5
const EMPTY_MP3_BASE64 =
'SUQzBAAAAAAAI1RTU0UAAAAPAAADTGF2ZjU3LjU2LjEwMQAAAAAAAAAAAAAA//tAwAAAAAAAAAAAAAAAAAAAAAAASW5mbwAAAA8AAAACAAABhgC7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7u7//////////////////////////////////////////////////////////////////8AAAAATGF2YzU3LjY0AAAAAAAAAAAAAAAAJAUHAAAAAAAAAYYoRBqpAAAAAAD/+xDEAAPAAAGkAAAAIAAANIAAAARMQU1FMy45OS41VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVf/7EMQpg8AAAaQAAAAgAAA0gAAABFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV';
Copy link
Copy Markdown
Contributor

@corinagum corinagum Dec 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😯

Copy link
Copy Markdown
Member

@cwhitten cwhitten Dec 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've never seen this in an application before but it makes sense!

Copy link
Copy Markdown
Contributor Author

@compulim compulim Dec 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😅 I found a better way to prepare white noise programmatically. But need time to prove whether it works. You know, Safari is quite picky on Web Audio and this code already works in our adapter.

Not to mention the new code could be just about same length as this one.

BTW, this one is generated by ffmpeg -f lavfi -i anullsrc=r=44100:cl=mono -t 0.01 -q:a 9 -acodec libmp3lame out.mp3.

corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/src/createWebSpeechPonyfillFactory.js
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/serve.json Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/core/src/sagas/connectSaga.js
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/component/src/hooks/internal/useSettableDictateAbortable.js Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/.babelrc.js Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/.babelrc.js Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/.prettierrc.yml
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/public/index.html Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/public/index.html Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/public/index.html Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/public/index.html Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/public/index.html Outdated
corinagum
corinagum reviewed Dec 4, 2019
View reviewed changes
Comment thread packages/directlinespeech/src/DirectLineSpeech.js Outdated
compulim and others added 23 commits December 4, 2019 04:13
@coveralls
Copy link
Copy Markdown

coveralls commented Dec 4, 2019

Coverage Status

Coverage decreased (-2.8%) to 60.848% when pulling fb4ad40 on compulim:feat-dlspeech into 0fcd7fe on microsoft:master.

@compulim compulim merged commit 1f100e7 into microsoft:master Dec 4, 2019
This was referenced Dec 10, 2019
Closed
Closed
Merged
@corinagum corinagum mentioned this pull request Dec 13, 2019
Closed
73 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cwhitten cwhitten cwhitten requested changes

@a-b-r-o-w-n a-b-r-o-w-n Awaiting requested review from a-b-r-o-w-n a-b-r-o-w-n is a code owner

@tdurnford tdurnford Awaiting requested review from tdurnford tdurnford is a code owner

@tonyanziano tonyanziano Awaiting requested review from tonyanziano

+2 more reviewers

@corinagum corinagum corinagum approved these changes

@stevengum stevengum stevengum left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Pre-release: Need Manual Testing

Projects

None yet

Milestone

No milestone

5 participants

@compulim @cwhitten @coveralls @corinagum @stevengum