GWT use webaudio instead of SoundManager2 lib by MrStahlfelge · Pull Request #5659 · libgdx/libgdx

MrStahlfelge · 2019-06-05T20:00:54Z

New PR for WebAudio implementation by @barkholt (original PR was #4220).

I picked the important commits of this PR and merged it against the latest libgdx versions. After testing, I can safely say that this is better than SoundManager2.

Some advantages I found so far:

Supports unlocking the sound system on user input and thus works on mobile and is future proof for desktop
Prefetches sounds when they are loaded, so they are played without a lag
No extra js files needed anymore
Direct access to the Web audio API lets us use its rich featureset
Pitch is working

Caveats I found so far:

While autoplay (playing sounds before the user made an input) works on desktop browsers with SoundManager2, it does not work with this implementation if the browser or user does not allow it. A user input is needed for sounds and music to play. We can expect that it will also not work anymore with SoundManager in the future, so I think this is not a reason not to use this API. (Game controller interaction is not counted as user interaction by Chrome and Firefox, so user needs to click or tap or press a key - this is more a wrong behaviour of the browsers)

Live example to try

Tested so far on

For those wanting to try or to use this in existing 1.9.8/1.9.9/1.9.10 games: you can find Jitpack dependencies to use with libGDX 1.9.8 (working with 1.9.9, too), 1.9.10 and 1.9.11 here.

About the changes on the Eclipse files: I don't know if they are needed because I don't use Eclipse. If needed, I can revert the changes to .project/.classpath files.

Changes.md:
I did not update changes.md so far because I am not sure if this will get merged (soon) and are afraid of merge conflicts. The following should be added there when this gets merged.
* HTML: changed audio backend to WebAudio API. Now working on mobiles, pitch implemented. Configuration change: preferFlash removed. When updating existing projects, you can remove the soundmanager js files from your webapp folder and the references to it from the index.html

# Conflicts: # backends/gdx-backends-gwt/src/com/badlogic/gdx/backends/gwt/GwtApplication.java # extensions/gdx-controllers/gdx-controllers-gwt/.classpath

# Conflicts: # backends/gdx-backends-gwt/src/com/badlogic/gdx/backends/gwt/GwtApplicationConfiguration.java

mgsx-dev · 2019-06-05T20:30:41Z

About Eclipse files modifications, i checked with my Eclipse configuration (i only have GDT Google Dev. Tools) and eclipse doesn't complain about your changes, i think it's safe to keep it. It could be useful for people using these other plugins.

mgsx-dev · 2019-06-05T22:53:20Z

First of all, thank you to bring this up! I made a few tests with your PR using libgdx tests.

AudioContext is created too early and print this warning :
The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu
It's not easy to reproduce : i had to open a new tab (without chrome console) and then open console in order to see the warning. I can't reproduce it by refreshing the page while chrome console still opened.

Even though it's just a warning, there is some issue cases. My test case was to just play a sound or a music when clicking on any test launcher button (first user interaction).

with sounds there is no issues : as per google documentation, audioContext is automatically resumed if start is called on any node in the audio graph (which is what happens).
with music there is an issue : playback doesn't work.

Music issue can be fixed either :

by calling resume on audio context
or by recreating audio context (and audio graphs).

I quickly test by resuming audio context just before playing the empty sound here and it worked.

I'm not sure if it's possible to get rid of this warning. Some people are always afraid about simple warnings (eg Gradle compile 🙄 ).
We could create audio context after first user interaction but it would mean it requires a user interaction again when the user load the page later. I'm not sure if it's a good solution.

For now, I think it's safe to just resume audio context after first user interaction.

barkholt · 2019-06-06T07:09:17Z

First of all, thank you @MrStahlfelge and @mgsx-dev for reviving this, I had all but given up on it :)

In my opinion, the warnings associated with attempting to start playback before a user interaction, is not really feasible to solve from our end. The developer needs to be aware of this, much in the same way as they have to request the right permissions to use hardware features, if they do mobile development.

Furthermore, there are many cases where the request for user interaction is not required - eg. if you wrap your HTML5 game using Cordova, Electron or if you support "Add to home screen".

I am not sure, but maybe the Chrome developers are coming to our rescue here?
(https://developers.google.com/web/updates/2018/11/web-audio-autoplay)

Source nodes commonly represent individual audio snippets that games play, for example: the sound that is played when a player collects a coin or the background music that plays in the current stage. Game developers are very likely to be calling the start() function on source nodes whenever any of these sounds are necessary for the game.

Once we recognized this common pattern in web games we decided to adjust our implementation to the following:

An AudioContext will be resumed automatically when two conditions are met:
The user has interacted with a page.
The start() method of a source node is called.
Due to this change, most web games will now resume their audio when the user starts playing the game.

MrStahlfelge · 2019-06-06T08:14:57Z

Hi @barkholt , thanks for engaging here and for the good work. I can understand that you were demotivated. Perhaps you want to join the libGDX discord server linked on the main page, makes the discussion easier. :)

At the moment, we are still trying to improve on the autoplay problem. When testing, I discovered that your original solution had the problem that, when the context gets unlocked, it plays all sounds at once that were tried to play so far. I changed that with 073a953, but this commit will also suppress playback if audio is not locked at all. EDIT: solved by 7e661eb

fix autoPlay for musics fix obtain graph from pool instead of create a new one removed unused constructor revert line ending changes for readability in history avoid implicit int to float conversion for consistency fix music completion listener fix documentation

…first interaction otherwise

intrigus · 2019-06-08T19:50:34Z

+		this.audioContext = audioContext;
+		this.destinationNode = destinationNode;
+
+		setupAudoGraph();


intrigus · 2019-06-08T20:18:01Z

+	@Override
+	public void pause () {
+		// As the web application looses focus, we mute the sound
+		disconnectJSNI();


Would it be possible to change this dynamically?
I.e. allow to not mute on focus loss?

intrigus · 2019-06-08T20:18:45Z

+
+	public void ended()
+	{
+		if(this.onCompletionListener != null) this.onCompletionListener.onCompletion(this);


Maybe remove this?

intrigus · 2019-06-08T20:19:28Z

+
+	@Override
+	public void setOnCompletionListener (OnCompletionListener listener) {
+		this.onCompletionListener = listener;


Maybe remove this?

intrigus · 2019-06-08T20:24:58Z

+		audioBufferSourceNode.loop = looping;
+	}-*/;
+
+	public static native boolean getLoopingJSNI (JavaScriptObject audioBufferSourceNode) /*-{


Maybe rename to isLoopingJSNI?

intrigus · 2019-06-08T20:25:35Z

+	}-*/;
+
+	public static native float getResumeOffsetJSNI (JavaScriptObject audioBufferSourceNode) /*-{
+		return audioBufferSourceNode.pauseTime


Is this formatting intended?

intrigus · 2019-06-08T20:34:41Z

Nice work @barkholt @MrStahlfelge @mgsx-dev .
Some parts of the source code would need a tour of the formatter I think :)
Also there are different styles used throughout the code:

if(foo)
          bar()
     else
          bart()
vs.
if(foo) {
          bar()
     } else {
          bart()
}

BoBIsHere86 · 2019-09-15T15:50:45Z

Just wanted to say Thanks to @MrStahlfelge for your work on this and keeping it alive. The sound issues in my HTML5 projects were causing major headaches on Mobile platforms with the inability to have more than one Music or Sound playing simultaneously. Every time a sound was played it would cause the previously playing sound to stop.
Switching over to this back end completely resolved the issue.

The only thing I would like to point out is that, if this were to ever get merged. In HtmlLauncher.java, the re-size code that's commented out has this line "cfg.preferFlash = false;"
This no longer exists with the removal of sound manager and the new back end so it would have to be removed to avoid a confusing error.

crykn · 2020-04-27T00:30:20Z

This PR would probably be a good candidate to be added as an alternative implementation after #6017 was merged.

MrStahlfelge · 2020-08-20T19:49:03Z

@crykn I don't see this as an alternative implementation for SoundManager because SoundManager just does not work as intended any more. There's no thing in keeping it, and that's the reason why I won't publish this as an add-on or so.

That said, I don't understand why this didn't get merged.

# Conflicts: # backends/gdx-backends-gwt/src/com/badlogic/gdx/backends/gwt/GwtApplication.java # backends/gdx-backends-gwt/src/com/badlogic/gdx/backends/gwt/GwtAudio.java # backends/gdx-backends-gwt/src/com/badlogic/gdx/backends/gwt/soundmanager2/SoundManager.java # extensions/gdx-setup/res/com/badlogic/gdx/setup/resources/html/war/soundmanager2-jsmin.js # extensions/gdx-setup/res/com/badlogic/gdx/setup/resources/html/war/soundmanager2-setup.js

tommyettinger

This has been in use by projects that use MrStahlfelge/gdx-backends for a long time, and it seems quite stable. With Flash essentially unusable either now or very soon, an alternative is needed, and this seems like
a very good one.

SimonIT

I tested it successfully on Google Chrome (Android and Windows), Firefox (Android and Windows), Edge (Windows) and Samsung Internet Browser

MrStahlfelge · 2020-08-29T19:02:20Z

It is done. Thank you for the efforts and patience to all contributors in this case. To preserve the original author information, I did not sqash the merge this time.
@barkholt , no need to use a fork anymore. If you have any more improvements or additions to GWT, submit a PR or let us know via Discord.

barkholt and others added 10 commits August 8, 2016 01:47

Added Web Audio API support to GWT backend

d48f180

Update new GWT audio branch to latest version of libGDX

100d180

fixes to volume control in music

599019d

remove unused code

1706ebc

fix problems with callback, and removed debug

b62eb78

Removed SoundManager2

cbc3079

Merge branch 'release/1.9.8' into gwtaudio_webaudio

d108079

# Conflicts: # backends/gdx-backends-gwt/src/com/badlogic/gdx/backends/gwt/GwtApplication.java # extensions/gdx-controllers/gdx-controllers-gwt/.classpath

Merge remote-tracking branch 'libgdx/master' into gwtaudio_webaudio

a3ce59c

# Conflicts: # backends/gdx-backends-gwt/src/com/badlogic/gdx/backends/gwt/GwtApplicationConfiguration.java

WebAudio better behaviour for unlocking sounds

073a953

Merge branch 'gwtaudio_webaudio' into libgdx_gwtaudio_webaudio

6962a75

mgsx-dev reviewed Jun 5, 2019

View reviewed changes

Comment thread backends/gdx-backends-gwt/src/com/badlogic/gdx/backends/gwt/webaudio/WebAudioAPIMusic.java Outdated

mgsx-dev reviewed Jun 5, 2019

View reviewed changes

Comment thread backends/gdx-backends-gwt/src/com/badlogic/gdx/backends/gwt/webaudio/WebAudioAPIManager.java

mgsx-dev and others added 2 commits June 6, 2019 16:18

Webaudio API: autoplay when allowed by user agent, unlock context on …

7e661eb

…first interaction otherwise

intrigus reviewed Jun 8, 2019

View reviewed changes

MrStahlfelge mentioned this pull request Jul 28, 2019

Changed mp3 required to false #5730

Closed

crykn mentioned this pull request Mar 26, 2020

Sound and Music do not function on iPhone 11 in HTML5/GWT game. #5954

Closed

7 tasks

MrStahlfelge added 2 commits August 22, 2020 16:15

Conflicts with 1.9.11 resolved

58c1f19

MrStahlfelge added gwt non-breaking Label for non-breaking PRs labels Aug 27, 2020

MrStahlfelge requested review from crykn, mgsx-dev and tommyettinger August 28, 2020 08:26

tommyettinger approved these changes Aug 28, 2020

View reviewed changes

MrStahlfelge added the audio label Aug 28, 2020

MrStahlfelge requested a review from Tom-Ski August 28, 2020 15:32

SimonIT approved these changes Aug 29, 2020

View reviewed changes

MrStahlfelge added 2 commits August 29, 2020 20:45

Merge remote-tracking branch 'origin/master' into gwtaudio_webaudio

f4bdc2b

Added GWT webaudio switch to CHANGES

9ab569b

MrStahlfelge merged commit fa2f3b5 into libgdx:master Aug 29, 2020

MrStahlfelge deleted the gwtaudio_webaudio branch August 29, 2020 19:00

Uh oh!

Conversation

MrStahlfelge commented Jun 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mgsx-dev commented Jun 5, 2019

Uh oh!

Uh oh!

Uh oh!

mgsx-dev commented Jun 5, 2019

Uh oh!

barkholt commented Jun 6, 2019

Uh oh!

MrStahlfelge commented Jun 6, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

intrigus Jun 8, 2019

Choose a reason for hiding this comment

Uh oh!

intrigus Jun 8, 2019

Choose a reason for hiding this comment

Uh oh!

intrigus Jun 8, 2019

Choose a reason for hiding this comment

Uh oh!

intrigus Jun 8, 2019

Choose a reason for hiding this comment

Uh oh!

intrigus Jun 8, 2019

Choose a reason for hiding this comment

Uh oh!

intrigus Jun 8, 2019

Choose a reason for hiding this comment

Uh oh!

intrigus commented Jun 8, 2019

Uh oh!

BoBIsHere86 commented Sep 15, 2019

Uh oh!

crykn commented Apr 27, 2020

Uh oh!

MrStahlfelge commented Aug 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tommyettinger left a comment

Choose a reason for hiding this comment

Uh oh!

SimonIT left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MrStahlfelge commented Aug 29, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

MrStahlfelge commented Jun 5, 2019 •

edited

Loading

MrStahlfelge commented Jun 6, 2019 •

edited

Loading

MrStahlfelge commented Aug 20, 2020 •

edited

Loading

SimonIT left a comment •

edited

Loading