Preload audio for sync with timeline


(theron_hp) #1

Hi Everyone.

I feel like I’m always chasing the same stuff, but here goes: I have many audio files that play with different timelines depending on a possible 7 selection parameters in the project.

I’m loading the audio with javascript: window.Tie_1 = new Audio('${resourcesFolderName}/Tie_1.m4a');

I trigger the audio with javascript for example with conditions:

  	else if ((boombox.currentTimeInTimelineNamed('1_T1') != 0) && 
  				(boombox.currentTimeInTimelineNamed('1_T2') == 0) && 
  					(boombox.currentTimeInTimelineNamed('1_T3') == 0) && 
  						(boombox.currentTimeInTimelineNamed('1_T4') == 0) && 
  							(boombox.currentTimeInTimelineNamed('1_T5') == 0) &&
  								(boombox.currentTimeInTimelineNamed('1_T6') == 0) &&
  									(boombox.currentTimeInTimelineNamed('1_T7') == 0)){
			
				Tie_1.play();
				hypeDocument.startTimelineNamed('Tie1', hypeDocument.kDirectionForward);
			
			}

How do I preload the audio, so that it stays in sync with my animated timeline the first time and not only from the second time?
There are quite a few conditions for the triggering of which audio clip to play that is set up with javascript. When they are met, it triggers a specific audio file.

Thanks!


#2

Here’s a few ways to approach preloading audio:

  1. Creating a new audio element binds that potential source to the element, but unless you run a ‘play’ function, the URL of the audio file will not be hit. Browsers might get the metadata for the audio element (with information like that duration) but you won’t have that data in memory. To ‘preload’ the audio you would need to play the audio element briefly and immediately pause it. This will tell the browser to start downloading it because the data is needed. So my first suggestion would be to find a way to play then pause the audio element before you need it.

  2. Add Preload=auto.

Adding this property in some browsers will start to download the file:

window.Tie_1.preload='auto';

  1. You can also detect whether the canplaythrough media event is fired. (In combination with #2 above).

The canplaythrough event is fired when the user agent can play the media, and estimates that enough data has been loaded to play the media up to its end without having to stop for further buffering of content.

via: https://developer.mozilla.org/en-US/docs/Web/Events/canplaythrough

This will give you confirmation that your audio is ready to roll. That would look something like this:

window.Tie_1.addEventListener( "canplaythrough", function( e ) {

  console.log( "canplaythrough fired!" );

}, false );
  1. Use a Hype audio element and check ‘preload’ in the resources library. We use the Web Audio API which is a bit more capable in terms of preloading audio. You can access the preloaded file by using a regular HTML audio element. See here for example.

(theron_hp) #3

Thanks Daniel!


(theron_hp) #4

HI Daniel

So I drag the audio element off-scene with pre-load selected in the resource library. I give it an ID in the inspector of AllTie. I create a button on-scene and trigger the audio element with the following javascript: hypeDocument.getElementById('AllTie').play();

To no avail. Please save me from my ignorance!

H


(theron_hp) #5

The un-working example.
Untitled.hype.zip (115.2 KB)


#6

Apologies, I gave bad advice :frowning:

In the attached document I left the JS action in place, but I created an off scene rectangle containing this:

<audio id="allTieInnerHTML" height="100" width="400" preload="auto">
  <source src="${resourcesFolderName}/AllTie.m4a" type="audio/mpeg">
</audio>

The ‘preload’ property in combination with the preloaded option in the resource library should result in immediate playback when clicking the button.

You can see the preload in action by watching the Network tab in Chrome’s developer tools.

preloadAudio.hype.zip (115.5 KB)


(Hans-Gerd Claßen) #7

… or simply place a audiostart on a timeline and run this timeline with hypes API … preloading checked

but using hypes builtin audio with preloading-option will result in preloading all of your audio before your page is shows …


#8

Yeah that would also def work, but sounds like he’s got a setup difficult to manage with just timelines:


(Hans-Gerd Claßen) #9

also howler may simplyfie things https://howlerjs.com/


(theron_hp) #10

Thanks Daniel and Hans

I trigger it like you set it up for me Daniel, but in a mobile device it still does not pre-load. In general there aren’t any issues on my laptop or desktop, but to get iOS or Android to pre-load the sound or buffer the sound, is the main issue.

Can Howler achieve this Hans? I had a look and I can’t get it to trigger audio:
howlerAudio.hype.zip (131.4 KB)


(Hans-Gerd Claßen) #11

howlerAudio.hype.zip (134.5 KB)
first: to give preload a chance and allow direct audio playing on mobile you’ve got to setup the howls before playing :slight_smile:

howler on mobile: https://github.com/goldfire/howler.js#mobile-playback

good luck :slight_smile:


(theron_hp) #12

I tried h_classen’s method of triggering the sound in a timeline, but was skeptical because I thought sound will not trigger on mobile devices in this way. Oh boy was I wrong![quote=“h_classen, post:7, topic:10693”]
… or simply place a audiostart on a timeline and run this timeline with hypes API … preloading checked
[/quote]

The timeline trigger for audio just has to be the first thing that is triggered in the timeline and it can’t be grouped with other actions.

I’m also looking at howler.js
It opens up other possibilities.

Thanks again!


#13

Looks like you’ve got something going but just to add a little bit in here and to highlight what Hans has said…

I personally would change the approach and rather than sync the audio to the timeline I would do it the other way around, sync the timeline to the audio… bear with me!

What i mean is when you want to play the audio and timeline together. Start the audio and then look for the “canplaythrough” that has been mentioned and then when this fires play the timeline. The added bonus of this is that you don’t need to preload the audio which if you have many will impact on the load time of the entire document. I did this in a few documents (quite a while ago) and it worked fine.


(theron_hp) #14

This is an excellent idea. Thanks. In theory I understand, but in practice I’m struggling. So Daniel gave a script. At what level do I implement it? You have helped me before in previous projects and I will be grateful if you could show me how to implement. I’ve attached a project with an audio resource and a script. Now I’m holding my breath.
canPlayThrough.hype.zip (116.6 KB)


#15

If you look at the file the gentleman known as @h_classen provided here, your audio element is installed in the head using this code:

	<script>	
	var sound = new Howl({
    src: ['${resourcesFolderName}/AllTie.m4a']
    });
    </script>

He also added the howler.js file to the resources library, which automatically loads it when the document is loaded.

Howler is preloading the audio file if you look at Chrome’s developer tools / network tab:

And also in Mobile Safari, it Howler is the ‘initiator’ of a webaudio api preload (the same method that Hype uses)

So using this technique on mobile your audio should be preloaded. You just need to run sound.play() (or whatever.play() when you want to play the sound.


(theron_hp) #16

Thanks @Daniel. I was able to implement with @h_classen 's help the howler.js. I’m also interested in the canplaythough solution from you and @DBear. From where I was last night, pulling out my hair, to now where there are a few possible solutions is because of the help I’m getting! Thank you.


#17

@theron_hp

Here is your document re-worked. I have also added a text animation to show you the syncing of the sound with the animation.

This way … the animation in Hype waits for the start of the audio before playing. That way … you don’t have the problem of the sound buffering causing a mismatch between the animation and the animation playing before the sound.

Interesting things to know – 1 frame (at 30fps) is roughly 0.33 seconds so you can work out how to sync up sounds with the timing if you need.

canPlayThrough-vDBear.hype.zip (120.8 KB)


(theron_hp) #18

No smiley emoji does justice to smile right now. Thanks @DBear, @Daniel and @h_classen!


(theron_hp) #19

Hi. @DBear and @JimScott.

It’s been a while, but I’m trying to create something with the Wad Audio library and something’s not right.

According to the library one must be able to set parameters during playback, but for some reason I can’t get it to work.

This is what it says:
Changing Settings During Playback

If you want to change an attribute of a Wad during playback, you can use the relevant setter method for that attribute.

saw.play() saw.setDetune(-50)

By default, this change will occur smoothly over 10 milliseconds, to help prevent ugly clicks and pops. Optionally, you may pass in a second parameter to specify the duration of this transition, in seconds.

saw.play() saw.setPanning(1, .015) // pan to the right over 15 milliseconds

I’m including a project which is my attempt to not stop, but reduce(mute) the audio level.
wadSetParamater.hype.zip (440.2 KB)

Your assistance is appreciated.

Thanks


#20

I see this error when hitting Mute:

Gain.gain.value setter called at time 13.39791383219955 overlaps event linearRampToValueAtTime(1, 2.015893015873016) to linearRampToValueAtTime(1, 8002.015903015873)

So hold is setting the volume which restricts any other function from adjusting the volume. Odd. Why not just use the ‘stop’ function which would do the same thing?

I did notice that changing ‘hold’ to ‘sustain’ allows you to mute the track, but it does not allow for endless looping which I think you want to keep.