Sync animations to Video & Audio in Tumult Hype


#1

In this quick tutorial, I’ll show how to start a timeline after a media element (in this case, a video) has played for five seconds. This example could be easily modified to do the following things:

  • Jump to the next scene after a video has played
  • Run a timeline after an audio element has played half way through
  • Play another audio element after one has completed

To sync your video with an animation in Tumult Hype, we’ll first need to detect the current position of your video. This is known as the timeupdate value, and is well supported in modern browsers. Next, we’ll need to set specific actions using the JavaScript API in response to different timeupdate values.

In the example below, we’ll be detecting the current time of a video, and then starting a timeline when 5 seconds have elapsed.

(Please download the attachment at the bottom of this article to see this JavaScript in action).

This function also works with audio files that are contained within a standard audio element – timeupdate is a value generated by both audio and video elements. 

// This establishes the T1 variable as an integer.
var T1 = 0;

// Listen to the 'timeupdate' value: 
hypeDocument.getElementById('video1').addEventListener('timeupdate', function() {
var currentTime = hypeDocument.getElementById('video1').currentTime;

// If the current time is greater than 5, run the RedRobin timeline. 
if (currentTime >= 5 && T1 == 0) {
hypeDocument.startTimelineNamed('RedRobin');

// This ensures that the 'startTimelineNamed' action only occurs once, not at every point beyond time = 5. 
T1 = T1 + 1;
}
});

The hypeDocument.startTimelineNamed('RedRobin'); function can be replaced with virtually any action you would like to trigger. Please see the JavaScript documentation for options.

Running multiple actions at different times

Another option: Perform an action at multiple time periods (5 seconds and 10 seconds, for example) by adding additional ‘T’ variables:

// This establishes the T1 variable as an integer.
var T1 = 0;
var T2 = 0;

// Listen to the 'timeupdate' value: 
hypeDocument.getElementById('video1').addEventListener('timeupdate', function() {
var currentTime = hypeDocument.getElementById('video1').currentTime;

// if the current time is greater than 5, run the RedRobin timeline. 
if (currentTime >= 5 && T1 == 0) {
hypeDocument.startTimelineNamed('RedRobin');

// This ensures that the 'startTimelineNamed' action only occurs once, not at every point beyond time = 5. 
T1 = T1 + 1;
}
if (currentTime >= 10 && T2 == 0) {
hypeDocument.startTimelineNamed('ParrotParty');

// This ensures that the 'startTimelineNamed' action only occurs once, not at every point beyond time = 10. 
T2 = T2 + 1;
}
});

Example document: syncVideoWithAnimations.hype.zip (4.2 MB)

Detecting the end of a video or Audio element

Run the following script (in Hype’s built in JavaScript editor) to detect when an element has completed playing:

    var video = hypeDocument.getElementById('video1');    
    video.onended = function(e) {
      /*Do things here!*/
    };

For more HTML5 Media events, visit the W3.org page on the topic, or jump straight to the ‘timeupdate’ event.


Let videos execute actions upon end or when set frame is reached
Getting .mp4 to play on another timeline
Syncing soundcloud widget with timeline
Syncing animation to fixed audio track
How would I use getCurrentTime() of video to add text to the scene at a specific time?
Start timeline when audio has finished loading
Knowledge base: Guides from the Tumult Team
(Bas) #2

Hi Daniel,

This is all great, really helpful. Now I have one question. I have looked at and played around with your example and have noticed that when I replay the video the actions don’t reappear. Is there a way to make it so when a user scrolls the video timeline back that he will again see these animations without having to reload the whole project/page.

Looking forward to your suggestions.
Thank you,
Bas


#3

In the code above I’m checking to see if one of the two triggers have run by incrementing the value:

if (currentTime >= 10 && T2 == 0) {
hypeDocument.startTimelineNamed('tenseconds');

// This ensures that the 'startTimelineNamed' action only occurs once, not at every point beyond time = 10. 
T2 = T2 + 1;
}

If you wanted this ‘tenseconds’ timeline to run again if the video’s playhead advances again past 10s, then you can add this function. Make sure it is within the last });

if (currentTime > 10 && T2 == 1) {
hypeDocument.startTimelineNamed('tenseconds');

// This ensures that the 'startTimelineNamed' action runs again if it has already run. 
// (T2's value is 1 if it has already run.  
T2 = T2 + 1;
// After running a second time, T2 will = 3. 
}

I’m sure there’s a smarter way to do this, but that’s one solution.


(Bas) #4

Hi Daniel,

Thank yo uso much for looking into this, however this does not completely answer my question. What I’m trying to achieve is that when I play the video, the animation plays once, like in your first example. After the video has played I rewind it and play it again. The animation does not play the second time after scrubbing back to time = 0 in the movie.

I hope its clear what Im trying to achieve. An animation that occurs every time I play the movie at the same moment in the movie, also after quickly scrubbing back just before the moment the animation is triggered.


#5

Hi Bas,

You need to fire some sort of even like a button click that rewinds the video but also resets the T1 or T2 variable

or maybe the video.onended call above you can reset the variables there.

or another if statement like

if (currentTime  < 1) {
T1 = 0;
T2 = 0;
}

What is happening is that the second time around the T1 or T2 variable is 1 (or increased by 1) so the if else statement doesn’t run true.

D


(Bas) #6

I now solved it by pausing the “fivesecconds” timeline just after it has finished playing and than resetting the T1 to 0.
This is not ideals and I would really just want to pause the timeline “fivesecconds” after it has reached its end, any suggestions on that?

if (currentTime >= 7 && T1 == 1) {
hypeDocument.pauseTimelineNamed(‘fivesecconds’)
T1 = 0;

}


(Matt) #7

Hi @Daniel -
I’m trying to do something seemingly very simple with the code you’ve provided above.

When my video gets to a certain time I am triggering a Hype javascript and that works great!

However, what I’d like to do is have the video pause also. I’ve tried putting

hypeDocument.getElementById('video1').pause();

In the initial javascript function like this:

	// This establishes the T1 variable as an integer.
var T1 = 0;

// Listen to the 'timeupdate' value: 
hypeDocument.getElementById('video1').addEventListener('timeupdate', function() {
var currentTime = hypeDocument.getElementById('video1').currentTime;

// If the current time is greater than 5, run the RedRobin timeline. 
if (currentTime >= 10 && T1 == 0) {
//hypeDocument.startTimelineNamed('RedRobin');

hypeDocument.functions().playSteps(hypeDocument, element, event);
hypeDocument.getElementById('video1').pause();


// This ensures that the 'startTimelineNamed' action only occurs once, not at every point beyond time = 5. 
T1 = T1 + 1;
}
});

As well as in the javascript function as below and neither one pauses the video. Any thoughts?

	window.scoreView.setPartProperties(1, {muted:true});
	 window.scoreView.setPartProperties(0, {visible:true});
	 window.scoreView.setPartProperties(1, {visible:false});
	 
	window.scoreView.playFromMeasure(0)
	
	hypeDocument.getElementById('video1').pause();

#8

That’s odd – can’t imagine why that wouldn’t work. Do you see any errors in the browser console? Happy to take a look at your document.


(Matt) #9

I figured it out! It was a simple syntax error. Thanks for your time :slight_smile:

Matt