Hello! I have a photo with audio from an mp4 file and used text for subtitles. I have the subtitles be hidden/visible on the timeline to come in at the correct time. When you begin the scene, the audio automatically begins and the timeline begins too. So, as long as the audio isn’t paused, the subtitles will be correctly timed. If you pause the audio, the text continues on the timeline, though. How can I link the subtitles to play/pause of the video?
Thank you!!
Carey
Unfortunately synchronization isn’t something that Hype automatically handles. A solution depends on how you are playing back the audio, so feel free to post a zip of your .hype document if you need more details. Either way, it will require some code.
If the audio is an HTML element and has controls, you’d need to listen to the onpause event and at that point call hypeDocument.pauseTimelineNamed() from the JavaScript API. Alternatively if you are pausing via a Hype action, then you’d need to access the audio element and call pause() on it.
That’s the basics anyways! Even with this, there isn’t a guarantee that the audio and timeline will be perfectly in sync. To do a better job of this you’d also need to look at the audio or animation timeline and set them to be equivalent in pause situations.
Hi Jonathan!
Thank you for the reply! I’m really new to web development and Tumult and, to be honest, I got a little lost in what you said. Where I would set the onpause events and the other code you mentioned… I’d like to share the zip of my .hype document but I’m in Bolivia right now and the internet isn’t uploading the document. Is there another way I can send a zip file?
Hi Carey!
Here is an example of one way to sync video & sub-titles: VideoSyncDemo_JHSv1.hype.zip (2.3 MB)
In this demo the video & Hype project timelines are synced for "Play" and "Pause" operations; additionally You can "scrub" the video playhead back and forth on the video media player's timeline and the Hype project timeline sets itself to match the video.
Demo here.
Upon opening the Hype project You will see in the "Main Timeline" layers a rectangle element called "VideoHolder"; in the innerHTML of that element is the following script:
This code in the innerHTML is called a "video tag". We will manipulate the video using this approach instead of dragging the video directly into Hype's main animation window.
I have given this video tag an ID of "prologue" - You would use your own ID name. Also I have set the height, width and other parameters of the video. The source pathway of the video is identified in the tag as:
source src="${resourcesFolderName}/prologue.m4v"
The resourcesFolderName is a substitution variable for the document's resources folder and is used to reference documents added via Hype's Resource Library.
prologue.m4v is the name of my video file - replace it with the name of your video.
So, at this point we have covered the innerHTML of the rectangle element used to contain the video file. One more thing - the code to sync the video with the timeline.
The function "syncTime()" appears in a tab just above Hype's main animation window - click on this tab to view this function in the code editor.
Since it appears You are brand new to JavaScript coding, etc. there is no point in going into details of the code at this time. The only thing for You to do here is replace the ID in the line near the top:
window.myVideo = hypeDocument.getElementById('prologue');
Replace prologue with the ID named You used in the video tag of the innerHTML of the VideoHolder rectangle element - make sure You keep the quotes around the name.
The function is triggered by the "On Scene Load" event which is found in the "Scene Inspector" near the bottom of the Scene Inspector's pane:
We're done!
Note: You can use an audio file instead of a video, and use animated images that are synced to the audio's timeline. (What follows below is not in the Demo file - it is an example only in this post).
The code is almost identical - the only difference is a slight variation in the innerHTML of the VideoHolder element:
The video tag now becomes an audio tag and a different file type, in this case ".mp3", type="audio/mpeg":
<audio id="prologue" width="60%" preload="auto" controls="controls">
<source src="${resourcesFolderName}/prologue.mp3" type="audio/mpeg">
</audio>
Since an audio file has no "official" height such as a video would have, we will use the height of the browser's default audio player (though You could change this situation - but that's for another post). Also we will set the width of the audio media player to a percentage of the VideoHolder element that contains it. Above I used 60% - i.e. the audio player is 60% of the VideoHolder's width.
The code in the "syncTime()" function would stay the same, but many of the variables have "video" in the name (i.e. "myVideo"). Everything will still work but You may wish to change the variable name to something like "myAudio" - or "myMedia" - if You are using both video & audio... it is a good practice to have variable names match the mission (or at least not be misleading).
Hi Jim! Thank you so much for your very detailed and clear directions. This is exactly what I was looking for! I don’t know why, but it isn’t allowing me to enter the inner HTML of the video element, but I was able to give the element a video id tag under the “identity” section.
One thing that is different is that - looking at the photo I attached - I am using a static photo/image with an mp4 file, but the “video” file is only audio. I did this instead of using the audio element because of the video controls that I want for the audio.
I followed your instructions to run the javascript and add the code on “on scene load” with the video id tag. It works so that when I scrub through the video, the subtitles change with it. But, now it isn’t allowing me to pause the video! Any idea why that is?
This is what the script looks like
window.myVideo = hypeDocument.getElementById('video1');
myVideo.addEventListener('timeupdate', syncTimelines);
function syncTimelines() {
videoTime = myVideo.currentTime;
hypeDocument.goToTimeInTimelineNamed(videoTime, 'Main Timeline');
if (!myVideo.paused) {
hypeDocument.continueTimelineNamed('Main Timeline');
}
else if (myVideo.paused) {
hypeDocument.pauseTimelineNamed('Main Timeline');
}
}
Thanks so much!
Carey
Hi Carey!
I have a fairly good idea of what the issue is, but rather than guess it would be best for me to see your Hype project file. The Hype forum has a limit of approximately 3 MB per upload.
If your file is under this then ZIP it up and post here - or if larger than 3 MB - maybe You can remove some of the media to get it under this limit without changing the nature~function of the layout (such as shortening the length~number of pix, trimming the length of the audio, etc.)
Alternately You could Private message me with a download link to a file transfer service such as Dropbox, etc.
Thank you so much for this code! It really saved me yesterday. I had an interactive video with various click zones that the user could tap to go to different scenes (think, choose your own adventure). Client approved and was going to demo it at in a presentation at a trade show today. However, at the last minute we found out they were displaying it on an iPad!
Unfortunately, the autoplay flag doesn’t work on mobile browsers, so at the last minute I had to completely change approach. I compiled all the scenes into one with a single video and a “click to play” at the beginning. Because it’s user initiated, the browser then allows playback control with javascript after that. BUT, then the selection buttons weren’t popping up at the right times because the video would play independently from the Hype timeline.
I know this is an old thread but I just wanted to acknowledge how helpful this was and I hope that Hype ads a built in option like this for syncing in the future.
This is actually the functionality I wanted to try, particularly ensuring mobile sync works. Did you notice a consistent offset or did the video sync drift over time?
Welcome to the Forum!
The video sync should not drift over time. The “timeupdate” event which forms the core of the syncing mechanism is updated every time the playback position is updated.
However, this kind of processing is not free - if there is a significant system load the processing time can vary (i.e. frequency of the “timeupdate” event) which is dependent on the user agent to manage.
Please see this link for more info:
Mich obliged @JimScott! I am hoping to be able to sync up a few different media using videojs and perhaps mojs.io and/or greensock powered animations. I have experience with a player based on popcorn.js and sync across systems can be tricky as you noted. Assuming i can keep elements in sync without drift, even if it isn’t smooth, that would be a win.
Thanks again!
