Video/screen issue with Surface Go 2 when doing transitions

,
1

We are building a kiosk app in Hype (4.1.7). Details are:

Target device: Surface Go 2 - 4GB
Windows 10 (up to date)
Chrome browser (latest version as of 3/11/2022) - in kiosk single app mode
Local files only (HTML page & hyperesources folder on desktop)

The application consists of 50 scenes in Hype, each containing a video, some text, and 3 navigation buttons (as images). These are set to previous scene, next scene, and home, which is an index landing page with buttons to navigate directly to each of the scenes. The scenes are set to 1280x1920px to match the Surface resolution.

The videos are typically 20MB each and play in a loop. This all worked fairly well in the initial tests.

Originally, we had no transition effect between scenes (set to instantaneous) but noted that sometimes rapid touches of the next or previous buttons (before the transition took effect) could cause the browser to hang.

We then added sliding left-right transitions between the scenes which solved that issue, but now we sometimes encounter flashing video and occasionally brief freezing of the screen.

We cannot duplicate this on a MacBook in preview mode, so the issue is likely a Surface/Windows 10 problem.

We are taking a close look at resource consumption in Windows to see if the Surface is running out of memory or CPU, but I thought we would ask here in case others have seen similar issues.

Thanks

2

I can't think of hearing about anything similar, but large documents often may encounter their own unique issues.

Would it be possible for you to upload/share a zip of your .hype document? If you need it kept confidential you can email it to support@tumult.com. If you can also provide any explicit steps from start to finish to help us reproduce that would be great (and/or a video with the steps should work too). Do you know if this happens on more "regular" windows machines? I don't have a Surface Go 2, but I do have a Surface Pro 4 that I can test on.

3

Thanks - the Hype doc is about 1.5 GB with all the videos, but I can create a smaller subset. I would imagine that a Surface Pro 4 has significantly more resources and may not have the same issues. I'll try to upload a modified version tomorrow.

BTW - if we can demonstrate that a Surface Pro works OK, I can lobby to purchase 2 of them. We have been using re-purposed Surface Go 2's for the project.

Thanks for your response - I'll get back to you!

4

I emailed a link (to support@tumult.com) to the full Hype document stored on Box.com. Let me know if you can download it OK.

Thanks

1 Like
5

A thought :thought_balloon:: Maybe reuse a single scene as a video player and hence reassign the video tag to avoid multiple videos clogging up memory. But that would probably depend on the design and if it is similar to begin with (in each scene).

6

Got it - thanks!

I tried running this on my Surface Pro 4 and the symptoms did not seem quite as severe, but I could see some stuttering that seemed to progressively get worse as I kept clicking next.

I would probably first try two techniques (both) that might improve the situation.

First, it seems like a lot of initial stutter is caused by the video trying to be played while the scene is sliding in. So what I'd do to delay this is probably turn off autoplay for the video and use a poster image. Then set the video to play on a timer that is equal to the transition speed.

Second, since there's probably some amount of resource utilization that is causing degradation over time, I would specifically make sure the video elements delete their sources when the scene unloads.

To add poster images, here's the basic steps:

  1. Grab a JPG that represents the first frame of the video. (For testing, I'd recommend just using one image for all videos, as capturing this might take a bit of work and it isn't worth investing the time until you know it works). I'm calling mine "poster-1.jpg" to represent that first video.
  2. Drag this JPG into the Resources Library, and uncheck "Automatically Optimize when Exporting"
  3. Select a video element, and in the Element Inspector, uncheck "Autoplay"
  4. In the Identity Inspector, add an Additional HTML Attribute with the key of poster and value of ${resourcesFolderName}/poster-1.jpg
  5. Repeat for all the scenes (or a small subset to begin with for testing)

To deal with the actual deletion of the video and proper restoration, you can just Edit the Head HTML and insert this code, without modification:

<script>

  function unloadCallback(hypeDocument, element, event) {
	var videos = element.getElementsByTagName("video");
	for(var i = 0; i < videos.length; i++) {
		videos[i].setAttribute("oldHTML", videos[i].innerHTML);
		videos[i].src = "";
		videos[i].innerHTML = "";
		videos[i].removeAttribute("src");
	}
  }

	var wasFirstVideoPlayed = false;
  
   function loadCallback(hypeDocument, element, event) {
   		var playVideos = (function () {
			var videos = element.getElementsByTagName("video");
			for(var i = 0; i < videos.length; i++) {
				var oldHTML = videos[i].getAttribute("oldHTML");				
				if(oldHTML != null &&  oldHTML != "") {
					videos[i].innerHTML = oldHTML;
				}
				
				videos[i].play();
			}
		});
		
		if(wasFirstVideoPlayed == false) {
			playVideos();
		} else {
			window.setTimeout(playVideos, 1100);
		}
  }

  if("HYPE_eventListeners" in window === false) {
    window.HYPE_eventListeners = Array();
  }
  window.HYPE_eventListeners.push({"type":"HypeSceneUnload", "callback":unloadCallback});
  window.HYPE_eventListeners.push({"type":"HypeSceneLoad", "callback":loadCallback});

</script>

I'd give that a try and see if it helps at all.

Clearly the poster images will change the visual behavior as it will revert to using these during the scene transition, so to some degree it is up to if you want this or not.

I think also @MaxZieb's method of using a single scene could help, but would require a lot of work to restructure the document for this.

1 Like
7

Thank you, Jonathan. This looks really promising. I appreciate the amount of thought put into this and I'll give it a try. It may take a few days to fully test. As an interim, we are removing the PREV and NEXT controls (leaving the HOME control) temporarily since the display is currently live. I'll post here with the results soon.

1 Like
8

Hi Jonathan. We deployed a new version with your script on 2 Surface Go 2's earlier this week, and it has been running flawlessly for 3 days, so we are pretty close to declaring success. This is in a live environment with museum visitors (several hundred a day) interacting with the tablets. Very few people have noticed the stutter between scene changes, although it is still there to a small extent.

Closing the video prior to the transition seems to have prevented the GPU in the Surfaces from getting overloaded, and probably makes the app run smoother overall.

I was starting to take a look at the second part of your suggestion - to slide in a poster image and delay the video start, but for some reason, I cannot turn off autoplay. If I uncheck it, the videos continue to start automatically with each scene change (at least this happens in Chrome preview).

Another thought I had was to move the first video frame to the right by at least 1 second, but the same thing happens - the videos play right away.

So if you have any further insights, they would be greatly appreciated, as is your script.

BTW - I am not sure if other transitions (e.g. Crossfade) would be equally good at what we want to do. Crossfade seems very smooth but not sure if it locks out the button actions as effectively as sliding transitions. I need to test that some more.

9

That's great to hear!

Hmm... you're welcome to send a link to your latest document and I can see how you set it up.

If you're using the script and setup from my comment above, then the transition would show the poster image, but then the video would start to load/play immediately when the transition is complete.

The performance issues (on my Surface Pro 4 at least) were in part because it was starting the video load during the transition causing small stutters, and then also getting progressively worse as computer resources were getting consumed. A crossfade can sometimes be a lighter transition, but without the actual hardware I'm not sure if I could say it would make a framerate difference. However, I think a crossfade would be less visible to the human eye if there was a stutter.

P.S. I won't have access to my Surface Pro 4 until Tuesday next week (I'm traveling now), but I should be able to give guidance in normal Chrome.

10

Just to be clear - we have not had the chance to implement both of your suggestions, only the script. The reason is we need to create 50 still images for posters and that will take some time. But so far the script alone seems to be a huge benefit. I am thinking of trying the script combined with crossfade and seeing if that works. Crossfade certainly seems to be more pleasing to the eye, and the script action of removing the video is hardly visible.

I have the same problem with access to hardware since the two Surface Go's are "in production" now and I have limited access. I might need to lobby for a 3rd Surface so we can more thoroughly test before launching.

Thanks for your continued support. I will email a link to the current version.

11

It looks like you still have autoplay checked in the versions you sent:

Screen Shot 2022-03-18 at 2.47.40 PM

You'll want to uncheck that.

Without a poster image though, it may do some amount of downloading to try to get the first frame to show something for the video. I'm not sure how much of a stutter that may (or may not!) cause. If it isn't too bad then you may be off the hook on poster images :slight_smile:.

12

I emailed a link to an experimental version with autoplay unchecked for some scenes and crossfade transitions (unchecking autoplay has no effect on my Macbook using Chrome preview). If the crossfade transition combined with your video unload script behaves on a Surface, we may go with that and not have to create 50 poster images. If we do go the poster route, we will probably extract the first frame from each video as the image, which should be seamless visually.

13

in case you'll do the posterimages ... a shell script using ffmepg could do the heavy work ...
this would create png's of the first frame of any mp4 in the directory the shellscript is located in ...


#!/bin/sh

OUTPUT_DIR=./

for FILE in *mp4
do
    FILENAME=`basename "${FILE}" .mp4`
    ffmpeg -i  "$FILE" -ss 00:00:00 -vframes 1 "$OUTPUT_DIR/$FILENAME.png"
done
3 Likes
14

Thanks - that is a pretty neat little script. I might use it if we go that route.

15

say you've got those posterimages within the resourcesfolder (and of course preloadattribute checked) you may be able to assign this posterimage on scenePrepareForDisplay ...

let posterImg = element.querySelector('video').querySelector('source').src.split('.').slice(0, -1).join('.') + '.png';
element.querySelector('video').setAttribute('poster', posterImg);

not tested, just a guess :slight_smile:

1 Like