3D + AR — A vision of the future…

Photics · January 28, 2024, 3:57pm

I haven't given it my full attention yet, but lately I've been researching how to do 3D & Augmented Reality in a web browser. It's surprisingly challenging, as Safari appears to be lagging in this area.

But, I did see something interesting…

…in the “Feature Flags” section of the Safari settings, the <model> element can be enabled. This makes it easy to add a USDZ 3D model to a web page.

photics-tv-voxel

It may not look like a 3D model, but it is! That USDZ file was used in a previous Photics.TV (YouTube) video…

(That example uses the "ar" relation, which is also kinda neat, but the browser support is limited. I think it's only Safari.)

So, there are problems…

Problem #1 — Browser support is lousy

Normally, I go to the caniuse website to determine if a new web feature is usable. As an example, something like Flexbox is now well supported. But, the model tag is so new that it's not even listed. Although, it is on the feature suggestion list…

github.com/Fyrd/caniuse

Add the `<model>` element

opened 10:30PM - 20 Dec 21 UTC

Schweinepriester

Support data suggestion

<https://github.com/WebKit/explainers/tree/main/model> Since it's being added… to Safari TP… <https://webkit.org/blog/12156/release-notes-for-safari-technology-preview-137/>: > ### Experimental Model Element > > - Added support for mouse-based manipulation of `<model>` on macOS ([r285986](https://trac.webkit.org/changeset/285986/webkit/)) > - Added audio support ([r286065](https://trac.webkit.org/changeset/286065/webkit/)) > - Added support for controlling looping animations ([r286066](https://trac.webkit.org/changeset/286066/webkit/)) > - Added support for getting and setting the camera ([r286019](https://trac.webkit.org/changeset/286019/webkit/)) > - Added support for pausing and resuming animations ([r286048](https://trac.webkit.org/changeset/286048/webkit/)) > - Added support for seeking animations ([r286068](https://trac.webkit.org/changeset/286068/webkit/)) > > ### Accessibility > > - Added accessibility attributes for `<model>` ([r286406](https://trac.webkit.org/changeset/286406/webkit/))

Problem #2 — The feature is in early development

Even though I can place the 3D model on a webpage, there's not much else I can do with it. The web documentation appears to be in the draft stages…

https://immersive-web.github.io/model-element/

Problem #3 — It doesn't work without the feature flags

While the model element is nice, most people aren't likely to have this feature turned on — not until it's widely supported by the web browsers. That's why this conversation is entitled… a vision of the future. It looks like 3D could be much easier to do in the future.

Although, I'm not sure if I can rely on the model tag. It seems that WebXR might be more suited towards what I'm looking to do. Again, Safari has feature flags for that. But when I tried to run an example on my iPad, it didn't work.

Basically, I'm planning to build games in 3D / AR, but cross platform friendly. Web tech doesn't seem ready for that just yet, but perhaps I can start building regular 3D games with the idea of adding an AR feature later.

So, what does this have to do with Hype?

Well, basically Hype is a WebView… right? While I built the test in Hype, and it worked when previewing in the browser, the model does not display in Hype. That's when I thought… hmm, perhaps feature flags should be a setting in Hype too.

Perhaps the extreme delay in the launch of Hype 5 is a lack of eye-catching features. Well, perhaps Hype could jump on the AR/VR bandwagon. Since Hype is a Mac exclusive… and Apple is going heavy on AR/VR… then perhaps that's a way for Tumult to join the party.

jonathan · January 29, 2024, 8:26pm

Thanks for your writeup!

I did in fact apply for the Apple Vision Pro dev kit, as I had similar ideas about Hype + WebXR content. However Apple was silent never sent one (as is the case I think with 99% of all developers that applied).

The visionOS simulator does have flags that can be turned on for <model> tags and WebXR content, but the issues really here would be:

They are still off by default. I was hoping they'd at least be on by the end of the beta period. If they are not on or will never be on, then the opportunity for this is very small.
Most sites that had WebXR content didn't recognize Safari as having a usable XR device
Some that did still didn't work right, and still showed up in a small window instead of taking up the whole space (incorrect)

It isn't clear to me if these are simulator limitations or device limitations. We'll find out on Friday .

For what it is worth, I did experiment with WebXR long ago with my Quest and found that event that device could competently display content with reasonable performance.

Photics · January 29, 2024, 8:41pm

Wow, how incredibly short sighted.

jonathan · January 29, 2024, 8:49pm

Yeah, I'm not sure what happened on their part. I could have gone to a lab to test theoretically, but I was expecting like any other dev program (Intel, Apple TV, Arm) that devices would be easily available. Then next thing you know they are shipping them to everyone.

I'm not so miffed that I won't consider the platform, but given the amount of money I've sent Apple's way... I would have expected better. At least in so much to set expectations on who may or may not get a device.