ARkit in Webview

I’ve worked on some AR files using Apple’s Reality Composer and they work on the app, but only if I link to an external file and open Safari outside of the original WKWebview.

You can try it on your device using Safari:

https://alphapodapp.com/ar/owl.reality

And that’s acceptable, but ideally I would like to open the .reality file within the app. Not a lot of questions about AR and Hype out there, but maybe someone has an idea? I’ve read ARkit does not work with WKWebview but maybe that’s wrong?

3 Likes

Thats pretty cool.

Could not see a way of embedding it in Hype.

I think if you follow the export for app details here

https://developer.apple.com/documentation/realitykit/creating_3d_content_with_reality_composer

And export it as an .rcproject
Then you may again be able to get a postMessage from hype to play the vr via the iOS app.

Yes, that’s what I did, used postMessage to link to the USDZ file and it plays in the browser. It’s acceptable.

1 Like

The only thing is I was pointing out you may be able to keep it in app

I did a quick test.

The class stuff that the apple doc says will be auto created do not seem to be when dropping the .rcproject in to one of the examples I made earlier to show postMessage.

So I added a new class file with:

import UIKit
import RealityKit

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        // Load the "Real" scene from the "Mtest" Reality File
        let boxAnchor = try! Mtest.loadReal()
        
        
        // Add the box anchor to the scene
        arView.scene.anchors.append(boxAnchor)
    }
}

note the vr scene is named Real which is named via the scenes side bar in Reality Composer.
Where the line let boxAnchor = try! Mtest.loadReal() . in the code above uses that name.

loadReal()

Mtest is the file name

( you removed your link to your file so I could not use that which would probably make things clearer in the above code.)

I then added a new storyboard file and composed a RealityKit AR View and linked it up to the ViewController class’s @IBOutlet var arView: ARView!.

I also gave the scene a Storyboard ID of realv ( via the class inspector )


In the wkwebview controller.

I used this code. ( again just an example and not verified as the correct coding )

if message.name ==  "openStore" {
          print("\(message.body )")
          //-- Get the message passed to app
            let storyBoard: UIStoryboard = UIStoryboard(name: "realv", bundle: nil)
            let balanceViewController = storyBoard.instantiateViewController(withIdentifier: "realv") as! ViewController
          self.present(balanceViewController, animated: true, completion: nil)
          }

change and reference the actual postMessage name you use. But here I just used openStore.

lastly i added the property Privacy - Camera Usage Description to the info plist file in the Xcode project. ( this asks the user to give access to the camera)

This post gave me some of the clues of where to head.

What this gives me is a modal that keeps the VR in the App. ( can be dismissed by dragging it down )

You will need to research a bit on how to set this up properly and cleanly though.

2 Likes

Hey Mark this is so helpful, thanks again.

I was able to do most of what you said in your post, but I have a few questions:

  1. In this line

let boxAnchor = try ! owl.loadReal()

I get a "Use of unresolved identifier ‘owl’

Owl is the name of my file owl.reality (which I’ve re-uploaded to my server https://alphapodapp.com/ar/owl.reality ). In that RC file my scene is called Real, just like in your sample

2.I created a new storyboard and added an AR view, but not sure how to link it to the @IBOutlet class as per your instructions:

" linked it up to the ViewController class’s @IBOutlet var arView: ARView!."

  1. Not sure how to add the Privacy Camera Usage to my info plist. This is all I see (attached)

Thanks again!

Don't exporting the file in RC.

I assume you have already save the RC project. So use the actual saved project file.

the file will be the one named owl.rcproject


--

See image below..

1, click this button to open a new split view in Xcode

Have the new view show the new view controller file and the other view showing the new storyboard.

2
Type in the line shown

3
Then click into it's outlet selector, hold and drag to the view in the storyboard as shown.
You should get the blue drag line and the view name should light up as shown. When it does release the mouse and you should have connected the view to the outlet.


If you hover your mouse over the top line in the property list that reads
information property list

You will see a + symbol appear next to it.
Click the + symbol.

A new property field will appear with a property named App Category highlighted.
Replace App Category with Privacy - Camera Usage Description

The Type should already be a string but if not change it to that.
You don't have to give a string but probably should give a simple one saying why your app needs the camera.


Also Note.

The swift code will want a object name that starts with a Capital.

So
owl.rcproject
The code will be

Owl.loadReal()

One thing I noticed is that you have an ARView Scene, I have a View Controller Scene.

For the life of me, I could not create a Storyboard and then add an AR View. What I ended up doing is I duplicated the LaunchScreen Storyboard and then added an AR view to that existing ViewController.

SO, I just created a new Storyboard, named it “realv”, and there’s no Scenes. I open the Object Library but when I tried to drag an AR view to the Storyboard nothing happened.

This is my new storyboard. Empty!

The Apple dev site says:

add additional scenes as necessary by dragging view controllers from the Object library to the canvas

Can’t make it work. Something I’m doing wrong clearly, but I gotta say, Xcode is not user friendly

From the library search for viewcontroller

Drag the view controller to the left hand scene panel.

Select it and set it's class to Viewcontroller (the new class file name you should have )
and its storyboard ID to realv.

Now drag a RealityKit AR View into the viewcontroller.
You may first want to remove the default view first.

Link the AR View to the outlet.


Note sometimes you may need to have the storyboard in view and do a build so Xcode picks up changes before you can connect things.

Also note I meant a RealityKit AR View not a AR View

Yes, but yours says ARView Scene, not View Controller Scene. That doesn’t matter? a VC is what I had, but I deleted because I thought I had to have an ARView instead.

No. Its just a label.

In the inspector (identity inspector) where you set the class.
The is a label field…
If you put ARview in that the display name will change. Its just a label for you to organise things.

Probably me confusing you a bit with my wording… and you just learning on the fly before your wings have fledged.

Yes, the file Owl is the rcproject, but how do I get rid of that error "Use of unresolved identifier 'Owl"

Use a capital O, Xcode normally helps with trying to suggest what you are trying to type..

I was able to link the IBOutlet line to the AR View but I get same error

click into the left split view then in the navigation panel select your viewcontroller class file and..
Wait.. where is your class file..??

I was able to link by openning the assistant

But the error remains

where did you actually put the class. I assume if you did not create a new one you just added the class to the WKWebViewController file

Correct, Added to the WKWebView

I also do not see the owl.rcproject file