How Could Hype Plug Into Apple Intelligence?

1

It's funny! I was getting text message from friends, how the WWDC 2024 keynote was underwhelming. But, when you think about it… and I did …this could be a good direction for AI. I've seen lots of tech demos for AI, but Apple's felt less dystopian. It felt like they struck the right balance for this new technology.

So, this seems like an opportunity for Tumult. Perhaps some of those AI features should be available in Hype…

Anyway, here's the video if you missed it… https://www.youtube.com/watch?v=RXeOiIDNNek

1 Like
2

I thought they have done so much.
Unrelated to programming, I have been giving feed back to apple every now and then for them to extend their privacy of what photos apps have access to, to your contacts. So access to your contacts are enclaved per app. They finally did it.. Well chuffed.

But back to the subject I think to add the API for Apple Intelligence to you app is just a few lines of code and in some cases where textfields are used you just get it.

Within the Hype IDE:
The first things I think of:
Hype could use the image creation within the Apple Intelligence to allow users to create new images within the Hype IDE using natural language.

Hype could possibly include a way to tap into the new auto Localisations within the Apple Intelligence API. channeling that output to a new hype localised json file or even a hype function.

On another note. The Apple Intelligence will only be included in the iPhone15x onwards and more importantly for Hype M1 Macs onwards.

The iPhone bit is annoying to me personally is it means most of the billion people with iPhones will need to upgrade. I luckily I now havde an M1 Mac

I think Apple chose to do this because of speed and data compression of running the AI onboard the devices. But maybe to get use all to buy new devices?.

2 Likes
3

I believed that Hype could support the continuity feature for previewing on the phone while displaying on the main computer.

1 Like
4

The Image Playgrounds is the obvious item to add since apple created an API for it. However, from what I can tell, the API is super simple. I'm not sure there's actually an any advantages to using it vs. users opening the Image Playground app and dragging and dropping into Hype. (And the disadvantage is that it seems to want a sheet, which would just cover the scene's content instead of being able to use it side-by-side with Hype...). I feel like the API makes more sense on iOS apps where you're not really expected to leave an app.

However if there's other ways to get more into the internals of apple's image generation, I think that'd be pretty incredible. I just didn't see it in the video (and can't find actual documentation, perhaps because it isn't out yet).

1 Like
5

@Jonathan , What do you think about the localisation idea.

Just thinking further on that and although it likely would be possible, it probably makes more sense to go the same route as you mention for the images for pretty much the same reasons.

But anther idea I get after thinking about what I saw in regard to Xcode .
Was the code completions. Now not saying you would be able to use Xcode's code completion API.

But it would be great if Hype's own function views had Linting and a code completion AI that also can code suggest.

6

That's a good idea… maybe… isn't there the ChatGPT store? So, if Apple lets you use a ChatGPT subscription, is this a way for Tumult to make money with code completion for Hype? I'm not sure if there's a huge demand for that. Most people use Hype because they don't want to code.

More reason for an iOS / iPadOS version of Hype. :wink:

…but what is this…

It doesn't just let you add a picture to an image box? Hype is basically an unofficial iWork app. How does it work in Pages and Keynote, but not Hype? :thinking:

That was surprising. I thought there'd be a developer workshop, but I didn't find anything more than some marketing text.

That remote iPhone feature is going to be great for testing with Hype… but I don't know if Hype can plug into that app. Maybe this is something that could be scripted. :thinking:

I think Hype could use a feature like that in general, similar to the Multilingual template… Free Template Tuesday #11 – Tumult Hype “Multilingual” – Photics.com …where I'm using a multidimensional array, but that's similar to JSON.

Basically — Dynamic text.

There are two problem, one is multilingual, but the other is layouts. If I add another layout, I have to recopy the text — right? So, if Hype could manage that text… and then add a multilingual mode… then it makes sense for something like ChatGPT to auto populate the multiple languages. Would you trust that though? :thinking:

7

My main idea is how they will now be using AI in Xcode in part using natural language input to for it to write and suggest code using models leant from their own engineers. Such a great idea.

Not sure I understand what you mean???

There is a lot of demand from non coders to be able to use code once they realise that it can enhance their productivity and end result. But are challenged with were and how to start.
The forum reflects this. It would be interesting to see what the percentage of coding request are.

At the moment the demand falls on a few of us on the forum to help with that.

Now I am not saying we stop this but it does make sense in the new world order of AI to incorporate it into Hype. Would I trust ChatGPT.. Not from past experience with it (3.5) but that does not mean which ever AI was used could not be kept to a strict learning modal to call upon.

8

I don't use ChatGPT, but isn't there a marketplace?

…or couldn't Hype just create their own AI assistant and include it as a subscription inside Hype?

I'm not a fan of subscriptions, and I doubt the demand is there, but is that a possible way for Tumult to make more money with Hype?

It doesn't seem to be that many people asking for help lately. But, if people could talk to Hype… create an animated banner… or something like that… wow!

Personally, I think a template marketplace might be a better idea in the immediate future for Hype. AI tech is too new and conversational programming doesn't seem to match the level of what we post on this forum. :smile:

9

In the meantime:

1 Like
10

Maybe they should just hire you to add that feature to Hype. :thinking:

There's still opportunity with Hype. They missed a big moment with the death of Flash. A converter like Ruffle or Swiffy would have been awesome at that moment in time. For some reason, they didn't add a Marketplace. For some reason, they didn't make an iPad version of Hype. For some reason, we still don't have App exporting or even PWA support.

Now there's AI.

It's been almost five years since the launch of 4.0. I wasn't sure what to add to Hype a few years ago. But now, there are a lot of nice features Tumult could add to Hype.

1 Like
11

I mean that the API is apparently a sheet UI. This would cover the Hype content you're working on, and make it hard to iterate. There may be a way to stick it in a different window, but they why not just use the app that does it anyways?

From what I could tell, there was only a bit of API that helps you seed the prompts and not too much beyond that where there's a use case for integration. But of course we'll see later when they release the missing docs.

Pages shows it this way:

Screenshot 2024-06-14 at 1.01.11 PM
Screenshot 2024-06-14 at 1.01.11 PM1920×1392 101 KB

Definitely not our bag :slight_smile:

Can we instead talk about how Apple has not made the iPad a good platform for pro apps like Hype? :stuck_out_tongue_winking_eye:.

good idea!

1 Like
12

That would have been a heavy lift, but it could have been awesome too!

Ha ha, I was watching this Max Tech video earlier today… https://www.youtube.com/watch?v=ame_9x0F6M8 …and it was so sad. The iPad M4 has a massive… MASSIVE …amount of power. Yet, WWDC was a big disappointment for the iPad.

I thought that Math Notes was amazing, but apparently it's not really exclusive to the iPad… not counting the Apple Pencil.

That's why Apple should be throwing money at Tumult. They should be motivating you to add a pro app to the iPad. :smile:

Do you mean the popup?

I don't see that as a big problem if the image ends up in the right box. That's more convenient than cutting and pasting from a separate app. Also, if it's right there, it's more accessible. Maybe some users aren't aware of the process, maybe others have brain-freeze and forgot about the option. The point is that it could be another bullet point on a long list of features for Hype 5.

1 Like
13

You all seem truly dedicated to this feature in this thread. If this is the first Gen-AI feature to hit Hype, it would be beneficial to consider adding a “Remove Background” feature for characters and items. While discussing generative AI, I would personally prefer a coding assistant or an overall Copilot to guide you through the interface. However, to be honest, there are more pressing features and a new release I would like to see, but could this be a quick win? I'm not sure.

2 Likes
14

Not really, I've been pondering the possibility of moving to a cabin in the woods, playing video games on carts, switching to a flip phone… or maybe Linux …and then growing lots of vegetables. Heh, but then I realize that lots of food is heavy and it would be nice to have robots carry that. :smile:

It's more about seeing Tumult… DO SOMETHING! :smirk:

There's certainly the potential for Hype to catch up. I think a great comparison is Pixelmator. That app has templates, it imports Photoshop files, it already has some AI features (such as removing the background), and it's updated constantly. The app has to be good, because they have to fend off the competition like Affinity Photo and open source image editors.

It sounds like it's a little bit of programming to add the “Image Playground API”. It's not perfect, but it's moving forward with Apple. That's what could get new energy around here, as that seems like something Apple would feature.

That alone is not enough for a 5.0 release though. :thinking:

(Although, Chrome is around 127 and Firefox is around 128, so maybe lots of little updates is more important. :smile:)

But, if improved code editing was added to Whisk and Hype simultaneously… wow! I don't like that I use a code editor from Microsoft. But unfortunately, Whisk doesn't clean up code like Visual Studio Code, Whisk doesn't have code completion or Emmet Abbreviation, tab doesn't add a predetermined (five) amount of spaces, it doesn't spell check code, no git, no coloring of brackets, and some other stuff. Probably the biggest feature is cross platform support. Visual Studio Code works on Linux.

So, if Hype is to go the code assistant route, it seems like they'd need to strengthen the editor first.

And if I seem excited, it because I'm more optimistic about Apple's future. WWDC didn't wow… beside Math Notes …but what I did like was the humanization of AI. It wasn't showing off how your robot looks like something from the Terminator movie, or how AI is going to kill all the jobs. It was like… oh look, cute emoji.

Oh, but they “Sherlocked” lots of companies… so I suppose it depends on your perspective. Automation is great until it comes for your job.

1 Like