Apple Vision Pro

jeanlain

Ars Tribunus Angusticlavius
6,770

I thought this deserved its own thread.

The Vision Pro isn't presented as a VR device, but mostly as an AR device.

I have mixed feelings about it, mostly because they didn't demo any VR experience... Is there any information about the field of view? It may not be enough for VR.
 

xoa

Ars Legatus Legionis
12,061
Subscriptor++
So, it’s big and bulky and it’s not at all clear where they can cut down that size and weight as they iterate on the design. As cool as it is, the real revolution they need is in the optics.
If you haven't already check out the RETISSA II, which is retinal direct imaging from a few years back. That eliminates much of the optics, though it was much lower resolution. I don't think Apple's wearable display plans are tied to a specific tech, what they're working on at this point is starting the cycle of UI/UX, developer feedback, sensor fusion and use stuff, interactions with their ecosystem etc. The pixel stream that gets fed into whatever display tech out of that whole stack is going to be agnostic to how that translates into photons-to-eyeball. It's an Apple v1.

Edit: Attached picture. And this is a number of years old now as well. WDs are clearly going to be a decade+ project, but I think the raw potential is obvious, and Apple can't afford to miss the boat. Finding the right timing to get going is always challenging, because if you wait until it's fully done you get wiped, that's The Innovator's Dilemma in a nutshell. Almost by definition if you're doing it right you're doing it "too early". But this is way, way more important, core to Apple, and sensible than all that stupid car nonsense which I hope they've dumped.
retissa-display2-body.jpg
 
  • Like
Reactions: dal20402

cateye

Ars Legatus Legionis
11,760
Moderator
Reflecting over the last couple hours, my feeling is the reveal had moments of real brilliance. The idea of performing computing tasks spatially is very interesting to me. I love the idea of essentially an infinite canvas for applications that could take advantage of that, like photo editing, certain kids of design and 3D creation, and so on. "Placing" windows in various locations and allowing them to have varying degrees of interaction with their environment feels very science fiction and I'd like to explore that to its fullest extent.

There's lots of stuff Apple wants me to be excited about, particularly all the disquieting scenes of people looking at family photos or video conferencing with your creepy over-real avatar, but none of that feels interesting or even necessary. That all feels very aligned with the dystopian vision of replacement reality pitched by Meta, et al. which I want absolutely no part of. Things like watching movies leaves out the social nature of those activities—what, everyone at movie night has to be wearing $3500 space goggles now?—and so that falls into the same bucket for me as "interesting only as a marketing hook, not as something I'd actually want to do." Nor do I like the idea of supporting Apple's (to me) morally bankrupt media and app store constructs. I'm going through some efforts to extract myself as much as possible from ever using the App Stores if I can get away with it, not until legislation or legal action deeply curtails Apple's control.

As an alternate display device, it has me curious. But it feels like Macintosh 1.0 in that regard. There's years and years of developers getting their arms around a different paradigm and exposing new opportunities before it becomes a natural choice to augment or supersede existing ideas. As a result, I'm left feeling like I was shown a super interesting technology demo, one that might become relevant to me in another decade.
 
I'm quite excited for the me in about 4-5 years who will buy a lighter, more powerful version of this with better battery life. I'll be rooting for it to succeed until that point so that it's a category worthwhile for Apple to iterate on further.

I can't imagine wearing one outside, or for more than a few hours at a time, at least in that form factor. But for focused work sessions, media consumption, and possibly some games, I see a lot of value for myself.
 
  • Like
Reactions: KevinM1

Horatio

Ars Legatus Legionis
24,069
Moderator
So how much are the Zeiss add on lenses gonna cost me?
I expect them to be relatively inexpensive - with all the talk about myopia during the earlier part of the talk, it would be incredibly short-sighed of them to force people that need glasses to spend $4k where others only need to spend $3.5k. $149, $199 max.
 
The battery life is going to be the deal breaker here, "up to 2 hours" using a power brick that has to go in your pocket is a problem, the only way to use it for a reasonable amount of time is to be sitting down and connected to a wall socket which defeats much of the point. I would guess that future versions are going to focus heavily on extending that battery life, but I wonder how much improvement they can really make in that area, I doubt there is much low hanging fruit in terms of power budget.

I still think the future of AR involves transparent displays that have the ability to sleep and conserve energy when there is no need to display any AR content, doing a continual video pass through to a user's eyes is going to be too much of an energy drain for this type of device to ever become closer to an unobtrusive pair of glasses.
 

jeanlain

Ars Tribunus Angusticlavius
6,770
The battery life is going to be the deal breaker here, "up to 2 hours" using a power brick that has to go in your pocket is a problem, the only way to use it for a reasonable amount of time is to be sitting down and connected to a wall socket which defeats much of the point. I would guess that future versions are going to focus heavily on extending that battery life, but I wonder how much improvement they can really make in that area, I doubt there is much low hanging fruit in terms of power budget.
A solution would be to use two batteries and swap them. In general, using a headset for more than 2 hours straight isn't a good idea.
 
I was instantly unimpressed with the app icon honeycomb. The most exciting thing about AR & general-purpose computing is potential for new UI paradigms, and I really thought we'd see something like that today. Looking at the Messages icon to open it in a big window in my living room is not compelling. The iOS approach feels wrong for a platform that relies on such a high level of hardware sophistication. At a minimum it would be very cool to work on a giant wide monitor using only the headset and a bluetooth keyboard; but it appears to be limited to one "4K monitor" and tethered to a Mac. Ultimately I am more interested in the new AR features of Airpods.
 

jeanlain

Ars Tribunus Angusticlavius
6,770
If you have no or limited VR experience then you probably don’t realise how much better this is.
How much better is it really? It would have been more convincing if Apple showed true VR experiences, rather than just AR. And everything they presented looked like renders rather than the real thing (though a couple of animations were jerky, for instance, the tab exposé in safari).

I couldn't find any info about the screen refresh rate. Hopefully, it's 120 Hz or more.
And can the headset be used as a screen only? The M2 may be fast, but it's still an ultrabook SoC. That's not going to power VR games at 120 fps...
 
Last edited:

Horatio

Ars Legatus Legionis
24,069
Moderator
Hopefully, it's 120 Hz or more.
They said 12ms in the presentation, that's ~90Hz (same as Quest 2 I think)
That's not going to power VR games at 120 fps...
You don't need to render 120fps for games - you use frame interpolation to render ~half the frames (and even less if you use something like DLSS), plus they have foveated rendering, so you can spend most of your cycles rendering what the user is looking at anyways.
 
D

Deleted member 397475

Guest
There in a practical sense no difference between VR & AR. AR, is in the case of Apple, VR but transparant. Or opaque if you wish (‘regular VR’) as seen in the demo. The VR part is not the difficult part, the interface, sensors, usability & room scale is what is waht sets this apart.

I have never seen Apple lie about the experential parts of their products, i have no reason to believe to would start now.

You’re just going to trust me when i say: this is in all aspects better than setting up VR with mounted sensors in a room, wearing a plastic bulky headset and constantly having to take it off to talk to people. You think you can take other headsets with you to the fridge?
 
D

Deleted member 397475

Guest
How much better is is really? It would have been more convincing if Apple showed true VR experiences, rather than just AR. And everything they presented looked like renders rather than the real thing (though a couple of animations were jerky, for instance, the tab exposé in safari).

I couldn't find any info about the screen refresh rate. Hopefully, it's 120 Hz or more.
And can the headset be used as a screen only? The M2 may be fast, but it's still an ultrabook SoC. That's not going to power VR games at 120 fps...
That’s your take..?

I dont think you understand this at all. VR headsets don’t power games, they are just headmounted displays.

You can use this tethered to your Mac. And on top of that, it has on-board compute.
 

jeanlain

Ars Tribunus Angusticlavius
6,770
I thought 12ms was the processing time between the SLAM cameras to the screen, which would limit the frame rate.
For me it's the same thing. But the time it takes for an image to be processed may not limit the refresh rate if several images can be processed simultaneously (but at different stages of the "pipeline").
 
Last edited:

jeanlain

Ars Tribunus Angusticlavius
6,770
There in a practical sense no difference between VR & AR. AR, is in the case of Apple, VR but transparant.
The environnement is filmed, not render as a 3D scene. That requires less power.

I dont think you understand this at all. VR headsets don’t power games, they are just headmounted displays.
That's what I'm saying. The M2 cannot handle that kind of power. The headset has to be used as a screen for a VR game.

You can use this tethered to your Mac. And on top of that, it has on-board compute.
I know it has on board compute, I've watched the keynote.
What I don't know is how it can be tethered to a Mac to play a VR game rendered by the Mac GPU. I'm not sure how anyone could know outside Apple, since there is no macOS VR game and they did not demo one.
Showing a virtual screen like they did for FCPX is not the same. The experience is not ruined by a small lag or lower frame rate, as opposed to a VR game.
It's a wireless device right? Is there a wireless solution that allows streaming a 90 fps video to dual 4k displays with minimal lag?
 

andgarden

Ars Tribunus Angusticlavius
7,581
Subscriptor++
I couldn't think of a way to make VR goggles anything other than heavy and extremely uncool looking, and apparently neither could Apple. I was negative on this concept going in, and having read the coverage (but not watched the keynote) i remain so. I can't remember the last time I wanted so little to do with an Apple product. (I generally love new hardware and tech gadgets.) I mean, look back at Steve's iPhone keynote in '07, the whole thing was a demo of how cool the product was going to be in your everyday life! I'm feeling nothing like that about the Apple Goggles.
 

jeanlain

Ars Tribunus Angusticlavius
6,770
i can’t even be bothered.
Read it all again. Really this is not hard.
"You can use this tethered to your Mac" does not say what this enables. Watching a movie that's on your Mac? Sure. Using FCPX on a virtual screen? Certainly. Playing a VR game at 90 fps, maybe, but we've not seen it yet.
Apple hasn't even detailed its I/O, AFAIK. Does it have wired video input?
I'm not aware of the latest development in VR, but is there a single VR headset that allows wireless VR gaming while tethered to a PC?

Yes, the Vision Pro does amazing things, but it may not be the best at everything VR.
 
Last edited:

xoa

Ars Legatus Legionis
12,061
Subscriptor++
Since I don't see it mentioned in this thread yet, another unique characteristic of wearable displays is total privacy of content, even in busy environments. No shoulder surfing. Can watch/read things that you don't want anyone else to see without having to worry about being alone or having a screen in a corner facing a wall or the like. That potentially further combines well with the iris ID authentication, which will be very hard to spoof. In some uses that could be a huge deal, and not just stuff like on planes, even for people at home with families. Have young children? No worries about viewing a PG-13 or R video or for that matter news stories or whatever and having little eyes catch you by surprise. Need to review some private information? Again, extremely secure way to do it even in an apartment without any special private room. And then of course all the obvious stuff, games and so on on planes/trains/busses.

Longer term ergnomics again deserve special mention IMO. At home and work we've all gotten fairly pricey Ergotron arms to improve screen positioning and it's made a huge difference. WDs will obsolete that as well, and craning the neck down at a phone/tablet or the like. Over the course of decades add up the cost of even a few massage of acupuncture or PT sessions.
 
D

Deleted member 397475

Guest
I really don’t feel like having to fire up UE4 to show you how this works.

It is inconsequential what you are doing with those glasses wether it’s FCP, a movie or a ‘real VR experience’. It is, in the end, a flat projection of a 3D calculated scene.
When tethered to a device (both wired and wireless, i know, confusing) it is streaming a video signal. It is adjusting the projection based on spatial input data.

If this uses the standard relevant API, it will very likely function exactly the same as VR glasses in Unity & UE. I dont see how it wouldn’t as, agaib, AR & VR are in this case the same. It’s not an instagram filter even though you could argue that is real 3d …
 
Last edited by a moderator:

xoa

Ars Legatus Legionis
12,061
Subscriptor++
That's what I'm saying. The M2 cannot handle that kind of power.
First: of course the M2 can power doing "VR games", for certain values of VR games. An A15 or A14 or A13 or going back quite a ways could too, it's just a matter of figuring out the appropriate level of detail/art style. Sure, if you are after top class photorealistic ray tracing performance like cutting edge PC games on a top tier GPU (or multi-GPU) system, M2 can't do that. But that's like saying the iPhone 4 couldn't do games like an iPhone 14, there were still lots of fun games people really enjoyed. Developers who are interested will match available power same as it's always been.
The headset has to be used as a screen for a VR game.
Second, could you clarify what you mean here? The headset certainly doesn't appear that it has to be used that way, it can also be used as the screen for a regular standard game, not VR. Just as any regular display could. VR would be a bonus superset of games, not the only ones. If nothing else M2 would easily be able to handle every single iOS game in existence, displayed on an iPad-sized virtual screen. Apple has pushed for controller support in general for a little while now too, IIRC everything in the Apple Arcade is required to have it, so zero dev work should be needed here in terms of new UX. Controller talks to AVP, that's that.
 
  • Like
Reactions: Tagbert

ZnU

Ars Legatus Legionis
11,694
I couldn't think of a way to make VR goggles anything other than heavy and extremely uncool looking, and apparently neither could Apple.

What the headset looks like really isn't so important until they're targeting the all-day phone-replacement use case, which they're not doing with this first-generation device.

If the screen resolution is really sufficient that you don't feel like you're looking at pixels and the eye and hand tracking are non-janky, I expect using this device will be a pretty amazing experience, even for uses as commonplace as the presentation sharing demo. Interacting with things on little 2D screens might rather rapidly start to feel claustrophobic.

I'm also reminded of a comment I made in BF about the iPad in 2011:

But in any event, DR. M has an interesting point, and it's actually all tied up with what Apple means by "magical". Assuming the former actually worked, what would the difference between psychic remove viewing and television be, that made the former magical and the latter not? Essentially, the difference would be that the former lacked an observable physical implementation mechanism. This is precisely what Apple means in calling the iPad "magical". It performs computing tasks, but there's no mouse or keyboard, no humming box, no wires. The whole experience of the device is the interface of the software -- it's almost like software that exists without hardware.

But really, iPad wasn't quite there. It got rid of the humming box, etc. but there was still this discrete thing you were interacting with. Vision Pro is a big step closer. As Apple said during the keynote, it's the first Apple device you look through rather than at. Ideally, the device itself no longer exists in your awareness. The lack of controllers is significant here. People laughed at Jobs for using the word "magic" in reference to computing, but here we are 12 years later with a device that you interact with, more than anything else, as a wizard casting spells. You speak, you gesture, and things happen. This seems like it might be rather rapidly addictive, if it works well enough.
 

ghub005

Ars Tribunus Angusticlavius
8,619
I won’t be buying this. But I hope that enough other people will buy it to make it a viable product. Like the first generations of iPad I expect it will take a few years before it matures to the point of genuine utility.

My only concern is that it will become a device for media consumption. While this is a perfectly legitimate and viable product strategy, it would undermine the creative potential that this platform offers.

Having said that, I will be curious to see what impact this has on companies that make hardware for home entertainment. There will be less demand for big-screen TVs or fancy Sonos soundbars if consumers can access the same level of experience through this device. It might replace swathes of other devices in the same way that the smartphone replaced products like digital cameras, portable music players, and GPS devices etc.
 

jeanlain

Ars Tribunus Angusticlavius
6,770
First: of course the M2 can power doing "VR games", for certain values of VR games.
Sure, but that's the kind of VR games you'd rather not be limited to after spending $3500 on an AR/VR device.

Second, could you clarify what you mean here?
I'm referring to games that require beefier hardware than a tablet SoC to be enjoyable.
Being able to connect the Vision Pro to a PC/Mac that has a minimal catalogue of engaging VR titles (like, I don't know, No Man's Sky) would be nice.
Maybe the Visio Pro can do that, but Apple showed no evidence of it. Can it connect to a PC? I suppose not. I suppose it doesn't support SteamVR either. Will Apple allow developing VR games for the Mac? Hopefully they will.
 

Jonathon

Ars Legatus Legionis
16,541
Subscriptor
"You can use this tethered to your Mac" does not say what this enables. Watching a movie that's on your Mac? Sure. Using FCPX on a virtual screen? Certainly. Playing a VR game at 90 fps, maybe, but we've not seen it yet.
Apple hasn't even detailed its I/O, AFAIK. Does it have wired video input?
I'm not aware of the latest development in VR, but is there a single VR headset that allows wireless VR gaming while tethered to a PC?

Yes, the Vision Pro does amazing things, but it may not be the best at everything VR.
You're not going to get the same kind of frame rates or latency that you would via a wired connection, but the Oculus Meta Quest and Quest 2 can stream VR games wirelessly from a PC via the (third-party) Virtual Desktop app and has been able to for a few years now.
 

jeanlain

Ars Tribunus Angusticlavius
6,770
I really don’t feel like having to fire up UE4 to show you how this works.
You don't need to. I know that a display just receives a video signal, and it doesn't "care" how it's generated.

It is inconsequential what you are doing with those glasses wether it’s FCP, a movie or a ‘real VR experience’. It is, in the end, a flat projection of a 3D calculated scene.
The Vision Pro wireless I/O solution may have limited throughput. A movie may be sent in compressed form and decoded on device. The virtual Mac 4k display suggests that Apple has developed a high throughput solution, but a VR game may require a higher frame rate (more than 60 fps), very low latency and two images (one per eye). Can this be streamed to the headset with low latency? Possibly, but I would have liked if Apple had showed it in a demo.

EDIT: if lesser devices can do it (see post above), then I suppose that the Vision Pro will.
 
Last edited:
  • Like
Reactions: hestermofet

jeanlain

Ars Tribunus Angusticlavius
6,770
You're not going to get the same kind of frame rates or latency that you would via a wired connection, but the Oculus Meta Quest and Quest 2 can stream VR games wirelessly from a PC via the (third-party) Virtual Desktop app and has been able to for a few years now.
Interesting. I must have used the wrong keywords when googling. I'm curious to know if the gaming experience is enjoyable.
 
Last edited: