The State of Augmented Reality(sayduck.com)
sayduck.com
The State of Augmented Reality
https://sayduck.com/2019/09/27/state-of-ar/
11 comments
I'd argue AR via a phone in your hand is entirely useless and a gimmick. The phone has a much smaller viewing area than even hololens and walking around staring through it is not just annoying it's dangerous.
That is not to say that on your face AR is any good yet, only that phone AR is mostly pointless IMO.
I did get to use a Magic Leap at GDC this year to shoot aliens coming out of portals in a physical living room stage. The graphics were PS2 level but the attachment to the real world in that contrived setup was rock solid as was the virtual gun projected in my hand. As I ran about the living room the portals with aliens coming through mapped to the walls of the living room worked well. It at least shows a step toward real AR.
That is not to say that on your face AR is any good yet, only that phone AR is mostly pointless IMO.
I did get to use a Magic Leap at GDC this year to shoot aliens coming out of portals in a physical living room stage. The graphics were PS2 level but the attachment to the real world in that contrived setup was rock solid as was the virtual gun projected in my hand. As I ran about the living room the portals with aliens coming through mapped to the walls of the living room worked well. It at least shows a step toward real AR.
That is why I am excited about TiltFive - AR TableTop. Restricting the environment to a table (31" by 31") makes the experience more aligned with the existing hardware. Still teathered to a laptop/desktop but you get 110° fov with 720p each eye. Once developers and users get a sense for mixed reality better/more novel applications will get more development. AR productivity software feels revolutionary.
Yeah. Maybe if you didn't need to move around as much and you were looking at a fixed point in space I could see how it could feel more immersive. It's interesting to note that AR feels more natural the less you move around.
While there has certainly been progress with AR, the VR experience has seen vastly more significant improvements since 2012. Automated photogrametry also keeps getting better and better, making it faster and easier to fully reconstruct real world locations in VR. I think this has already become more compelling than AR for the use case presented in the article (seeing how stuff looks in your house).
AR Quick Look is not Web AR. You cannot write web code (JS, HTML, CSS) to interact with your AR elements within your scene. It's very easy to spot fake Web AR: do they have a link on their website that you can click on and it works on all phone browsers? No? It's some proprietary stuff that is wasting your developers' time.
The current state of AR is * Hardware is not there yet. Phone-based AR is the current hardware until then. * Fantastic content is still in the work. Content creators and developers are aware that app download is the biggest slog on distribution. Nobody is going to download (and keep) your app to experience AR for 5 minutes. Most AR experiences last less than 30 seconds. * True Web AR, where you can literally go to a link on your browser and experience interactive AR, is the best medium for content delivery. Write HTML, CSS, JS and get AR content. This works on both short-form (< 5 minutes) experiences and long-form (>5 minutes) experiences. * Foundational layers of AR are either missing or vendor lock-in: persistence object (e.g. come back to the same space tomorrow and see the same thing), multiplayer (e.g. see the same content from your own angle with your friend), distribution (e.g. the same content getting specialized for different form factors), creation tools (e.g. environment scanning, object scanning, ...).
Many (see other comments) people are correct when they say that phone AR is not what AR should be. That even the Hololens or the Magic Leap are not it. This will change with time but it's a catch-22. You need some hardware to get more content to get more hardware to get more content.
The current state of AR is * Hardware is not there yet. Phone-based AR is the current hardware until then. * Fantastic content is still in the work. Content creators and developers are aware that app download is the biggest slog on distribution. Nobody is going to download (and keep) your app to experience AR for 5 minutes. Most AR experiences last less than 30 seconds. * True Web AR, where you can literally go to a link on your browser and experience interactive AR, is the best medium for content delivery. Write HTML, CSS, JS and get AR content. This works on both short-form (< 5 minutes) experiences and long-form (>5 minutes) experiences. * Foundational layers of AR are either missing or vendor lock-in: persistence object (e.g. come back to the same space tomorrow and see the same thing), multiplayer (e.g. see the same content from your own angle with your friend), distribution (e.g. the same content getting specialized for different form factors), creation tools (e.g. environment scanning, object scanning, ...).
Many (see other comments) people are correct when they say that phone AR is not what AR should be. That even the Hololens or the Magic Leap are not it. This will change with time but it's a catch-22. You need some hardware to get more content to get more hardware to get more content.
At least for the browser, current AR efforts all fail short of having low enough latency to allow the projections to "stick" to the marker area when the camera is moved. Another thing is the poor support for bigger, multi-marker areas.
I wanted to create a local multiplayer game with smartphones rendering stuff on the ground but the current technology just isn't there.
All those demo gifs omit the one thing they all require.
Hands.
Stumbling around trying to interact with a magic window your holding is a crappy experience. AR is “second-nature”, now?
Very few bother with the novelty a second time.
Hands.
Stumbling around trying to interact with a magic window your holding is a crappy experience. AR is “second-nature”, now?
Very few bother with the novelty a second time.
>Stumbling around trying to interact with a magic window your holding is a crappy experience. AR is “second-nature”, now?
That is the primary way that most people interact with a computer and has been for years, so "second nature" seems pretty accurate. The people "stumbling around" are the ones with the VR goggles strapped to their heads, getting headaches and vertigo and running into furniture.
>Very few bother with the novelty a second time.
I feel like a few Ingress and Pokemon Go players might disagree with you.
That is the primary way that most people interact with a computer and has been for years, so "second nature" seems pretty accurate. The people "stumbling around" are the ones with the VR goggles strapped to their heads, getting headaches and vertigo and running into furniture.
>Very few bother with the novelty a second time.
I feel like a few Ingress and Pokemon Go players might disagree with you.
Is there a sense in which Pokemon and Ingress and the Harry Potter game are "AR"?
Sure.
But they aren't the same thing that hololens or magic leap are trying to do, and conflating them with that kind of AR is silly.
Sure.
But they aren't the same thing that hololens or magic leap are trying to do, and conflating them with that kind of AR is silly.
Read the article expecting to find a review of different apps and ecosystems, maybe some big picture thinking.
Instead I read a description of Apple products and a sales pitch for OP’s company.
Title is perhaps a tad inaccurate.
Instead I read a description of Apple products and a sales pitch for OP’s company.
Title is perhaps a tad inaccurate.
Outside of Pokémon and Ingress - interacting with AR through a tiny phone screen still seems gimmicky and only mildly helpful.
Kinda feels like AR needs smart glasses to be adopted culturally before it has its moment. Thoughts?
Kinda feels like AR needs smart glasses to be adopted culturally before it has its moment. Thoughts?
Yes - to me the real revolution in AR will happen when we can walk around the real world and have ambient intelligence and visualization without having to drag a bunch of crap around with our fingers. Nobody wants to do that. They just want to be more intelligent - automatically.
The shopping experience is one of those, but to me, not anything like a killer app.
The shopping experience is one of those, but to me, not anything like a killer app.
I don't think it's a hardware issue as much as a content one.
What no one seems to be working on is mass market content creation. YouTube and Twitch are built in user content. How can we get user generated AR content, to drives sales, to push hardware, to build a market for professional content.
https://www.minecraft.net/en-us/earth
The sandbox of minecraft I think offers some more exciting and unique possibilities for AR.
The sandbox of minecraft I think offers some more exciting and unique possibilities for AR.
What happened to magic leap?
It's ok. I have both a HoloLens 1 and a Magic Leap One [0]. The Magic Leap is kind of like HoloLens 1.5.
Unlike a sibling comment, I don't think it's a "borderline scam", but I do think the company is struggling to live up to their own hype. When it was released, it had a lot of problems. The latest OS update is much better. I first tried it around OS v0.17 and it was pretty shaky and crashed a lot. By v0.20, it was running much better, but that was like 6 months later. I think they felt pressured to release earlier than they should have.
I've been pretty busy at work with a mobile VR project, so haven't touched the Magic Leap in a while. Current OS version is 0.22. Curious to see how it's doing.
[0] I keep a full list of my hardware here: https://seanmcbeth.com/vr.html Ask me anything about anything on the list. The notes on the page are a little outdated, as I have now used all of the hardware I have (and have it all readily accessible on a shelf at work). Maybe I'll update the notes tomorrow.
Unlike a sibling comment, I don't think it's a "borderline scam", but I do think the company is struggling to live up to their own hype. When it was released, it had a lot of problems. The latest OS update is much better. I first tried it around OS v0.17 and it was pretty shaky and crashed a lot. By v0.20, it was running much better, but that was like 6 months later. I think they felt pressured to release earlier than they should have.
I've been pretty busy at work with a mobile VR project, so haven't touched the Magic Leap in a while. Current OS version is 0.22. Curious to see how it's doing.
[0] I keep a full list of my hardware here: https://seanmcbeth.com/vr.html Ask me anything about anything on the list. The notes on the page are a little outdated, as I have now used all of the hardware I have (and have it all readily accessible on a shelf at work). Maybe I'll update the notes tomorrow.
> I've been pretty busy at work with a mobile VR project, so haven't touched the Magic Leap in a while. ... Curious to see how it's doing.
Magic Leap has always raised a swath of significant red flags, but for me, one of those red flags is that the company is based in Florida. As Roger Stone famously said, "Florida is a sunny place for shady people". I'm pretty skeptical of any company or person based there that makes transformative claims.
I would love to be downvoted and proven wrong about this in comments below, so if you have good counter-examples, please share.
Magic Leap has always raised a swath of significant red flags, but for me, one of those red flags is that the company is based in Florida. As Roger Stone famously said, "Florida is a sunny place for shady people". I'm pretty skeptical of any company or person based there that makes transformative claims.
I would love to be downvoted and proven wrong about this in comments below, so if you have good counter-examples, please share.
Apparently, last I heard (was a good year ago),
- the whole thing was overblown, overhyped, etc.
- tech nowhere near ready at the level of expectation they built. Basically was more limited than Hololens (narrow field of view) but some cool effects. Resolution/performance was apparently not good to see.
- Software wasn't much evolved either.
The entire thing was a borderline scam, despite big names etc. who apparently got hyped like the rest of us.
- the whole thing was overblown, overhyped, etc.
- tech nowhere near ready at the level of expectation they built. Basically was more limited than Hololens (narrow field of view) but some cool effects. Resolution/performance was apparently not good to see.
- Software wasn't much evolved either.
The entire thing was a borderline scam, despite big names etc. who apparently got hyped like the rest of us.
True yet I am excited about the future. Eventually this tech will be in an eyeglass form factor with perfect resolution. It might take 30 years but when it happens I think it will be more society changing than the TV ever was.
Why? TV was about broadcasting a visual experience to many people in real time for the first time ever.
AR is about creating a purely individual visual experience. Much more the domain of art than business.
What will be world changing about it?
AR is about creating a purely individual visual experience. Much more the domain of art than business.
What will be world changing about it?
I actually think business adoption will be the primary driving factor at first. When it is unobtrusive enough for the average person to use all the time the integration with your life will become deep.
https://www.youtube.com/watch?v=fMZaCVq5__g
https://www.youtube.com/watch?v=fMZaCVq5__g
I don't think it's 30 years, more like 10 max. Hopefully :)
Agreed, AR is big. Very big. This is going to be a fascinating century if we keep the current pace of acceleration.
Agreed, AR is big. Very big. This is going to be a fascinating century if we keep the current pace of acceleration.
Nreal has already eyeglass form factor: https://www.engadget.com/2019/01/08/nreal-light-mixed-realit...
An unrelated 2012 article with the same title: https://news.ycombinator.com/item?id=4350924
Wow, thanks. Sobering to see how, in a sense, how little has changed since.
If Apple created a way to launch proper AR experiences basically from a link without having to download a full app - a bit like a snapchat lens, I think they’d clean up in AR.
WebAR is OK but too limited. Just give me some javascript control of not just the objects but how people interact with them and I’ll make you something awesome.
I think there’s a huge missed opportunity here for Apple, for developers, for users.
WebAR is OK but too limited. Just give me some javascript control of not just the objects but how people interact with them and I’ll make you something awesome.
I think there’s a huge missed opportunity here for Apple, for developers, for users.
WebAR is a term these guys invented. On iOS there are main 2 technologies:
- usdz (standardized zip file of pixar usd and assets) - reality composer (proprietary new format in iOS 13. you can add limited interactivity, but all scripting is in a limited graphical language) - also apps with ARKit + your custom render or SceneKit
- usdz (standardized zip file of pixar usd and assets) - reality composer (proprietary new format in iOS 13. you can add limited interactivity, but all scripting is in a limited graphical language) - also apps with ARKit + your custom render or SceneKit
Basically I’m saying why not make something that merges the best of both of these - quick/instant to download but with proper scripting so you can make more I sterwetinf stuff than just a object viewed in front of you, at least JavaScript, ideally swift. I’d love that from both an end user and developer standpoint.
AR Quick Look is not Web AR. You can already do true Web AR with all the Javascript you want with 8th Wall.
But this doesn’t work with iPhone right? No camera access with JavaScript?
Call me old fashioned, but 2nd screen AR is not really augmented reality to me. Augmented Reality in its core means there can be virtual objects in my “reality” that look real.
It’s not about looking through a phone screen.
The article is annoying, but the technology is impressive. Some videos of actual apps show a solid lock on position in the world.[1] Unclear how well this works when stressed, but at least in brightly lit areas with lots of edges the tracker can follow, it looks OK.
[1] https://www.youtube.com/watch?v=DuO31Ro2ypg
[1] https://www.youtube.com/watch?v=DuO31Ro2ypg
In fact AR is especially bad compared to VR where the experience is fully immersive. With AR you get a confused blurry reality where you can literally see the seams and this messes with your brain that something isn’t right.