Since October I’ve been saying that Apple is about to uncork a major new mixed reality strategy in September, and probably tomorrow at its developer conference.
Tomorrow at its developer conference, if Apple is really serious about AR, or its even better sister, mixed reality, then it will need to show developers a set of foundational technologies, including a new SLAM system (which really is a virtual copy of the real world that can be used for a wide range of things from self-driving cars, to navigating drones and robots, to mixed reality glasses).
But my faith is being tested this weekend for sure. Why? A certain Foxconn insider.
First, lets talk about my fears, then let’s dig into the detail that this insider brought to us and what I think it all means.
This insider has a TON of new info about Apple’s plans, but one thing he says is that there’s a good chance that Apple will kill its AR glasses, and that even if they ship they are fairly low in resolution. In fact, Apple’s glasses, as described by this insider, sounds very similar to Google Glass. Which I wore for a year and my experience led to studying the wearable/AR market a lot deeper. Yes, and that’s me wearing them in the shower, an image you have probably seen and made fun of. Funny on my wife for shooting that photo, which was designed to test if they could put up with a bit of water (something I should have tested a lot more thoroughly, turned out they were really bad about standing up to water).
So, what if I’m wrong? What if Apple’s AR efforts are much more constrained than I was expecting? Less Black Mirror and more Google Glass without the camera Google Glass had.
If so, then Apple is going more in the direction of just simplistic AR and less in the future of total mixed reality like Microsoft HoloLens shows us (which also has significant problems on field of view to solve, not to mention weight, expense, and usability problems) then I need to change strategy to more information displays and fewer polygons.
In other words, these glasses look much more suited to putting lines on the ground to help you navigate, and letting you see short bursts of text and other info than watching a movie or playing a new kind of video game. VR is certainly out the window with such a constrained product.
Think about a bigger Apple Watch screen out in front of you than something like Microsoft HoloLens that is trying to scan the room around you and “mixing” reality. Or, more accurately, an Apple CarPlay screen out in front of you. Interesting to be sure, but not the mixed reality dreams I was having.
It also means that these won’t bring the benefits of multiple virtualized monitors around you either. You can’t work on virtualized monitors at such a low resolution.
I love showing users the app RoboRaid on my HoloLens because it shows aliens blowing holes in your own walls and crawling through those holes into your room where you must shoot them before they shoot you. Lots of fun and demonstrates effectively that mixed reality, er, AR, or whatever this industry calls it (some call it “Immersive technology,” others call it “XR,” while yet others call it something else) is amazing and coming soon to all of us. I expect mixed reality glasses by 2020 that will be totally mind-blowing and already have seen several that will be released in 2018.
Apple’s Tim Cook has been sticking with “AR” which always bugged me because what’s coming in the next few years is much more like HoloLens and less like Layar, which did augmented reality on mobile phones years ago. If he were really aggressive he would be using the term “mixed reality” instead of “AR.” Or, he would come up with a new term and avoid the academic definitions.
But what if I’m wrong about Tim Cook? If I am, why did he spend a year telling the press about how cool AR will be? Why did he spend billions buying the best AR startups?
If I’m wrong, the mixed reality industry will develop slower than I am thinking it will (I was just at Augmented World Expo, where Microsoft had lines around its booth most of the time for people to get a look at HoloLens). The industry is clearly happening without Apple anyway (I know of 10 glasses under development around the industry and I’m sure there are a few I have no idea about) but it sure would be better with an Apple that pushes the market forward.
If either Apple stays out of mixed reality, or if Apple has a lackluster product offering, then that will help out the rest of the industry, but it will delay the day we see mainstream adoption by three to five years. Why? Only Apple really has the ability to prime the market. Only Apple has the necessary “ingredients” to get the entire market to care — everyone else, whether Samsung, Google, Huawei, has to build certain parts of its distribution channel to make mixed reality glasses a mainstream success. What are those?
1. A brand we want on our face. Apple yes. Microsoft no.
2. Stores in best markets so people can try on glasses.
3. All the tech in place (Apple has bought many companies in AI and AR).
4. All the relationships with content folks (music, movies, games, etc).
5. The market power to push it through (Apple’s keynotes are watched by many more people than, say, Microsoft’s keynotes).
If we need Google, Amazon, Facebook, Samsung, Huawei, Snap, or, some new well-funded startup like Magic Leap, ODG, Meta, etc to bring mixed reality to market it will happen much slower. It will still happen because of other forces like all the investment into self-driving cars which uses very similar technology to what mixed reality glasses need. Eventually Moore’s Law gets us to a place where we have amazing optics and glasses that do true mixed reality.
It will be a shame because Tim Cook’s legacy won’t improve if he doesn’t have a new hit. While Tim has made a TON of profit we look to Apple to do more than just make profit. He needs to come up with something that reminds us of Apple’s story (Apple has been the one to really push the market into new user interfaces, something it has done three times before).
The market is hoping for something new that most people don’t expect. Just like most people didn’t expect Apple to do a phone back 10 years ago when it brought us the iPhone.
We expect a new OLED screen because many of our friends who have Android devices already have OLED screens (that’s coming in the next iPhone and will bring us much better color, sharpness, and way better battery life).
Anyway, by this time tomorrow we’ll know how aggressive Apple is about augmented reality and artificial intelligence.
If it shows developers nothing then I’ll be depressed for a while. I have bet hard that Tim Cook has a major new user interface under development and if he shows up tomorrow without any magic then it’ll be depressing.
If Apple is just bringing us glasses that do what Google Glass did four years ago (albeit without the camera that people wrongfully blamed Google Glass’ demise on) then I’ll also be depressed. Google Glass did have a lot of interesting ideas, particularly when navigating us around, or showing us who is calling, and all that, but it hardly was a convincing product and wasn’t one that really showed us what the future is like when the iPhone came along (or the Apple II, or the Macintosh).
Yes, I will probably buy them, but I will feel about these the way I feel about my Apple Watch. Nice to have but I’m hardly going to gush over them. That said, look at Apple’s AirPods. Selling well even though there are lots better headphones on the market. These will probably be the same.
If all this is true then I’ll be forced to stay with Windows for my VR work and look for others in the industry to supply what I want in terms of new ways of working, shopping, entertaining ourselves, etc.
It will dramatically help the nascent VR market too. Why? There is no way in hell such a small resolution and field of view can provide immersion. Anyone who has been in VR knows what it feels like to be “immersed” in the media. It’s magical and I was hoping that Apple would have an answer to that. It is starting to look like Apple is far less aggressive than I was hoping for.
Also, it shows that Apple bought into the anti-camera feelings that I believe are wrong to listen to, but understandable in today’s marketplace. Apple knows the number one thing is these things have to be light and can’t damage its brand. Apple didn’t ship a camera in the first iPad and this might be smart way to get people used to wearing a pair of glasses before bringing us a full-blown mixed reality pair.
This leaves a HUGE opportunity for Microsoft to exploit and one I expect a rejuvenated Microsoft to use in a big way.
Interesting how so few people really understood Google Glass and its problems. Hint: the real problem isn’t with the camera. It’s with the screen and I felt that the only way to get over the screen issue (which introduces a new social contract problem that makes people around those wearing such a gadget feel nervous because the screen gets in the way of eye contact and it also introduces information asymmetry — if you are wearing it and I’m not you have information on me that I don’t have on you).
It’ll be interesting to see what Apple decides.
Aside: Shel Israel and I have started a consulting business, Transformation Group, that helps businesses figure out what they should do regarding mixed reality. We both are about to get on a plane to San Antonio, Texas, to speak to GEOINT, and you can bet these leaks are getting us to change our strategy too.
So, back to the details that I see from this Foxconn Insider:
Details on the glasses:
Called “Project Mirrorshades, Apple Iris.”
The glasses have a resolution of 428×240 per eye. There’s a prism used to reflect light into your eye and make a virtual screen in front of you. Has a microphone, an accelerometer, a microphone for Siri control, and a “small capacitive strip” for accepting calls and adjusting volume, etc.
No camera. That also seems to mean there’s no 3D sensor too, since those and cameras are very close to the same thing. If so, then it’ll be hard to do mixed reality because doing that requires sensing the real world.
Has both polarized and prescription lens snap ins. If ever released will be available in different sizes for men and women.
Also the field of view is about 14.5 degrees, which is even smaller than HoloLens.
This source believes it’ll cost about $ 600. This leaker thinks there will be a 65% chance it will be cancelled.
Here’s all the details:
Kopin NED acetate frame with polarized or prescription lens with Zeiss smart optics.
Speaker: Bone induction modules with noise cancellation.
Light sensor accelerometer for step tracking and head movement.
Magnetometer for navigation.
Capacitive Pavel Ceramic battery.
Charging circuit is a BL5 induction module.
Colors: crystal, champagne, and black.
Cellulose acetate injection mold frames into an aluminum mold. Colors are added and tumbled for finish.
It has a prism that conveys the NED (Near Eye Display) display image to lens.