iPad Pro 2020 LiDAR Demo

We’ve had our iPad Pro 2020 with LiDAR for just over a month now. We did an initial investigation of the data when we got it and quickly realized we would not be able to use the new sensor like a TrueDepth device. Here’s a tweet we sent out about this revelation: https://twitter.com/ScandyCo/status/1242895423544659976

Well, good news! We got some free cycles back for R&D this week and were able to use the data that Apple does provide with the LiDAR camera flowing through into our Roux SDK. Here’s the demo of where we are at as of 11am on April 29, 2020: 

‍

‍

Getting to this demo had a fun little development narrative. Things were a little artistic at first:

This is pretty typical for a new R&D effort. Get the data in a raw format and try to make sense of it. After spending a day working with this data and making some headway we were able to at least sort through the point data correctly. This looked a whole lot better:

So now the final piece. Where are my faces at? The Apple example for the LiDAR setup shows how to get the face data but all the indexing was weird. Shout out to the folks at Zeitraum for doing a great blog post about how they added a save option to the existing Apple demo. Find their medium article here: https://medium.com/zeitraumgruppe/what-arkit-3-5-and-the-new-ipad-pro-bring-to-the-table-d4bf25e5dd87

After getting the face indices sorted out everything was gucci!

Checkout the file on Sketchfab: https://skfb.ly/6Spuv

We’ll be working hard the rest of the week to get this fixed up and ready for shipping into Scandy Pro ASAP. With any luck it’ll be with Apple for testing by the weekend 🤞🏽.

‍

Happy scanning!