Interestingly, a version of this technology was built by a government contractor some years ago, because they too wanted to track people indoors. I don't think it used a smartphone though...
I think an acquisition would be unfortunate. This could become really huge / useful to the world without being locked up as a private company's IP. Personally I would license it rather than sell it, as well as offering offline apps and limited SaaS. You don't need an established enterprise to sell it commercially; you just need a sexy product and some industry vets with contacts. If you do end up going that way, and need someone in IT Ops / DevOps / SysEng stuff to work on the "going enterprise with a billion users", give me a shout.
Having used this and other localization techniques for large indoor robot fleets over the past decade, the hard part seems to be reliability. It was relatively easy to get a sensor fusion solution to work but much harder to get it to work consistently.
Drift is one issue but the other gremlins also cause problems with repeatability: initial localization errors (wrong aisle), electromagnetic environment change (wifi strength is not static), physical environment change (if using optical or lidar).
I think this page is a nice intro to the wifi localization space and the website has an exciting tech startup vibe. But I think if you want to really capture the undivided attention of my peers: find a clear, compelling way to demonstrate that it’s robust and repeatable. For example, walk the same real-world space twice a day for a month, stitch together the tracks in a time lapse, and promise you haven’t cherry picked the results. Especially important is demonstrating this in areas with long, repetitive aisles (grocery stores, hardware stores, warehouses, etc.) That would go far! Good luck on being acquired!
How fast do the WiFi signal strength maps get out of date?
Just someone changing the angle of an antenna or shifting a pile of stock near the router has to have a pretty big impact on signal strength.
And obviously a WiFi system upgrade where all the Mac addresses change must be a fairly big change and effectively gives a full service outage for all the users till you remap.
We've had the solution live in stores for years without any issue. Our algorithms can clean up and filter out unreliable data, based on all of the other data we have, from other APs, from previous data etc..
If they re-fit the WiFi, which they might spend a month doing once every 5-10 years, it would need a re-survey, which could take 1-2 hours. But thankfully no new infrastructure.
How does your system handle the massive variance in sensor quality (accelerometer, gyro, WiFi radio) between a high-end iPhone and a budget Android device? Does the 1m accuracy hold up across the board, or does it degrade gracefully? Getting this right seems critical for scaling to a 'billion people'
All of the videos shown on the speed-run of our technology is on 4 year-old Android devices, such as Samsung S21, Pixel 6 etc.. We always test and gather statistics on older devices to fairly represent what's available, rather than latest-and-greatest.
Peripheral chips aren't differentiated by user values like the end products that use them. You don't get more preciser sensors in high end phones. Everyone gets the same thing. You pay more only for more materials.
Sensors that are actually a lot better than standard offerings would also be subject to and/ofs of ITAR or EAR or MTCR or local equivalents thereof, so progress in IMU appears to have been stagnating a bit due to that issue. Sony Semiconductor Solutions had a Arduino IDE compatible clustered IMU board that they say you can see rotation of Earth in data, they ended up selling it with scary warnings and without any of the cool stuffs.
Awesome breakdown of the challenges of indoor navigation! One thing I was curious about... given that many modern phones now have UWB radios built in, was UWB ever considered as part of the solution stack? From what I understand UWB can get to sub meter accuracy, and I know it's used in several sports applications where precise tracking is really critical.
Is the constraint more about infrastructure (installing anchors, device compatibility, power) or something else that made you lean towards WiFi + SLAM fusion instead?
Great question. Partly on device compatibility, but mostly that we can achieve an amazing, reliable experience right now without those anchors. If we weren't getting the accuracy we needed from our current methods, we would have to look at alternatives like UWB and consider the trade-offs.
This is a very cool project / startup! I'm curious, how do you get the ground truth data? Is that just you marking down where you are as you walk through the store?
Thank you! I tried to keep it interesting but not get lost in rabbit holes.
We have around 300 hours of ground-truth data now, in 1-second intervals, which we use for algorithm training and refinement. The same as performing a survey, our team marks their location on the map, then walks to the next location, and through post-processing we can correct any errors and interpolate the locations in between. You can see this process in the diagram with the large black dots, where the user marked their location.
Huge respect to anybody who ships a product, even software. I have yet to use any indoor navigation or even have that offered, but it would be nice. Question is, is SLAM the correct term? I thought mapping meant like the device doesn't have any prior knowledge of the environment and is, well, mapping it. This more feels like sensor fusion with a kalman filter.
Love reading about these types of R&D efforts, especially when they are successful. I started in robotics in 01 and was present at some of the first commercial vSLAM efforts and materially contributed to a hardware solution utilizing IR beacons on the ceiling. We also looked at RF radio mapping at the time, but the computational power wasn't there. Great to see how far it has come and LOVE that it doesn't require infrastructure!
I am curious how deeply you have had to study the impact of how busy the store is with your signal error? Considering that humans are bags of water which is quite detrimental to RF signals, my guess is that your error increases along with the density of people.
This is very, very impressive. I have three questions. How do you know where you are within the store? Do you manually upload the floorplan? How does it know where all the items in the store are?
Very, very impressive, if (intentionally?) a little vague on how you got to 1m accuracy given you WiFi only gives you 3m.
I guess you must use the constraints on the directions a person can walk imposed by the shelves and other structures to give you orientation of the accelerometers. Which in turn means the person doing the ground truth mapping must walk down every aisle, and into every gap. That's not so difficult if your staff are doing it, but I bet you have trouble training the store staff gathering that data to do it well.
Thanks for the feedback. I wanted to keep it balanced to be accessible but also insightful.
To answer your point: we have the digital map, can use that to understand obstacles etc in the space. In some of those larger stores you see in the visual, we typically survey the entire store within 2-3 hours, it's low-effort work, not a blocker.
The few examples they show do look pretty good for a wifi-based method, although who knows how cherry-picked they are. I wonder how much the "SLAM" part is contributing and how sensitive that is to the sensor quality on the phone. I would've assumed that they'd be using vision, which seems to be the method of choice for other companies like niantic. The ground-truth data part for vision would certainly be more onerous, though.
He explains it fairly well if you understand how you'd go from wifi accuracy to SLAM. THE WIFI was providing 3m accuracy and the SLAM down to 1M. how much it provides is those two numbers. I'm sure the algorithms are complex but he points out that SLAM is corrected by the actual maps made by the self service app. So it's fairly easy to understand: the map provides a probability space, the wifi puts you within 3m and the SLAM is use to fill in the blanks with help from the probability space.
Really interesting. Feels like it would be a natural addition to a company like Brain (https://www.braincorp.com/) who is already using robots to regularly perform those kinds of survey missions and have an overlapping customer base.
Yes - one of the limitations with mobile is engaging the camera/quality of the SLAM. With a robot, they're already using SLAM with strong tracking, and controlling the hardware stack means no device limitations (uncalibrated sensors, limited WiFi pings etc.)
In my view, you need to 'streetview' the world's shopping malls and make this a free app, the Hyper Browser. Add product search & reviews and drive traffic away from non-customer stores.
Do you think the basic physics and sensor tolerances would let you go to 10^-2 meters if the environment (e.g. wifi station placements, location of RF-interfering elements) was designed by you?
If you want high performance indoor positioning , look into 5G carrier phase positioning which claims cm-level accuracy. Fuse that with IMU and optical sensing and you certainly should get decent results
Many times these stores have floor cleaning machines - either robotic or driven by a human. An employee could zip-tie their sensor to it, let it do its cleaning trip around the store, and return to collect the data later.
This would allow an employee to do several stores in a town in a single day. And potentially less chance of a workers-compensation claim being filed if they fall down while walking around looking at their device.
It's been said in one of the comments that the initial mapping by a human takes like 2-3 hours. Knowing the speed of a Roomba, I guess that it would take much more time to do it. And 'humans' are COTS devices available on any department store (sorry ;P).
My understanding is they use bluetooth for coarse positioning (e.g. you're within 30 feet) then they use a special chip (ultra-wide band) for the precision location (within a foot or two).
Of course, there's a good bit of magic within all of that to make it work seamlessly.
when I read "1m-accurate indoor GPS", my first gut reaction was along the lines of "so what? Nowadays outdoor differential/rover GPS/GNSS has <2cm precision with RTK+dual band receivers, even for civil use… and indoor allows for yet way better parameter control than outdoors, so it should be possible to achieve at least that precision" (besides, it would allow for robots to get a sufficiently precise location for indoor navigation and to shelve wares)
The thing is: that would require additional hardware though, whereas the relevant limiting criterion of the project presented in the article is obviously that it has to run with just the hardware provided by common mobile phones. Which is what makes the project interesting despite the comparatively lower precision.
Honestly, I've wanted a system for a while where shops can provide a map, and I can search for an item and it will show me where in the shop that item is. I don't think I've ever been in a shop big enough where I'd need satnav to tell me how to get there though, is that an American thing?
Tesco in the UK literally has this but only for staff. If you work for Tesco you can access any shop, view a map, view where stock is on a shelf, check stock numbers and expected delivery and all that stuff.
Things that would absolutely be an amazing QOL improvement for any shopper. But they won't let you have it because they WANT you to bumble your way around a shop. They don't want you to know where things are. That's why they move shelves around seemingly at random.
> They wanted to bring indoor maps and navigation to their retail stores... It turns out that this doesn’t just apply to retail. Every office, university campus, events venue, hotel, airport, warehouse, factory — basically everywhere indoors have some need to navigate people around, provide relevant information, and improve efficiency.
You'd think they would add this information to openstreetmap then or at least put a map on their website (and put it in the public domain so others like OSM can add it to their maps). Or put it in the store so people can take a picture. I go into target and there are posters saying to install an app for maps. Put the map on the poster!
> and they could pop up relevant promotions along the way
Oh, right, they don't want to provide information. They want to track people and spam them.
1/ they need to collect ground truth data for their algorithm to work, it doesn't magically work everywhere.
2/ the ground truth data was collected mostly by their clients, it is not their data to give away for free
I honestly don't see a problem with this technology, and I am a huge privacy advocate. First off, it uses the wifi signal strength + a model based on ground truth data to accurately position you in a map. This means that it's entirely opt-in, they can't accurately track you if you aren't using their app / connected to their wifi (yes I know some data does go out to wifi access points even if not connected, but I doubt it would be enough for this kind of tracking, and it can be disabled by the user)
Yes, they mention promotions, but again the promotions would be opt-in – if I use their app to find a product I'm looking for, they might suggest other products along the way that I might also find useful, or they might take me in a route that passes right by them. This is no different to the way retailers stock up their shelves already, placing products next to others you might want, and moving necessity items around when they want to direct you to another part of the store.
I don't know, I think it's a bit harsh to criticise this when the technology has so many applications outside of retail. I would love this in a museum or library, and even in retail I absolutely hate those interactive map displays that modern shopping malls have, where only one person can use them at a time and you have to navigate through 200 store names for the one you actually want to visit
0xbadcafebee|7 months ago
I think an acquisition would be unfortunate. This could become really huge / useful to the world without being locked up as a private company's IP. Personally I would license it rather than sell it, as well as offering offline apps and limited SaaS. You don't need an established enterprise to sell it commercially; you just need a sexy product and some industry vets with contacts. If you do end up going that way, and need someone in IT Ops / DevOps / SysEng stuff to work on the "going enterprise with a billion users", give me a shout.
Waterluvian|7 months ago
Drift is one issue but the other gremlins also cause problems with repeatability: initial localization errors (wrong aisle), electromagnetic environment change (wifi strength is not static), physical environment change (if using optical or lidar).
I think this page is a nice intro to the wifi localization space and the website has an exciting tech startup vibe. But I think if you want to really capture the undivided attention of my peers: find a clear, compelling way to demonstrate that it’s robust and repeatable. For example, walk the same real-world space twice a day for a month, stitch together the tracks in a time lapse, and promise you haven’t cherry picked the results. Especially important is demonstrating this in areas with long, repetitive aisles (grocery stores, hardware stores, warehouses, etc.) That would go far! Good luck on being acquired!
ENadyr|7 months ago
AndrewHart|7 months ago
londons_explore|7 months ago
Just someone changing the angle of an antenna or shifting a pile of stock near the router has to have a pretty big impact on signal strength.
And obviously a WiFi system upgrade where all the Mac addresses change must be a fairly big change and effectively gives a full service outage for all the users till you remap.
AndrewHart|7 months ago
If they re-fit the WiFi, which they might spend a month doing once every 5-10 years, it would need a re-survey, which could take 1-2 hours. But thankfully no new infrastructure.
mentar|7 months ago
AndrewHart|7 months ago
numpad0|7 months ago
Sensors that are actually a lot better than standard offerings would also be subject to and/ofs of ITAR or EAR or MTCR or local equivalents thereof, so progress in IMU appears to have been stagnating a bit due to that issue. Sony Semiconductor Solutions had a Arduino IDE compatible clustered IMU board that they say you can see rotation of Earth in data, they ended up selling it with scary warnings and without any of the cool stuffs.
thijson|7 months ago
https://en.wikipedia.org/wiki/Kalman_filter
It can combine several inaccurate sources and output a result that is more accurate than any one of them.
I was at an Amazon Fresh grocery store, and saw squares in the ceiling that look like QR codes. I guess that's how they are mapping the store.
jacobzweig|7 months ago
Is the constraint more about infrastructure (installing anchors, device compatibility, power) or something else that made you lean towards WiFi + SLAM fusion instead?
AndrewHart|7 months ago
jspann|7 months ago
AndrewHart|7 months ago
We have around 300 hours of ground-truth data now, in 1-second intervals, which we use for algorithm training and refinement. The same as performing a survey, our team marks their location on the map, then walks to the next location, and through post-processing we can correct any errors and interpolate the locations in between. You can see this process in the diagram with the large black dots, where the user marked their location.
Neywiny|7 months ago
AndrewHart|7 months ago
MobileVet|7 months ago
I am curious how deeply you have had to study the impact of how busy the store is with your signal error? Considering that humans are bags of water which is quite detrimental to RF signals, my guess is that your error increases along with the density of people.
dabumere|7 months ago
rstuart4133|7 months ago
I guess you must use the constraints on the directions a person can walk imposed by the shelves and other structures to give you orientation of the accelerometers. Which in turn means the person doing the ground truth mapping must walk down every aisle, and into every gap. That's not so difficult if your staff are doing it, but I bet you have trouble training the store staff gathering that data to do it well.
Best of British.
AndrewHart|7 months ago
To answer your point: we have the digital map, can use that to understand obstacles etc in the space. In some of those larger stores you see in the visual, we typically survey the entire store within 2-3 hours, it's low-effort work, not a blocker.
dimatura|7 months ago
cyanydeez|7 months ago
mhb_eng|7 months ago
mifydev|7 months ago
AndrewHart|7 months ago
grizzles|7 months ago
ckrapu|7 months ago
zokier|7 months ago
jimbobthrowawy|7 months ago
I wonder how much better this does than CCC's attempts at indoor navigation for their conferences. Those are usually pretty fun to read about.
mastercheif|7 months ago
It’s the Built for simple campus navigation video.
I’d recommend converting it to SDR.
hhhc|7 months ago
subhajeet2107|7 months ago
chiph|7 months ago
This would allow an employee to do several stores in a town in a single day. And potentially less chance of a workers-compensation claim being filed if they fall down while walking around looking at their device.
tecleandor|7 months ago
8n4vidtmkvmk|7 months ago
amelius|7 months ago
Dachande663|7 months ago
[0] https://en.wikipedia.org/wiki/Apple_silicon#Apple_U1
SkyPuncher|7 months ago
Of course, there's a good bit of magic within all of that to make it work seamlessly.
LarMachinarum|7 months ago
The thing is: that would require additional hardware though, whereas the relevant limiting criterion of the project presented in the article is obviously that it has to run with just the hardware provided by common mobile phones. Which is what makes the project interesting despite the comparatively lower precision.
mastermage|7 months ago
TheRealNGenius|7 months ago
[deleted]
voidUpdate|7 months ago
Philip-J-Fry|7 months ago
Things that would absolutely be an amazing QOL improvement for any shopper. But they won't let you have it because they WANT you to bumble your way around a shop. They don't want you to know where things are. That's why they move shelves around seemingly at random.
aaronax|7 months ago
AnnaMere|7 months ago
Peoples at x, y (lat, lng) ain't the same when placed them in multiple floors?
asadm|7 months ago
ndriscoll|7 months ago
You'd think they would add this information to openstreetmap then or at least put a map on their website (and put it in the public domain so others like OSM can add it to their maps). Or put it in the store so people can take a picture. I go into target and there are posters saying to install an app for maps. Put the map on the poster!
> and they could pop up relevant promotions along the way
Oh, right, they don't want to provide information. They want to track people and spam them.
voidUpdate|7 months ago
mpeg|7 months ago
Yes, they mention promotions, but again the promotions would be opt-in – if I use their app to find a product I'm looking for, they might suggest other products along the way that I might also find useful, or they might take me in a route that passes right by them. This is no different to the way retailers stock up their shelves already, placing products next to others you might want, and moving necessity items around when they want to direct you to another part of the store.
I don't know, I think it's a bit harsh to criticise this when the technology has so many applications outside of retail. I would love this in a museum or library, and even in retail I absolutely hate those interactive map displays that modern shopping malls have, where only one person can use them at a time and you have to navigate through 200 store names for the one you actually want to visit
ChoGGi|7 months ago
It tells me how many are in stock, aisle number, and bay number. No need for an app or walking advertisements.
hoppingturtles|7 months ago
dang|7 months ago
https://news.ycombinator.com/newsguidelines.html