This is something I built while learning RL, and I decided to open-source. I noticed that every major RL algorithm (Actor Critic, A2C, PPO, TD3 etc.) would be written from scratch, even though they shared a lot of the same features. And every implementation of the same features would be slightly different across each implementation. So trying to take a feature from one algo to another, or even to try building your own features, was a massive pain and error-prone.
HelloRL is a new modular framework, built around a single `train()` function, which scales up to every algorithm. The difference between Actor Critic (discrete, online, monte-carlo, simple) and TD3 (continuous, offline, 1-step rollout, targets, reference critic, etc.) is just a different set of modules. Easy to swap between algorithms, or mix and match features, or build your own modules.
Great question. Partly on device compatibility, but mostly that we can achieve an amazing, reliable experience right now without those anchors. If we weren't getting the accuracy we needed from our current methods, we would have to look at alternatives like UWB and consider the trade-offs.
We've had the solution live in stores for years without any issue. Our algorithms can clean up and filter out unreliable data, based on all of the other data we have, from other APs, from previous data etc..
If they re-fit the WiFi, which they might spend a month doing once every 5-10 years, it would need a re-survey, which could take 1-2 hours. But thankfully no new infrastructure.
Thank you! I tried to keep it interesting but not get lost in rabbit holes.
We have around 300 hours of ground-truth data now, in 1-second intervals, which we use for algorithm training and refinement. The same as performing a survey, our team marks their location on the map, then walks to the next location, and through post-processing we can correct any errors and interpolate the locations in between. You can see this process in the diagram with the large black dots, where the user marked their location.
Thanks for the feedback. I wanted to keep it balanced to be accessible but also insightful.
To answer your point: we have the digital map, can use that to understand obstacles etc in the space. In some of those larger stores you see in the visual, we typically survey the entire store within 2-3 hours, it's low-effort work, not a blocker.
All of the videos shown on the speed-run of our technology is on 4 year-old Android devices, such as Samsung S21, Pixel 6 etc.. We always test and gather statistics on older devices to fairly represent what's available, rather than latest-and-greatest.
Yes - one of the limitations with mobile is engaging the camera/quality of the SLAM. With a robot, they're already using SLAM with strong tracking, and controlling the hardware stack means no device limitations (uncalibrated sensors, limited WiFi pings etc.)
Aside from saving time, I'm bad at writing. Especially emails. I often open ChatGPT, paste in the whole email chain, write out the bullets of the points I want to make and ask it to draft a response which frames it well.
You could say the same about software and app stores. If safety were the top priority, then the safest option is to say no apps, but that isn't competitive or lucrative. Apple's approach is to create safe frameworks and a review process that allows the App Store to exist.
I'd love to read more on this topic - how companies learn to do this, and then go about establishing it internally, so that it becomes part of their brand and culture.
I run an AR startup called Dent Reality - Apple lists us in their Indoor Maps program docs. We've built on IMDF to solve this for grocery stores, and provide shopping list navigation with maps and AR. 60 second demo video here:
HelloRL is a new modular framework, built around a single `train()` function, which scales up to every algorithm. The difference between Actor Critic (discrete, online, monte-carlo, simple) and TD3 (continuous, offline, 1-step rollout, targets, reference critic, etc.) is just a different set of modules. Easy to swap between algorithms, or mix and match features, or build your own modules.