No matter how expertly designed or rigorously tested, every robot is bound to encounter edge cases at some point or another. How the issues are handled and how field ops are done, determines the longevity of the robot system or product.
How Robotic Technology Can Mitigate COVID-19
Q&A with Jeff Linnell, CEO | Formant
Tell us about yourself and your role with Formant
My name is Jeff Linnell and I am the CEO of Formant, a cloud robotics company. As a 30-year creative technology veteran with expertise in Robotics, AR/VR and filmmaking, I lead Formant’s strategy, drive product development, and help customers build smart, data-driven robotics-based businesses.
I bring an unorthodox approach to robotics. Part cinematographer, part creative director, and part self-taught engineer, I founded my first of a series of successful companies whose commonality is the application of a unique, often orthogonal expertise to technical challenges. My work led me to San Francisco where I founded Autofuss and Bot & Dolly, a design/production company and an engineering studio specializing in automation and the novel application of traditional robotics. Both companies are widely regarded as revolutionary in their fields and resulted in numerous accolades including an Oscar for the blockbuster film “Gravity”. Both firms were acquired by Google in 2013 where I assumed roles focusing on human/machine interaction and digital fabrication as well as leading Product at Google Robotics.
Please give us a brief overview of Formant Capture.
Formant bridges the gap between autonomous systems and the people running them. Our best-in-class cloud operations platform allows autonomy companies to observe, operate, and analyze fleets of robots as they navigate the uncertainties of the physical world. An important aspect of managing fleets of robots is being able to collaborate and coordinate with the field personnel. Formant Capture lets someone who is standing next to a robot use their phone’s camera to stream video into Formant. This is an agile, intuitive way for teams that support robots in the field to get extra context about issues. The data is time-stamped appropriately and made available as part of that robot’s observability record. Formant Capture helps the team tackle issues, gain insights, and document events of interest for immediate review as well as auditing at a later date.
What’s the extent to which you can really manage a robot remotely?
To understand how robots can be managed remotely today, it is important to understand the lifecycle of a robot first, from deployment through troubleshooting.
It is typical to involve human operators during the initial deployment and early use of the robot. For example, consider a robot that seeds crops or transports goods from one point to another. In these cases, it is typical to have deployment personnel to collaborate closely with the field operators. The operators need to be trained during the deployment so they can use the robot appropriately. While the physical presence of a field operator is initially required in some form, Formant’s observability tools allow multiple field engineers to remotely collaborate in bringing up a robot as a distributed team.
Once the robot is set up, there is a kickoff and continuing education, which can all occur remotely using Formant Capture, Share, and other observability and collaboration tools. The training should continue at a regular cadence for a little while until the operators are familiar with the robot.
At this point the robot is in regular usage and can move to the monitoring stage. It does what it is trained and supposed to do. Fine-tuning happens at this point; the robot is analyzed to make the robot more efficient and improve its duty cycle. The point where a robot can truly be managed remotely is after the application is relatively stable. Online technicians can dictate it to move out of the way of objects blocking its path or update its parameters and software for improved productivity.
There are and will be cases where a technician needs to be called out to the field. With Formant Capture, the goal is to reduce the number of on-site visits. Previously, if a robot was unresponsive, a technician had to go out on-site, evaluate and troubleshoot the problem. In many cases, the technician would leave to get a replacement part or update and return to fix the robot. Formant Capture in many cases, takes the place of the initial visit. For example, an employee of a farm might be sent a CaputeLink that enables them to show a technician back at base an issue on an AGV without a field visit. This visual, alongside the robots logs and telemetry are oftentimes enough information to address an issue.
If you have issues, how do troubleshoot robot problems remotely?
Every robotic application is unique. As a result, there’s no one-size-fits-all solution to process automation. Nobody knows better which tasks are getting in the way of productivity than the field-ops team themselves. By allowing for fully-customizable automation, the team can tackle the most pressing time-wasters in the way they know is best. As they say, an ounce of prevention is worth a pound of cure. If CPU usage above 80% means the robot is likely to go offline, it is important to know about it before it happens.
No matter how expertly designed or rigorously tested, every robot is bound to encounter edge cases at some point or another. How the issues are handled and how field ops are done, determines the longevity of the robot system or product. Formant’s platform helps monitor and provide analytics to aid troubleshooting and in advanced cases provide predictive maintenance data.
Even though it is possible to remotely troubleshoot a robot, there are times when a technician is needed to replace a part or service part. But it could reduce the number of times a technician needs to go out. Additionally, it reduces the amount of time required to get the system back online.
Currently, what tools do field technicians have to collaborate with remote support?
Today, many robots are designed to perform specific tasks, like material handling, inspection, drilling, and involve supervision or control from an operator to perform these jobs. In order to scale, one operator needs to manage a fleet or robots and instead of performing the task, the operator troubleshoots the issues. Far more interesting than a robot and human collaborating, is a human supporting 25-100 robots.
Operators can observe, analyze, and operate a fleet of robots using Formant. Observability enables monitoring and visualizing live and historical states of the fleet with timely alerts to determine root cause issues. Analytics gains insight into hindsight by being able to drill down to meaningful dimensions to improve efficiency and track trends. Operators can also take instant control of a robot using teleoperation or intervention.
Formant offers several collaborative tools to field technicians including Formant Capture - to share field data with support staff. To share observability data with field technicians and to annotate specific moments in time for future analysis by business analysts or engineers are two of the most commonly used tools. The goal is to share the data that remote support sees with the field technicians, so they are both looking at the same situation.
Formant delivers tools for the whole team, including technical and non-technical members to help with troubleshooting. To start, an operator can rewind and replay to understand what happened. To add context, the operator can request additional video from the field, add comments, or annotate specific moments in time. The technician also has the ability to send a command or SSH to the robot from anywhere. The last step could be to teleoperate or take over the robot. To automate away these frequent issues and forever optimize the robot fleet behavior and dynamics, operators can also create workflows based on events or thresholds.
What do you see as the biggest barriers to implementing remote support of robotics?
Scalable, end-to-end remote support of robotics involves not only observability, but also other capabilities as mentioned above. One of the biggest challenges is education and awareness about solutions, like Formant that provides these capabilities.
Right now, widely known interfaces and tools (e.g. SSH/VPN or tools like rviz.) for remote support are mostly focused on technical people. Robot data can be technical and can take time to understand. They are not aware that options, like Formant, exist that enable and bring non-technical people to operate and manage robots remotely.
Training and being able to upskill non-technical people in related fields to manage robots, like training a forklift driver to understand robot data to operate a fleet of forklifts.
One other barrier to implementation is bandwidth costs. Robots generate an immense amount of video and data that can get expensive to ingest or transmit all the time from the field. Formant has built-in capabilities to ingest data on-demand - where data is stored locally on the robot and only sent out on explicit request or on important triggers and events.
Where do you see an opportunity for your technology to help mitigate manufacturing disruptions caused from the COVID-19 pandemic?
In the past, robots were designed to improve efficiency and automate manual labor tasks. Manufacturing is a more structured world and while there are several improvements that can be made, like robotics cleaning and sanitizing, to improve social distancing and safety. The real disruption that COVID created is in the supply chain. Today, logistics companies are working to implement and deploy fleets of robots and the future is expected to have a less risky supply chain and significantly increased efficiency and predictability. For example, in logistics warehouses, they are supplementing their workforce with robots that sort and transport items to different areas of the warehouse. This helps alleviate worker density and follow social distancing guidelines. This not only helps with worker's well-being but ultimately improves capacity to support the increased volume.
Formant enables many verticals, from logistics to healthcare, to deploy, manage, and monitor their fleets of robots, enabling them to get to market faster and remain efficient and insightful in the market. While Covid-19 accelerated the need for more robots, Formant enabled these companies to deploy and manage their growing fleet.
Data-driven robotics companies can contribute in meaningful ways. Formant is working with its clients to support social distancing through teleoperation, remote monitoring, collaboration, and technical support. As a result, operating costs are lower by reducing on-site and field visits by engineers and support staff. Root cause analysis is faster by being able to “look through the eyes” of the field operator and the robot both simultaneously, and in a single frame, to diagnose the issue and get the robot back to work.
Can you share with us how you think robots can be used to help enforce social distancing?
Today, there are companies using robots to enforce social distancing. In Singapore, Spot the robot reminds people to stay six feet apart. There are others who are broadcasting public safety measures.
However, the biggest opportunity for robots to help with social distancing comes from them being able to do essential, repetitive jobs - making human labor less risky, more productive, and definitely less crowded. There are robots working in hospitals sanitizing, delivering medicine or supplies alongside humans thereby reducing social interaction. Agriculture is another area, for example, robots help with picking, packing or carrying yield from the fields to destination thereby reducing human contact. In construction, worker density can be reduced by supplementing the workforce with robots that transport, survey or inspect, sort, or transport items to different areas of the construction site. Also, last-mile delivery of groceries or food to enable safer, socially distant services is a well-known example of how robots enable social distancing.
The content & opinions in this article are the author’s and do not necessarily represent the views of RoboticsTomorrow
Comments (0)
This post does not have any comments. Be the first to leave a comment below.
Featured Product
