Recently we have seen a change in technology that emphasizes the ability of technology to conform to human needs. The goal is to support humans rather than replace them. We are moving away from robots working in automated settings, separate from humans, and towards robots working alongside humans in a collaborative environment. It is a change that we are seeing happen quickly and one where 5D Robotics is taking a lead role to provide a tag-along robot that can lead or follow as a team member rather than a remote controlled tool. The 5D aim: to ensure the safety of humans and robots working together through thorough safety testing that will become the model for testing of commercial applications.

5D Robotics, Inc., a company based in San Diego that integrates human behaviors with leading robotics hardware, recently worked closely with the military on three major assessments with military users testing squad support robots designed to help lighten the load for war fighters. The robotic technology assists soldiers who walk long distances on dismounted patrols. The robots carry the soldier’s packs and explosive detection equipment, and scout in front of soldiers to detect and neutralize IEDs. To test the efficacy of the capability, 5D interviewed and observed soldiers during real-world robotic assessments with both small and large robots. The goal of the assessments was to learn how to make human and robotic interactions safer and intuitive. 5D’s vision is to create a world where humans and robotics work together safely and efficiently.

The testing 5D completed under the military’s direction and funding was completed at Ft. Leonard Wood, MO, Ft. Benning, GA and Phoenix, AZ this year. The testing involved soldier assessments of real-world issues they encounter on patrols and soldier reviews of working in conjunction with the robots on missions. In particular, elements of the tests focused on the overall endurance of the system and its ability to follow the squad over long distances with no human input. The testing showed that the soldiers benefited from working with the robots on patrol and that they appreciated having robots there to carry their packs, detect and neutralize hazards ahead, and provide a power source.  The military assessments focused on testing the ability of the robots to support soldiers on long-range dismounted missions, and the ability of the system to lower cognitive and physical workload by providing a user interface that is easy to use.  The assessments showed that the robots can follow squads for a full day (16 miles) and that soldiers on average dedicated less than 2% of their time to the robot throughout the mission.

5D Technology Working to Save Soldiers’ Lives

5D’s Behavior Engine (BE™) software uses a combination of ‘lead and follow’ behaviors to autonomously drive the robot in front of and behind humans at a minimal safe distance. The military used to think that robots and humans should be a long way off from one another and generally used GPS or video-based remote control to coordinate robots. Instead, there is now an effort to emphasize proximal interaction where the lead and follow distance is often quite small. With funding from the military, 5D created a removable IED Defeat (IEDD) kit that provided multiple robots ranging in size from 250 to 4500lbs with the ability to detect and map a wide variety of chemical, radiological and explosive hazards. The kits are designed to provide the robots with the same behaviors and ensure that each robot can be tasked from the same hand held display regardless of the manufacturer. The plug and play kit also allows the robots to use the sensors that soldiers already have in the inventory. The 5D software provides interoperability across a wide size range of robots including the Armadillo from MacroUSA that weighs less than 10lbs. Each robot has benefits and limitations depending on its size, weight, power capabilities and mobility.

The principal idea behind 5D’s work with the military is that just as in the biological world, peer-to-peer interaction is more robust than using world modeling, global positioning and centralized control. Although radio communications are possible, the robot behaviors are primarily reactive in nature and don’t depend on human input or external signals (such as GPS) in order to work. They can perform in GPS denied areas and if communications are unavailable. At the heart of the 5D technology is an innovative approach which emphasizes peer-to-peer positioning that utilizes UWB tags on each of the vehicles and squad members to provide highly accurate relative positioning. This has been a primary innovation and positions the technology for use in a wide range of commercial applications as well as military.

Robots lighten the load and provide power in the field

Figure 2: Three robots work together at Ft. Benning

At Ft. Benning, multiple robots including two ACERs and one Segway 440X were used together in a variety of exercises where the soldiers were permitted to try various configurations and formations of the robots in order to accomplish the mission of getting through a combination of rural and urban terrain. This study found that it was possible for soldiers to task a group of robots as if they were one robot, termed wagon training. The wagon training capability allows the squad to use multiple kinds of robots so that each can have unique capabilities and skills. For instance, a smaller robot can go inside of caves, bunkers, tunnels and buildings whereas the larger robot can carry more equipment and push a roller and/or flail to neutralize hazards.

Figure 1: Two ACER robots work together on a patrol task

At Ft. Leonard Wood, experiments were done to assess the use of the 5D capabilities for route clearance missions. The assessment involved 32 combat engineers who had been trained to accomplish mine detection and Improvised Explosive Device Defeat (IEDD) operations. During the assessment, users could use the mine roller developed with Humanistic Robotics or a Flail, which was developed by Mesa Technologies. One ACER was set up for tele-operation, while the other was configured to perform lead and follow behaviors using three lasers placed around the robot. Soldiers successfully used an X-box controller to cue both autonomous lead and follow capabilities while traversing unimproved paths. Soldiers liked the reduction in workload due to using the lead and follow behaviors as opposed to the direct tele-operation they had used in the past.

A lighten the load assessment with the Segway 440X was performed at Ft. Leonard Wood through a collaboration between the Rapid Equipping Force and the Maneuver Support Center of Excellence. This lighten the load test involved six participants using a single robot to carry 200lbs of critical equipment (including packs, water and food) in what is called a “lighten the load” follow formation where the squad members ignored the robot and relied on it to keep up, following behind them over various terrain. The Segway 440X system was loaded with approximately 200lbs (sandbags, water, and assorted personal items), powered on generator power at 0745 hours, and placed in "follow-me" mode while the soldiers walked through a variety of wooded terrain. The system covered 16 miles and traveled at an average of 2.1 mph. The system required no interventions from the user except in off-trail, wooded conditions where it occasionally saw weeds as obstacles.  After 16 miles, the test ended with approximately 50 percent of battery life and 33 percent of the fuel remaining in the generator. The Army program management representatives who ran the test stated the test results indicated the robot could keep pace with a patrol group throughout the course of a mission, which generally lasts 10-12 miles. Participants in this mission indicated that the system was easy to understand and that robots would be beneficial in carrying soldiers’ gear.

At a test performed in Arizona, 5D worked with a company called Exponent to demonstrate a follow capability on the JIEDDO Wingman project funded by the US Army Corps of Engineers (USACE) Engineer Research and Development Center (ERDC). The intent of the project was to provide a lighten the load capability while also demonstrating a novel interface that allowed the user to task the robot and visualize video and map data on the fly. This interface strategy included a Wii controller, glove interface, and a smart phone.

To assess the workload of the system, a UGV Test course was designed and built by Exponent at their facilities in Phoenix, AZ. The test course included a variety of sections: an open road, jersey barriers, a mock Afghan village, lightly wooded areas and a hill to ascend and descend. A total of 16 runs were performed, including two segments of each environment.

The operator interacted with the robot using a Wii-Nunchuk that was connected to an Exponent wearable Operator Control Station that had a 1.3 Ghz Microhard radio to communicate with the robot. The operator navigated the robot through each section of the environment one or two times in both follow-me and lead-me modes. Metrics were captured of the time to complete a run, the section of the course, the total number of interventions and the total time of interventions. The intervention % was recorded as a percentage of time the user spent intervening with the robot. An intervention is defined as a situation where the semi-autonomous capabilities of the system were unable to follow the operator and the operator had to tele-operate the robot to continue.

For the follow-me tests, in 12 of the 14 test cases, the operator had no intervention with the robots. In the other two cases, the operator had only one intervention. In the first instance the intervention took 29 seconds out of 763 seconds (3.8 percent of total time). In the second instance, the operator took 52 seconds out of 206 (25.2 percent of total time.)  Overall, the operator drove for 3462 seconds (57.7 minutes) and interacted with the system for 81 seconds (1.35 seconds) for an overall workload of 2.3 percent. Table 1 shows the summary of the follow-me trial runs.

For the lead-me tests, in 10 of the 13 test cases, the operator had no intervention with the robots. In the other three cases, the operator had one, one, and three interactions respectively. In the first instance, the interaction took 15 out of 822 seconds or 1.8 percent of the total time. In the second instance, the interaction took two seconds out of 104 seconds, or 1.9 percent of the total time. In the third instance, the interactions took 165 seconds out of 225 seconds or 73.3 percent of the total time. Overall, the operator drove in lead-me mode for 2903 seconds (48.4 minutes) and interacted with the system for 182 seconds (3.03 minutes) for an overall workload of 6.3 percent. Table 2 shows the summary of the lead-me trial runs.

Conclusion

Robotic safety testing of this nature is important to 5D since the goal of our robotic software is to improve human and robotic interaction. As humans operate in close proximity to robots more frequently, 5D’s work with the military to test human-robotic safety is taking robotic innovation an important step closer to routine use in military settings, as well as commercialization and mass applications.  

The real-world user assessments showed the need for proximal command and control for dismounted patrols, including lowering the workload to below five percent, maintaining an operational tempo of five mph and ensuring that one or more robots can autonomously follow the squad for up to 15 miles on a narrow trail. The assessments indicated the potential benefit of moving towards 5D’s approach – a reactive, nearest-neighbor interaction that is more intuitive and away from the underlying tasking scheme of a map-based, GPS-driven approach that requires continuous communication.

Our goal is robots that can make life simpler and safer.  It’s not about the robot, but about how we link ourselves to technology. Technology either adapts to us or forces us to change our lives in an effort to adapt ourselves. 5D is placing the onus on the robot. It should interact with us using the tools we already have like smart phones, Xbox controllers and Wii’s.

The same technology that has provided value for warfighters can be brought into the commercial world to provide safety and efficiency for forklifts, golfcarts, lawnmowers and porters to carry your luggage in resorts and shopping carts that can follow or lead you in retail stores. By linking humans to intelligent vehicles via their smart phones, we can change the way people transport themselves and their stuff. One way to do that is with the 5D UWB tag system that provides a new means to know where you are indoors and outdoors and to track position accurately in real time. Consider the screen shot of the 5D Safety Application for the Android Smart Phone. It uses the same UWB tag developed for the military but is now applied to keeping vehicles and pedestrians safe as they move through urban terrain. The applications are boundless.

Figure 3: Screen shot of squad support Android App

About Curtis Nielsen
Curtis Nielsen is Director of Software Engineering at 5D Robotics, Inc. He manages day-to-day software activities and leads the development efforts for the contracts and internal research and development at 5D. Curtis has identified, implemented, and customized the various software processes that have led to software engineering excellence at 5D Robotics.

Prior to 5D Robotics, Curtis worked at the Idaho National Laboratory in their Robotic and Human Systems group as a Principal Research Scientist. While there he was one of the recipients of the 2006 R&D 100 award and is a co-author on over 25 international and national conference, journal and book chapter publications. Curtis holds a Bachelor, Masters, and Ph.D. in Computer Science from Brigham Young University.

The content & opinions in this article are the author’s and do not necessarily represent the views of RoboticsTomorrow

Featured Product

Helios™2 Ray Time-of-Flight Camera Designed for Unmatched Performance in Outdoor Lighting Conditions

The Helios2 Ray camera is powered by Sony's DepthSense IMX556PLR ToF image sensor and is specifically engineered for exceptional performance in challenging outdoor lighting environments. Equipped with 940nm VCSEL laser diodes, the Helios2 Ray generates real-time 3D point clouds, even in direct sunlight, making it suitable for a wide range of outdoor applications. The Helios2 Ray offers the same IP67 and Factory Tough™ design as the standard Helios2 camera featuring a 640 x 480 depth resolution at distances of up to 8.3 meters and a frame rate of 30 fps.

More Products

Feature Your Product