The progress in information technologies such as AI has resulted in significant evolution in robotics. However, despite advancements in the robot “brain” and motion control technologies, it is impossible to utilize the full potential of a robot if the performance of its “eyes”, or its ability to perceive the state of the target object or surrounding environment, is less than satisfactory. By leveraging the optical and imaging technologies developed thus far, Canon has successfully made a huge leap forward in the functionality of robot eyes.

March 7, 2022

#Pioneering Technology #Open Innovation #IoT #Robotics #Camera

SLAM, or “Simultaneous Localization and Mapping,” is a technology that simultaneously estimates a user’s own current position (self-localization) as well as the structure of the surrounding environment (environment map building). This technology is garnering increased attention as an essential element of self-driving cars, autonomous robots and more.

In recent years, there has been growing demand for mobile robots such as AGVs (Automated Guided Vehicles) and AMRs (Autonomous Mobile Robots) as a means of saving manpower and enhancing work efficiency in logistics operations, such as transportation of parts and materials at plants or sorting goods in warehouses and transporting them to delivery trucks. Conventional AGVs are mainly guided using a track that is built by attaching magnetic tapes to the ground, and implementation of such a system is costly and time-consuming. SLAM technology eliminates the need for such work and enables easy changing of routes, and offers significant potential for various uses of AGVs and AMRs.

There are two main SLAM approaches adopted for guideless AGVs: Vision and LiDAR. The LiDAR approach, which emits laser beams to measure the shape of surrounding structures, is less susceptible to lighting conditions and allows measurement at dimly-lit areas. Due to cost considerations, however, LiDAR systems generally employ sensors that perform only horizontal scanning, which is only capable of obtaining information from two-dimensional surfaces. While it is possible to build a system for three-dimensional measurement, doing so requires a special mechanism that is extremely costly. In addition, if there is insufficient structure information for self-localization, 3D objects will need to be installed around the route to provide such information.

Visual SLAM technology, developed by Canon, employs the Vision approach. This makes use of cameras as sensors to lower cost while enabling high-precision measurement at the same time. By employing a proprietary analysis technique to identify the three-dimensional shapes of structures within a space based on the video images captured by the cameras, even information of objects with flat surfaces, such as posters on walls, can also be used for self-localization. Unlike the LiDAR system, this approach does not require installation of 3D objects and can therefore be employed in a wider variety of situations. What’s more, camera images of the Vision system can be utilized for image recognition by applications other than Visual SLAM, enabling the system to provide additional value, including a wide range of service robots and drones.

The SLAM technology uses different types of sensors for self-localization. Canon’s technology analyses video images from the left and right stereo cameras using its proprietary video analysis software “Vision-based Navigation Software for AGV” to create three-dimensional map data in real time and perform updates automatically. By obtaining accurate self-localization information that is computed based on highly precise map data of the surrounding environment, robots are able to navigate their way autonomously without relying on instructions from a human operator.

Operations at plants and logistics warehouses involve the movement of different types of goods. Because an AGV’s surroundings change dynamically, it is necessary to detect such changes and update the environment map accordingly. In many cases, two-dimensional LiDAR systems are unable to provide adequate information for detecting such changes. Conventional Vision-based systems cannot reflect all the changes on the environment map in real time and cause frequent stoppage in the operation of AGVs. The sophisticated analytical power and software design of Canon’s Visual SLAM technology allows a vast amount of information to be processed within a short time, even on low-end computers, ensuring that the environment map is optimally updated. This includes real-time updates of information on the environment map, which enable AGVs to navigate autonomously even in areas that are subject to rapid changes in the surrounding environment—a feature that is highly regarded by a wide range of industries.

Behind the development of the Visual SLAM technology is the MR (Mixed Reality) technology that Canon has acquired through its ongoing R&D efforts beginning in the 1990s. The original Visual SLAM system was created for real-time merging of the real and virtual worlds in a head-mounted display (HMD) which requires high-speed and accurate self-localization and determination of three-dimensional spaces.

Canon’s Visual SLAM System

Through a collaborative effort with NIDEC DRIVE TECHNOLOGY CORPORATION* , AGVs equipped with Canon’s Visual SLAM technology are now in use by many companies. There are numerous locations in which AGVs running on the LiDAR system are unable to operate. However, users reported that it was amazingly easy for an AGV to navigate its way smoothly through such places after they switched from LiDAR to the Visual SLAM system.

The merits of guideless AGVs lie in their ease of application as well as low implementation costs. Conventional guide systems using magnetic tapes require attachment of these tapes to the track of the AGV. Tapes must also be replaced regularly, which can increase running costs by several million yen every year at large-scale factories. In contrast, the Visual SLAM system doesn’t require magnetic tape installation, and the robot can create the first environmental map within one trip, resulting in a substantial reduction in costs.

*The official corporate name (trade name) change of Nidec-Shimpo Corporation to NIDEC DRIVE TECHNOLOGY CORPORATION on April 2023

There are high hopes for employing Canon’s Visual SLAM technology as solutions to challenges that various industries are currently facing. For example, this technology may be used in AMRs that transport materials inside steel and chemical plants where workers are more likely to be exposed to hazards, and at construction sites where there are fast-paced changes in the surrounding conditions and large quantities of materials moved around daily. Demand is also high in a wide range of other industries and use cases, such as clearing up of leftover food and dishes at hotels and restaurants, as well as contactless serving of food and medicine at medical facilities to reduce the chances of contact transmission amidst the COVID-19 pandemic. In particular, there is a growing need for automation in Japan due to the significant decline in labor population. Canon plans to collaborate with companies in a wide array of industries and fields through its policy of open innovation and engage in the automation of work processes through provision of the Visual SLAM technology, which functions as a robot’s “eyes.”

Canon has developed different types of “eyes” in the past. It has already put cameras and image analysis techniques into practical use, functioning as the “eyes” of fixed-position cameras used in surveillance, such as network cameras, as well as in robotic arms, aiding movement control as part of factory automation efforts. Visual SLAM technology is now being applied to a third area as the “eye of mobile robots”, but that challenge has just begun. By applying the optical, sensor and image-processing techniques that have been refined through the development of camera and lens products, Canon aims to develop new Visual SLAM technologies that are effective in backlit and dimly lit conditions. This will further expand the possibilities of automation through robots, thereby contributing to the safety, comfort and convenience of society and everyday life.