From entertainment to education, tourism, and more, VR (Virtual Reality) content is being applied to increasingly diverse fields of use. Previously, however, producing such content required massive equipment and special post-production setups.
Canon embarked on a quest to develop new concept lenses and software to cater to video creators’ desire for a more efficient way to produce high-resolution VR footage.
Yokoyama:
The idea of developing the lens came about when we were wondering how Canon could contribute to VR production, which had already been increasingly the subject of much interest as a form of video content.
At that time, YouTube had begun supporting 180°VR videos, and VR was expected to become increasingly important. We believed that the imaging technology and expertise that Canon had accumulated over so many years could be used to make high-resolution VR production more convenient.
Yokoyama participated in developing lenses for DSLR camera for many years. After that, he became involved in developing new key components while at the same time overseeing the XR imaging department, which handles products that use different kinds of rays. In the EOS VR System development team, he is the Lead Optical Engineer (Design & Assembly).
Noda:
When we started developing the VR lens, there was one major challenge in producing VR footage: compact, convenient shooting systems could not produce high-quality footage. Producing high-quality footage required bulky equipment and quite a bit of preparation. You had to prepare two cameras and make them precisely synced to each other.
Post-production was also very time consuming: you had to sync the footage from both cameras and match the colors and brightness for consistency.
Our solution was to streamline the high-quality VR production workflow and setup by developing a system where you could use just one camera with two lenses to record the necessary footage in one shot. With that vision, we began to develop the RF5.2mm F/2.8L DUAL FISHEYE.
Noda was the lead design engineer for compact digital cameras, and also engaged in their mechanical design. After that, he became part of the mechanical design team for interchangeable lenses. In the EOS VR System development team, he is the Lead Engineer for the aperture and focusing mechanisms.
A revolutionary lens that features two fisheye lenses in one lens body, enabling 3D 180°VR footage to be shot using just one camera. It is part of the EOS VR System along with the EOS VR Utility software, which streamlines the post-production workflow and has contributed significantly to the increasing pervasiveness of 3D VR production.
Stereoscopic footage was converted into an immersive 2D VR video that allows viewers to see the scene from different angles. VR goggles are required for the full 3D 180°VR experience.
Yokoyama:
Developing the products required support from different departments. We felt that the best way to convince them of the appeal of VR lenses was to show them actual VR footage, so we created some with help from colleagues who were good at dancing and playing musical instruments.
Using a newly produced prototype, we shot footage which we then showed to different people at Canon. Many of them had never seen a 3D 180°VR video before, and we received positive feedback that the footage “offered an even more immersive experience than expected.”
Noda:
3D 180°VR video production doesn’t end after shooting with a VR lens. The footage captured by the two fisheye lenses on the RF5.2mm F/2.8L DUAL FISHEYE needed to be converted into a special format called an equirectangular projection, which also required software development.
At the initial development stage, our team was tiny and there was no one who could take charge of the image processing aspects. So, I decided to give it a shot and became personally involved.
I had programming experience from my time as a student, so I obtained the necessary development tools from the image processing team and handled app development alongside my mechanical design development duties. This was the first time I had been involved in app development, so I faced some difficulties. Nevertheless, I managed to proceed with the task thanks to the image processing team's advice. Canon has a culture in which engineers can work directly with those in other departments without having to worry about organizational boundaries and reporting lines.
Noda:
We started collaborating with external VR video producers from early in the development process. Striving to create something that reflected their needs helped us to solidify our direction.
Canon has engineers based in the United States, one of the most advanced countries for VR production, who are tasked with collecting all information related to the imaging business, so we reached out to different VR content creators to hear their thoughts. We finalized the specification details of the product based on their input.
Yokoyama:
The RF5.2mm F/2.8L DUAL FISHEYE is unique for its ability to record two fisheye lens images with just one image sensor.
This eliminates the need to sync two different sets of footage and match their brightness and colors, thereby streamlining the workflow tremendously. Furthermore, high-definition 8K VR footage is possible when the lens is used on the EOS R5. The lens’ compact size, enabled by the RF mount’s short back focus, makes it very portable and easy to handle.
Learn more about the RF mount and short back focus at: Interchangeable lenses | Canon Global
Noda:
Feedback from the VR video creators who tested the prototype included comments about how it dramatically streamlined their post-production workflow, and how immersive the high-resolution footage felt, which gave us a sense of achievement. One creator told us how the system more than halved the time they usually required to create high-resolution VR video from setup to post-production.
Yokoyama:
We encountered various challenges when designing and manufacturing the lens.
As Canon has been producing DSLR fisheye lenses, including a high-resolution fisheye zoom lens, for many years, designing each fisheye lens to be used in the new lens was a rather smooth process as we could leverage existing know-how. However, it was our first time putting two lenses on one camera body.
In this first attempt, we put a prism inside each lens to bend the light so that both lenses could project their images onto the same image sensor. This was achieved by developing a new simulation method as well as by obtaining advice from the team that develops Canon’s binoculars.
During development, the team conducted many tests on the lenses under different environments, including darkrooms.
Interchangeable lenses are made by combining several lens elements. Each lens element has a coating to improve its light transmittance and adjust color balance.
As slight variations to the lens coating are unavoidable during the manufacturing process, the left and right lenses may appear to have different colors when viewed from certain angles.
When manufacturing binoculars, measures are taken during the coating process to prevent such variations as much as possible. These measures were adopted for the RF5.2mm F/2.8L DUAL FISHEYE manufacturing process, ensuring that high quality applies even to the product’s external appearance. As with all of Canon’s interchangeable lenses, strict product quality standards are applied to ensure consistent quality.
Noda:
It was a challenge to reduce mismatches between the two fisheye lenses. While setting strict standards prevents mismatches during the manufacturing process, slight mismatches may still occur depending on temperature and other environmental factors during use.
We decided to address that issue through software by including an automatic correction feature in EOS VR Utility.
While mismatches arising from environmental factors are usually so small that most ordinary viewers won’t notice them, studies have shown that their slightest presence in a VR production can cause “VR sickness” during long periods of continuous viewing. The RF5.2mm F/2.8L DUAL FISHEYE lens is created with utmost precision to realize quality so high that you can watch VR videos shot on them for hours and not feel tired.
Digital image stabilization improves blurring, one of the causes of VR sickness.
Yokoyama:
Thankfully, many people have bought and used the lens since its launch. We hope to eventually provide more products and services that allow even ordinary video creation enthusiasts to create VR content easily.
VR has begun to be used in applications beyond entertainment. It is increasingly being used in areas such as education and research, and we believe that the amazing immersion of high-resolution 3D VR will be useful for passing on traditional crafts and skills as well as transferring technical skills in the manufacturing and industrial sectors.
Noda:
We will expand the EOS VR System further with the involvement of our passionate team members. Also, we will develop more new technologies leveraging our experience of developing this system, as well as Canon’s supportive R&D environment.
