“Oh no, the code is broken. What has changed?”, is the first thing Maarten de Vries, software architect at RoboHouse, thought when he used an updated version of OpenCV, a piece of code that allows computers to recognise images. OpenCV is open source. This means anyone can apply it to their own projects. “The function was now only working on RGB colour images, but we needed it to work on black and white images”, says his colleague Hans Gaiser, a computer vision programmer. This is when the two developers started working on a piece code that would later be used on Mars.

Both developers work part-time for Fizyr, a provider of vision software that can automatically pick parcels, baggage or other items. In order for this to work, the computer needs to know what it sees. That is where you need computer vision engineers. And when Hans Gaiser and Maarten de Vries write code, they almost always use open source ingredients. Chunks of software that they find online. In this case, they were using a project called OpenCV. But since its functionality had been changed, Hans and Maarten decided to modify: “We wrote about 20 lines of code, changing a small part of their code called a patch.” When you add or adapt something in the open source community, it is considered common decency to flag this with a ‘pull request‘. Project-owners can then review the contribution, give feedback and may sometimes integrate it into a project. The duo from Delft submitted its pull request on 1 March 2017, and moved on.

On 19 April 2021, the world holds its breath as NASA attempts to fly a miniature helicopter on Mars in what is the first powered, controlled flight of an aircraft on another planet. The Ingenuity Mars Helicopter rises to 3 meters above the surface of Mars, hovers for 30 seconds, and then rotates before descending safely to a chorus of cheers here on planet earth. The developers in RoboHouse look on with admiration, and then continue with their work. Until ten days later, Maarten de Vries receives an email by a co-worker: “Check this out! You’ve got a badge for contributing to code used on the Mars helicopter!”

The email refers to Github, an influential website in the open source community: “Today, nearly 12,000 developers will see a new badge on their GitHub profile celebrating their contributions to the specific versions of projects and libraries used by NASA to fly the Ingenuity Helicopter on Mars.”

The Ingenuity autonomous helicopter is a fine example of robotic innovation, and it is the first aircraft to have ever flown on another planet. Something else is special about it as well: its flying program is open source. Many thousands of bright minds have contributed to that successful take-off on 19 April. And these contributions were mostly modest. So our two coders are properly humble about their badge. “In fact, the chances that our code physically resides on the Mars helicopter are extremely slim,” says Hans Gaiser. “NASA tends to use many different open source projects and always uploads the entire code. And the aircraft only addresses the parts it really needs during flight.” True. But that does not dismiss the possibility that their code may be on Mars. Look at the images. They’re black and white, aren’t they?

NASA’s Ingenuity Mars Helicopter took this shot, capturing its own shadow, while hovering over the Martian surface on 19 April 2021, during the first instance of powered, controlled flight on another planet. It used its navigation camera, which autonomously tracks the ground during flight. Credit: NASA/JPL-Caltech

What we find so cool about this story, is that working with open source seems to open up a whole world of positive surprise and unforeseen effects. Your tiny contribution to a solution for picking socks from a conveyor belt, may end up moving helicopters on another planet. Hans says: “For our team it is awesome to realise that even the smallest thing we make, can somehow be of great value to the world.”