A little bit ago, this would have been considered far off
impossible science fiction. Now we just keep doing it!
Flying on another planet!

rom
https://spectrum.ieee.org/mars-helicopter-ingenuity-50

What Flight 50 Means for the Ingenuity Mars Helicopter Team Lead Teddy
Tzanetos on the dual rotor’s milestone aerial missionEVAN ACKERMAN08 APR
202311 MIN READ

A small helicopter sits on the dusty, lifeless, ground, with a broad
view of rust-colored hills behind it
This image of NASA’s Ingenuity Mars Helicopter was taken by the
Mastcam-Z instrument of the Perseverance rover on June 15, 2021, the
114th Martian day, or sol, of the mission. NASA/JPL-CALTECH/ASU/MSSS
MARSMARS 2020MARS HELICOPTERROBOTICS
JPL’s Ingenuity helicopter is preparing for the 50th flight of its
5-flight mission to Mars. Flight 49, which took place last weekend, was
its fastest and highest yet—the little helicopter flew 282 meters at an
altitude of 16 meters, reaching a top speed of 6.50 meters per second.
Not a bad performance for a tech demo that was supposed to be terminated
two years ago.

From here, things are only going to get more difficult for Ingenuity.
As the Perseverance rover continues its climb up Jezero crater’s ancient
river delta, Ingenuity is trying its best to scout ahead. But, the
winding hills and valleys make it difficult for the helicopter to
communicate with the rover, and through the rover, to its team back on
Earth. And there isn’t a lot of time or room to spare, because Ingenuity
isn’t allowed to fly too close to Perseverance, meaning that if the
rover ever catches up to the helicopter, the helicopter may have to be
left behind for the rover’s own safety. This high-stakes race between
the helicopter scout and the science rover will continue for kilometers.

“Two years in, 10 kilometers flown, and we’re well over an hour now in
the skies of Mars.”
—Teddy Tzanetos, NASA

For the Ingenuity team, this new mode of operation was both a challenge
and an opportunity. This was nothing new for folks who have managed to
keep this 30-day technology demo alive and healthy and productive for
years, all from a couple hundred million kilometers away. IEEE Spectrum
spoke with Ingenuity Team Lead Teddy Tzanetos at JPL last week about
whether flying on Mars is ever routine, how they upgraded Ingenuity for
its extended mission, and what the helicopter’s success means for the
future of airborne exploration and science on Mars.

IEEE Spectrum: Is 50 flights on Mars a milestone for you folks, or are
things routine enough now that you’re looking at it as just another flight?

Teddy Tzanetos: It’s hugely meaningful. We’ll come back to the routine
question in a second, but it’s very meaningful for all of us. When we
hit 10 and then 25 it was big, but 50 is a pretty serious number now
that we’re 10X our initial flight count. Two years in, 10 kilometers
flown, and we’re well over an hour now in the skies of Mars. So hitting
flight 50, it’s a big thing—we’re probably going to set up a happy hour
and have a big party for the team.

Can you talk about some of the new challenges that Ingenuity has been
facing as it makes its way up Jezero Crater’s river delta along with the
Perseverance rover?

Tzanetos: The core of the challenge here is that the paradigm has
changed. When you look at the first year of Ingenuity’s extended
operations, we were still in the Three Forks area, where the ground was
flat. We could get line of sight from the helicopter to the rover from
hundreds and hundreds of meters away. Our longest link that we
established was 1.2 kilometers—a massive distance.

And then we started to realize that the rover was going to enter the
river delta in like six months. It’s going to start climbing up through
dozens and dozens of meters of elevation change and passing through
ravines, and that’s going to start presenting a telecom issue for us. We
knew that it couldn’t be business as usual anymore—if we still wanted to
keep this helicopter mission going, not only did we need to change the
way we were operating, but we also had to change the helicopter itself.

“We owe it to everyone who worked on Ingenuity and everyone who will
continue to work on rotorcraft on Mars to try and get everything out of
this little spacecraft that we can.”
—Teddy Tzanetos, NASA

This realization culminated in the most challenging flight software
upgrade we’ve ever done with Ingenuity, which happened last December. We
went into the guts of our algorithms and added two new features. One was
the ability to detect and react to landing hazards from the air, which
involved handing over a little bit of autonomy back to Ingenuity, with
the ability to tell it, “Fly to your terminal waypoint and try and land
where we think is good, based off of orbital imagery. But if you have
better information from your images than what we humans had here on
Earth, and you see a hazard, pick a safer site and land there instead.”
So that’s one huge change in what’s happening now. And we need that at
the river delta because we’re no longer flying in a parking lot—besides
the challenge of the elevation change, the terrain is different as well,
with more, larger rocks that Ingenuity needs to avoid.

The second feature that we added was to include information about the
terrain to Ingenuity’s navigation filter. When we designed Ingenuity, we
assumed we were only going to be deployed on the flat terrain of Three
Forks. Therefore, any change in the laser altimeter measurement we could
trust to be a real change in the motion of the helicopter, or we could
at least filter that into our altitude data. But that’s no longer the
case. Now, as Ingenuity flies, if the altimeter sees a big decrease in
elevation, that could be because the ground is rising to meet us rather
than because we’re moving down. So since December, we’ve been telling
Ingenuity about the elevation profile across its intended flight so that
it knows what the ground is doing underneath it.

Now that both the rover and the helicopter have begun the river delta
climb, we’re also paying very close attention to our telecom link budget
maps. You can imagine every hill or rise that could occlude the line of
sight between the helicopter antenna and the rover antenna will have a
big impact on your telecom link, and we have wonderful maps from orbit
where we can pick a potential landing point and propagate our radio link
budget calculation across that point.

We’re trying to plan these flights as aggressively as we can to make
sure that we stay ahead of Perseverance. We don’t want to run the risk
of having a situation where the rover may need to wait for
Ingenuity—that’s not a good thing for anybody. But we also want to
provide value for the rover by scouting ahead, and what we hope to do on
Flight 50 is to get some imagery of Belva crater, which is this
beautiful massive crater to the north of where Ingenuity currently is.
We’re going to get perspectives that the rover team would not be able to
provide for the science team, and it’s really exciting for us when there
are these moments that are uniquely driven by Ingenuity’s capability. We
want to go after those, because we want to provide that value while
she’s still healthy. While we still can. We owe it to everyone who
worked on Ingenuity and everyone who will continue to work on rotorcraft
on Mars to try and get everything out of this little spacecraft that we can.

“One of the best hallmarks of technology success is when you don’t
realize it, or when it becomes boring. That means the technology is
working, and that’s a wonderful feeling.”
—Teddy Tzanetos, NASA

At one point, NASA was very clear that Ingenuity’s mission would come to
an end so that Perseverance could move on to focus on its primary
mission. But obviously, Ingenuity is still flying, and still keeping up
with the rover. Not only that, but we’ve heard from a rover driver how
valuable it is to have Ingenuity scouting ahead. With that in mind, as
Ingenuity navigates this challenging terrain, will there be any
flexibility if something doesn’t go quite right, or will Perseverance
just leave the helicopter behind?

Tzanetos: We have to look at the big picture. The most important thing
at this point is for Perseverance to collect samples and do science. If
you look at everything that needs to be done across all of the rover’s
science payloads, every sol [Martian day] is precious. And the
helicopter team understands that.

We’re doing our best to become more efficient, and I think that’s a big
win that we don’t celebrate enough on the Ingenuity team internally—how
much more efficient we are today compared to where we were two years
ago. Earlier, you mentioned flying becoming routine. I think the team
has succeeded in doing that, and I’m extremely proud of that
accomplishment. One of the best hallmarks of technology success is when
you don’t realize it, or when it becomes boring. That means the
technology is working, and that’s a wonderful feeling.

There’s what’s called a tactical window that we have between the
downlink of the last sol’s activity and when we need to uplink activity
for the next sol, which is anywhere from five to 10 hours. A certain
cadence of activities have to take place during that window, and we need
to pass certain checkpoints to get our data uploaded and radiated
through the Deep Space Network in time. We’ve worked very, very hard to
minimize our footprint on that timeline, while also being reactive so
that we can move quickly on any last-minute changes that the rover team
needs us to accommodate. We have to get in, fly, and get out.

On 4/12/23 13:07, a425couple wrote:
> A little bit ago, this would have been considered far off
> impossible science fiction.  Now we just keep doing it!
> Flying on another planet!
>
> rom
> https://spectrum.ieee.org/mars-helicopter-ingenuity-50
>
> What Flight 50 Means for the Ingenuity Mars Helicopter

back at start--

How NASA Designed a Helicopter That Could Fly Autonomously on Mars The
Perseverance rover's Mars Helicopter (Ingenuity) will take off,
navigate, and land on Mars without human interventionEVAN ACKERMAN17 FEB
20218 MIN READ
NASA engineers modifying the flight model of the Mars Helicopter inside
the Space Simulator at NASA JPL.
NASA engineers modifying the flight model of the Mars Helicopter inside
the Space Simulator at NASA JPL. PHOTO: NASA/JPL-CALTECH
ROBOTIC EXPLORATIONSPACE FLIGHTMARSINGENUITYDRONESHELICOPTERNASAMARS
ROVERSPERSEVERANCE
Tucked under the belly of the Perseverance rover that will be landing on
Mars in just a few days is a little helicopter called Ingenuity. Its
body is the size of a box of tissues, slung underneath a pair of 1.2m
carbon fiber rotors on top of four spindly legs. It weighs just 1.8kg,
but the importance of its mission is massive. If everything goes
according to plan, Ingenuity will become the first aircraft to fly on Mars.

In order for this to work, Ingenuity has to survive frigid temperatures,
manage merciless power constraints, and attempt a series of 90 second
flights while separated from Earth by 10 light minutes. Which means that
real-time communication or control is impossible. To understand how NASA
is making this happen, below is our conversation with Tim Canham, Mars
Helicopter Operations Lead at NASA’s Jet Propulsion Laboratory (JPL).

It’s important to keep the Mars Helicopter mission in context, because
this is a technology demonstration. The primary goal here is to fly on
Mars, full stop. Ingenuity won’t be doing any of the same sort of
science that the Perseverance rover is designed to do. If we’re lucky,
the helicopter will take a couple of in-flight pictures, but that’s
about it. The importance and the value of the mission is to show that
flight on Mars is possible, and to collect data that will enable the
next generation of Martian rotorcraft, which will be able to do more
ambitious and exciting things.

Here’s an animation from JPL showing the most complex mission that’s
planned right now:

Ingenuity isn’t intended to do anything complicated because everything
about the Mars helicopter itself is inherently complicated already.
Flying a helicopter on Mars is incredibly challenging for a bunch of
reasons, including the very thin atmosphere (just 1% the density of
Earth’s), the power requirements, and the communications limitations.

With all this in mind, getting Ingenuity to Mars in one piece and having
it take off and land even once is a definite victory for NASA, JPL’s Tim
Canham tells us. Canham helped develop the software architecture that
runs Ingenuity. As the Ingenuity operations lead, he’s now focused on
flight planning and coordinating with the Perseverance rover team. We
spoke with Canham to get a better understanding of how Ingenuity will be
relying on autonomy for its upcoming flights on Mars.

IEEE Spectrum: What can you tell us about Ingenuity’s hardware?

Tim Canham: Since Ingenuity is classified as a technology demo, JPL is
willing to accept more risk. The main unmanned projects like rovers and
deep space explorers are what’s called Class B missions, in which there
are many people working on ruggedized hardware and software over many
years. With a technology demo, JPL is willing to try new ways of doing
things. So we essentially went out and used a lot of off-the-shelf
consumer hardware.

There are some avionics components that are very tough and radiation
resistant, but much of the technology is commercial grade. The processor
board that we used, for instance, is a Snapdragon 801, which is
manufactured by Qualcomm. It’s essentially a cell phone class processor,
and the board is very small. But ironically, because it’s relatively
modern technology, it’s vastly more powerful than the processors that
are flying on the rover. We actually have a couple of orders of
magnitude more computing power than the rover does, because we need it.
Our guidance loops are running at 500 Hz in order to maintain control in
the atmosphere that we're flying in. And on top of that, we’re capturing
images and analyzing features and tracking them from frame to frame at
30 Hz, and so there's some pretty serious computing power needed for
that. And none of the avionics that NASA is currently flying are
anywhere near powerful enough. In some cases we literally ordered parts
from SparkFun [Electronics]. Our philosophy was, “this is commercial
hardware, but we’ll test it, and if it works well, we’ll use it.”

Can you describe what sensors Ingenuity uses for navigation?

We use a cellphone-grade IMU, a laser altimeter (from SparkFun), and a
downward-pointing VGA camera for monocular feature tracking. A few dozen
features are compared frame to frame to track relative position to
figure out direction and speed, which is how the helicopter navigates.
It’s all done by estimates of position, as opposed to memorizing
features or creating a map.

Ingenuity viewed from below showing its laser altimeter and navigation
camera.NASA’s Ingenuity Mars helicopter viewed from below, showing its
laser altimeter and navigation camera.PHOTO: NASA/JPL-CALTECH
We also have an inclinometer that we use to establish the tilt of the
ground just during takeoff, and we have a cellphone-grade 13 megapixel
color camera that isn’t used for navigation, but we’re going to try to
take some nice pictures while we’re flying. It’s called the RTE, because
everything has to have an acronym. There was an idea of putting hazard
detection in the system early on, but we didn’t have the schedule to do
that.

In what sense is the helicopter operating autonomously?

You can almost think of the helicopter like a traditional JPL spacecraft
in some ways. It has a sequencing engine on board, and we write a set of
sequences, a series of commands, and we upload that file to the
helicopter and it executes those commands. We plan the guidance part of
the flights on the ground in simulation as a series of waypoints, and
those waypoints are the sequence of commands that we send to the
guidance software. When we want the helicopter to fly, we tell it to go,
and the guidance software takes over and executes taking off, traversing
to the different waypoints, and then landing.

This means the flights are pre-planned very specifically. It’s not true
autonomy, in the sense that we don’t give it goals and rules and it’s
not doing any on-board high-level reasoning. It’s sort of half-way
autonomy. The brute force way would be a human sitting there and flying
it around with joysticks, and obviously we can’t do that on Mars. But
there wasn’t time in the project to develop really detailed autonomy on
the helicopter, so we tell it the flight plan ahead of time, and it
executes a trajectory that’s been pre-planned for it. As it’s flying,
it’s autonomously trying to make sure it stays on that trajectory in the
presence of wind gusts or other things that may happen in that
environment. But it’s really designed to follow a trajectory that we
plan on the ground before it flies.

This isn’t necessarily an advanced autonomy proof of concept—something
like telling it to “go take a picture of that rock” would be more
advanced autonomy, in my view. Whereas, this is really a scripted
flight, the primary goal is to prove that we can fly around on Mars
successfully. There are future mission concepts that we’re working
through now that would involve a bigger helicopter with much more
autonomy on board that may be able to [achieve] that kind of advanced
autonomy. But if you remember Mars Pathfinder, the very first rover that
drove on Mars, it had a very basic mission: Drive in a circle around the
base station and try to take some pictures and samples of some rocks.
So, as a technology demo, we’re trying to be modest about what we try to
do the first time with the helicopter, too.

Is there any situation where something might cause the helicopter to
decide to deviate from its pre-planned trajectory?

The guidance software is always making sure that all the sensors are
healthy and producing good data. If a sensor goes wonky, the helicopter
really has one response, which is to take the last propagated state and
just try to land and then tell us what happened and wait for us to deal
with it. The helicopter won’t try to continue its flight if a sensor
fails. All three sensors that we use during flight are necessary to
complete the flight because of how their data is fused together.

This illustration depicts Mars Helicopter Ingenuity during a test flight
on Mars.An artist’s illustration of Ingenuity flying on
Mars.ILLUSTRATION: NASA/JPL-CALTECH
How will you decide where to fly?

We’ll be doing what we’re calling a site selection process, and that’s
even starting now from orbital images of where we anticipate the rover
is going to land. Orbital images are the coarse way of identifying
potential sites, and then the rover will go to one of those sites and do
a very extensive survey of the area. Based on the rockiness, the slope,
and even how textured the area is for feature tracking, we’ll select a
site for the helicopter to operate in. There are some tradeoffs, because
the safest surface is one that’s featureless, with no rocks, but that’s
also the worst surface to do feature tracking on, so we have to find a
balance that might include a bunch of little rocks that make good
features to track but no big rocks that might make it more difficult to
land.