2011-01-26

Spirit Sol 734

I woke up at 3AM with a bad feeling about the drive. I logged in from home, checked out the data, and sure enough, something bad happened. This would sound impressive, as if I'm psychic or something, were it not for the dozens of times that I've done exactly the same thing and the rover's been perfectly fine.

But she's not fine this time. Most of the automated post-pass processing isn't done yet, so I have to fumble around in the low-level data for a while. But I work out that Spirit had executed the 40m blind waypoint just fine, but when she turned to set up for her first autonav waypoint, she experienced some kind of motor fault.

Oh, this is bad. This is genuinely and truly bad. The only thing I can remember that's caused something similar to happen is when we picked up a potato while climbing Husband Hill, and the pain and the time it took to recover from that -- it doesn't bear thinking of. And we don't have time for an anomaly recovery now, considering how far behind our drive metric we are.

To get a clue as to exactly what might be wrong, I grep the past data for the warnings I'm seeing here. They turn up twice: once on sol 265, and once on sol 277. I check my notes to find out what was happening on those sols, and I'm almost relieved to find that it wasn't potato time, it was the dynamic brake anomaly. What happened then was that Spirit lost the ability to sense the state of some of the brakes that keep the wheels from wobbling (uncommanded steering) during driving. And since her FSW couldn't get responses from those brakes, she freaked out. Eventually, after exhaustive analysis and testing, we decided that the brakes were working fine -- she just couldn't tell they were fine after commanding them -- and sent commands to ignore their state. And then I just forgot about it, because everything has been working fine since then.

The brakes work in connected pairs: one relay controls the RF and LR brakes, another controls the LF and RR brakes. Last time, the first of those pairs failed. This time, the symptoms are the same except that the motor IDs are different, indicating that the other pair has failed.

If we were going to have a problem like this, this might be the best problem we could have had. We'll have to do some analysis, but the odds are that we'll just do the same thing we did last time: tell Spirit to continue commanding the brakes but ignore their status, and life returns to normal.

Oh, please, oh, please, oh, please.

At least it's not my fault. This doesn't mean I don't get ribbed for it. As Chris Leger walks into the SOWG meeting room, he rolls his eyes at me. "You just had to break the rover, didn't ya?"

"I know," I say mournfully. "You give it to me for one day, and look what I do to it."

(He also points out a 25cm rock that we juuust missed. The tracks skate right by it. We could have gone over it, but it's alarming to have come this close to something that's right at our danger threshold. I look back at yesterday's PCAMs, pick out that rock, and measure it. I feel vindicated when I see the same number I saw yesterday: 19cm. "Yeah, I don't trust those rock heights," Chris says, looking over my shoulder. "I'm starting to see why not," I reply shamefacedly. It's possible the rock isn't actually that big; all we have are the HAZCAMs at this point, and their range data is sketchy. But still. As bad as things are, this is a vivid reminder that they could have been much, much worse.)

Chris is RP-1 today, and Ashley's RP-2. I'm neither -- I'm off shift today -- but I can't bring myself to ignore what's going on, so I decide to stick around for the SOWG meeting and the subsequent, hastily assembled anomaly meeting.

Options on the table at the SOWG meeting are to do mobility diagnostics and remote sensing, or mobility diagnostics followed by a drive. The latter option might sound surprising, but it's not terribly so: if Spirit fails the diagnostics, she'll raise an error and refuse to drive; if she passes, why not let her drive as far as she can get? We are, after all, 100m behind our drive metric, and 200m from Home Plate; we need to make all the progress we can. However, this isn't the place to make that decision.

That comes at the anomaly meeting. It's clear what the problem is, and we've been down this road before. (Or so we think, but one of the assignments coming out of this meeting is to ensure that this really is the same problem as before, as opposed to something else that's manifesting in a similar way.) We won't be able to do all of the diagnostics we'd like today, as we need to do some testing in the testbed first. (Luckily, Chris had the testbed reserved today anyhow.) But, somewhat to my surprise, we get the go-ahead to drive. We need to make sure we won't hit a sandy patch and dig in, but "with the appropriate bells and whistles," as John Callas puts it, we're cleared to drive 20m or so. Anything to get us closer to Home Plate!

Steve Squyres, dialed in on the telecon as usual, asks perhaps the most important question of the meeting. A scarred veteran of the ongoing Opportunity IDD anomaly, he asks, "Are we confident we can keep the program office and NASA headquarters out of our faces on this one?"

Callas cups his hand behind his ear. "Sorry, Steve, you're breaking up," he jokes.

I do my little part by looking into the timeline of events the last time we went through this -- what responses did we have, and in what order? Just as I'm finishing that, I get pulled into helping with the drive sequence.

Did I mention that I'm not on shift? I had other stuff I meant to do, but oh, well, who am I kidding? I love doing this.

Most of the rest of my day is taken up in another meeting, the Opportunity IDD science meeting. Two years and one day, it's been, and we're discussing how to drive with our gradually degrading arm. This meeting is to get the science team's buy-in on what we should do. The slides make the point, in large red letters, that we're time-limited, not usage-limited -- the IDD is going to fail at a certain date in the future whether we use it or not. The four options Ashitey presents are these:

1. When driving, stow the IDD, drive, then unstow again. This avoids leaving the IDD stowed during a thermal cycle (i.e., overnight), and has a narrow but nonzero window of vulnerability where the IDD could cease to work during the stow and be permanently stuck somewhere under the vehicle. This could block the FHAZ's view of the terrain, and worse, as the arm bounced around during drives, the elbow would gradually drop farther and farther until we were dragging the IDD on the ground like an anchor.

2. Drive in the thinker-stow configuration, as we did the last time. This will badly limit mobility -- best case, we could drive over 7cm obstacles, and our poor ability to reliably detect obstacles of that size would limit drives to 5-7m per sol.

3. Combine options 1 and 2: use option 1 for long drives, 2 for short drives.

4. Stow the IDD and never use it again.

"I don't think option 4 will be the favorite of the scientists," Ashitey jokes.

There's hardly any argument; everyone on the science team favors option 3. (As does the engineering team, but it's not our meeting.) Ray Arvidson's response is the most poetic: "Think fifty years into the future and think about our legacy. Our legacy is going to be the major stratigraphic sections we've done -- at Eagle and Endurance, and now this one in front of us. We have to protect mobility, and not at the rate of five meters per sol."

"Even if the IDD fails, we still have a bang-up mission in terms of remote sensing," he adds shortly afterward.

Squyres sums it up: "We all hate option 4, and option 3 combines the best of 1 and 2. As Jim Erickson said in the anniversary celebration the other day, our job now has become to use these rovers up. That doesn't mean we can be cavalier about using these vehicles, but we don't get any points for ending this mission with a healthy vehicle on the surface of Mars."

This is all correct, but depressing. All good things must come to an end, but is it so wrong to want them to live forever? Or at least until I get sick of them?

Not that there's a difference.

2 comments:

g said...

"All good things must come to an end, but is it so wrong to want them to live forever? Or at least until I get sick of them?"

Or at least until the replacement lands successfully?

Scott Maxwell said...

I appreciate your point, but there will never be a replacement. There will only be a third rover.