“Houston We Have a Podcast” is the official podcast of the NASA Johnson Space Center, the home of human spaceflight, stationed in Houston, Texas. We bring space right to you! On this podcast, you’ll learn from some of the brightest minds of America’s space agency as they discuss topics in engineering, science, technology and more. You’ll hear firsthand from astronauts what it’s like to launch atop a rocket, live in space and re-enter the Earth’s atmosphere. And you’ll listen in to the more human side of space as our guests tell stories of behind-the-scenes moments never heard before.
For Episode 75, Matt Lemke, Orion avionics, power and software deputy manager, discusses how Orion is radiation-hardened so the systems inside can withstand the harsh environment of space. This episode was recorded on September 5th, 2018.
Transcript
Gary Jordan (Host): Houston, we have a podcast. Welcome to the official podcast of the NASA Johnson Space Center, Episode 75, Radiation Shielding. I’m Gary Jordan and I’ll be your host today. On this podcast, we bring in the experts, NASA scientists, engineers, and astronauts all to let you know the coolest information about what’s going on right here at NASA. So, if you’re familiar with us, you may have heard us talk about radiation before. We’ve talked about the effects on living organisms from Dr. Zarana Patel on Episode 57. And we’ve had Dr. Steve Johnson tell us more about the physics and the day to day operations on the space station dealing with radiation and space weather on Episode 64. But today we’re learning once again about Orion and how the spacecraft is built to withstand the radiation environment of deep space from Matt Lemke. He’s the Orion avionics, power and software deputy manager here at the Johnson Space Center. He understands how Orion is radiation-hardened so the systems inside can withstand the harsh environment of space. The key word here is redundancy. So, with no further delay, let’s jump right ahead to our talk with Mr. Matt Lemke.
Enjoy.
[ Music ]
Host: Matt, thank you so much for coming on the podcast today to talk about radiation shielding on Orion, the deep space spacecraft that we have here.
Matt Lemke: Yeah, it’s a pleasure to be here.
Host: All right. So, we’ve talked about radiation in general before on the podcast. We’ve talked about how it affects the human body, we’ve talked about the physics of it, space weather, why we have to be concerned about it in the first place, but not specifically how it affects spacecraft. And this is an interesting one because Orion is going to go deeper into space and it’s going to pass through the Van Allen radiation belt and there’s a lot of concerns when it comes to traveling in deep space, specifically when it comes to radiation. So, why don’t we just kind of start off there. Radiation. What is it? Why do we have to be concerned about it?
Matt Lemke: OK. So, the radiation comes from the sun.
Host: OK.
Matt Lemke: And so you’ve probably heard that in your space physics lecture and they probably did a better job than I will, but you’ve got a couple different sources of it. Different types of radiation. So, you have this Van Allen belt you mentioned and we have a magnetic field around the Earth. And that traps a lot of protons and, you know, that size of particle, and that’s what we’ve been used to dealing with for so many years. Space station, space shuttle, we got really good about how to work in the proton environment that’s there, so there’s real good ways to test for it, see what our equipment’s going to do, make sure we can recover. But now as we go out beyond and through the Van Allen belt and into deep space, we have to start worrying about solar flares and galactic cosmic rays and these things that have a lot more energy and can come into the electronics and corrupt them. And so that’s what we’re really worried about versus the effects to the human is, what’s the effect to the computer?
Host: Yeah.
Matt Lemke: And so a radiation, a proton, a galactic cosmic ray, something like that can come in and it’ll change a 1 to a 0 in your electronics. So, your computer thought, hey, I was working along, I’m doing my instructions one at a time, executing my software, and suddenly the data is wrong or the software program has been corrupted by changing a 0 to a 1 or vice versa. And so we have to build systems that can handle that.
Host: OK.
Matt Lemke: You know it’s going to happen, radiation’s out there. It’s statistical whether or not you get hit, and so we designed systems that can tolerate those bit flips.
Host: Yeah. So, I mean just besides the human element, now we’re talking about– radiation, when it comes to being concerned about what radiation is going to do to a spacecraft, corrupting how it works, that’s really the concern.
Matt Lemke: Right. That’s the concern.
Host: And that’s a big concern.
Matt Lemke: Right. And so, on Orion, if you think about it, the computers control almost everything.
Host: Yeah.
Matt Lemke: The life support system, the communications, the propulsion system, navigation, so everything is controlled by the computer and so when the computer malfunctions you have a problem. And actually sometimes it’s good when it really malfunctions bad, because then you know you have it, right? You have your Windows machine and you get the blue screen of death. Well, at least you know something happened and you can reboot it and do something.
Host: OK.
Matt Lemke: So, that’s one type of error we can have. The other type of error is it just corrupts the data. So, you thought you were getting, you know, readout coms and it says it’s 15 whatever units and that’s a wrong number now because a bit got flipped. So, it can mislead you and it can just corrupt it and make it stop working.
Host: Huh. So, when it comes to understanding these problems and knowing kind of– trying to understand what you have to do to fix it, my first thought comes for detecting it. How do you know that you have a wrong number? How do you know that the data that you’re getting is in fact wrong and has been affected by radiation?
Matt Lemke: Right. That’s step one. And so we have a flight computer.
Host: OK.
Matt Lemke: And– we actually have four flight computers and we can get into why we have four because it’s due to radiation, but even within one flight computer, it’s really two microprocessors running together. And so each microprocessor is executing the same code and then it compares its outputs and they should be identical. So, one nice thing about radiation is it’s not like vibration that affects the whole spacecraft at once. It’s a, you know, very miniscule particle or wave– I’m not even sure what the right physics term is– but it comes through and it only hits one thing, right, it’s going to hit one piece of silicon, one transistor at a time, and mess it up. And so the chances of the same piece of radiation going through both of these processors at the same time and corrupting the same bit so that when the two processors compare their answers that they’re identical, that’s just really– we call that inconsequential. It’s not going to happen.
Host: OK.
Matt Lemke: So, if one of them gets an upset, the other one doesn’t, it compares its answer, and then we do what we call fail silent. We get a miscompare and we say something’s wrong, fail silent, I’m not going to do anything more. And then you depend on the other computers to take over for you. Now, when that happens, you can reset that computer and get going again.
Host: OK.
Matt Lemke: And so that’s a key to us is being able to reset the computer and go again. So, when radiation affects electronics, you can have a few different effects. One is it just sort of degrades performance a little bit. It doesn’t change a one to a zero, but over time– so, if you’re doing a long, you know, 10-year mission to Jupiter with heavy radiation, your parts may degrade and fail. We’re not really worried about on that Orion. We’re worried about those bit flips. And when the bit flips, it can be a– just a momentary error, it flips, and then it comes back and it’s correct, or it can be permanent. Like I burned out that transistor and it’s never going to work again. And so that’s how we do our radiation testing is to make sure that the failure modes in the electronics we fly are those that can be reset by a power cycle. If it’s going to burn something up, we reject that piece of hardware and we look for a different implementation so we don’t have those problems. So, everything we’re talking about is something you can fix with a power cycle, worst case.
Host: Wow. So, to defend against radiation, you won’t put a system on Orion that has the potential to burn out due to radiation.
Matt Lemke: Right.
Host: Wow.
Matt Lemke: Or at least we won’t put it on Orion in a flight-critical location. Right?
Host: OK. You’ve got to be selective.
Matt Lemke: Our flight computer, our propulsion system, our life support system. We’re going to fly some laptops for crew use.
Host: OK.
Matt Lemke: Email, videoconferencing, that sort of thing. Those might burn up and we’ll just accept that risk, but not on our flight computer. Not on the critical hardware that’s going to fly the spacecraft.
Host: Yeah. That’s where you need those redundancies, the computers checking computers and then back up computers to check those computers.
Matt Lemke: Right.
Host: OK. I see that.
Matt Lemke: And that’s actually why we have four computers. We looked at the environment in space. We looked at solar flares, solar particle events, we looked at the Van Allen belt. We looked at how much radiation there is and we predict what our upset rate is. To buy electronics that doesn’t upset, you know, that’s designed for radiation, extremely expensive.
Host: Yeah.
Matt Lemke: Right? If we built our spacecraft that way, it would be using technology that was very old, it wouldn’t be very capable. So, we have to find that right balance between how often it upsets, how bad it upsets, and what we can afford. So, our flight computers, when we predict it, worst case, there’s a one in three chance during a mission that a flight computer can mess up. So, that’s a pretty high–
Host: That’s significant.
Matt Lemke: That’s significant.
Host: Yeah.
Matt Lemke: Right. So, if– what happens if that happens?
Host: OK.
Matt Lemke: It upsets. It takes about 22 seconds to reboot and be going again. OK? So, one in three chance that a computer will fail and takes 22 seconds to recover. So, we have a backup computer. We put a second one on. And then you look at what is the chances of both of those computers getting an independent radiation hit during the same 22-second window. Right? Because if one goes down but the other one’s flying, you’re good. The problem is if both of them go down.
Host: Right.
Matt Lemke: So, we looked at that and the chances were one in 89 that both could in the worst-case radiation environment, one in 89 it could go down. We said that’s still not good enough. So, we had three computers. We said that wasn’t quite good enough. We had four computers and you say there’s a one in a 150,000 chance that all four could go down in the same 22-second window. We finally said, OK, that’s how many flight computers we’re going to fly.
Host: OK.
Matt Lemke: And that’s, once again, worst-case.
Host: Yeah.
Matt Lemke: What we really expect is that to be more like one in a million missions that could really happen.
Host: Right. Still, I mean– wow, that’s– that’s a big buffer. [laughter] I’m trying– I mean– but the logic that you said is critical, though. It’s– these are– that’s the– this is the thing that’s flying you, you know? You don’t want that thing to go down. So, it comes to– it sounds like it’s a team coming together and assessing what they’re comfortable with and the comfort level considering cost, considering redundancy, it came down to four.
Matt Lemke: It came down to four.
Host: OK.
Matt Lemke: And then that still wasn’t good enough–
Host: What? Really?
Matt Lemke: So, we have a backup flight system. So, those first four computers are all doing the exact same software, flying the vehicle, and then you worry in addition to radiation, what if you had a common-cause software failure? So, there was a bug in your code that you didn’t catch in ground testing that could cause all four to go down at once. We have a whole other computer that does backup flight software. So, written separately, wouldn’t have those same flaws.
Host: Huh.
Matt Lemke: So, we really have those four plus another one in the case of a radiation event to keep us safe.
Host: Wow. OK. Yeah, I think I would feel pretty safe flying on that thing.
Matt Lemke: And that’s the key. And what we use is– like you say, we balance all those different parameters and radiation is just one parameter we worry about, right? So, we manage the whole spacecraft based on a loss of crew probability. We have a requirement that says we want one in 240 missions is our chance of actually losing the crew due to something we know about, whether it be the parachutes going out, radiation hitting the vehicle, whatever. So, when we accumulate all that risk, we want to minimize that loss of crew probability. And that’s what we use to drive where we put our money and where we put our effort onto the vehicle.
Host: OK.
Matt Lemke: And, fortunately, radiation is something we can do something about, right? We can add redundancy.
Host: Right.
Matt Lemke: We can do all of that. And so right now radiation is not one of those top program drivers to the risk of losing a crew.
Host: Huh.
Matt Lemke: Because we could– there’s something we can do about it.
Host: Yeah.
Matt Lemke: You know, a heat shield, you have one.
Host: Right.
Matt Lemke: The parachutes, you have many, but three main ones.
Host: Yeah.
Matt Lemke: And you could lose one of those and still land safely.
Host: Right.
Matt Lemke: But you can’t lose two. So, some of those things, we want to be the ones that drive– the things we can’t do anything about, we want them to drive our risk, not things we can do something about.
Host: Yeah. Because you have these redundancies because you honestly don’t know when radiation is going to hit exactly that computer and which one and how many.
Matt Lemke: Right.
Host: Wow. OK. So, that’s– when it comes to radiation shielding, the way that you are shielding the spacecraft is through redundancy. That’s really–
Matt Lemke: I’ve never thought of it that way before. That’s not what shielding means, but that’s how we handle the probability of the radiation happening is through redundancy. Shielding is very difficult.
Host: Yeah.
Matt Lemke: Right?
Host: Yeah.
Matt Lemke: Space station has, you know, a certain amount of aluminum on it. Orion has, you know, aluminum, titanium, some back shells. But these particles are so energetic, they’re just going to go through. So, we don’t get a lot of physical shielding from anything like that. You know, if you had a space craft that was surrounded by a foot of water, like a water bladder, that would really help reduce your radiation risk.
Host: Yeah.
Matt Lemke: Kind of heavy and not very practical.
Host: Right. Yeah. That’s a lot of water.
Matt Lemke: Right. So, basically we do do what you said is we make sure our redundancy is there in order to accomplish a mission safely.
Host: So, that’s– when it comes to radiation, you know, we’re talking a lot about software and the computers that are controlling. What else can radiation affect on Orion? What else can it affect– can it– can it blast through some of the materials themselves? Can it– can it burn out wires? Can it affect the power source? Can it do any of that?
Matt Lemke: Yeah, radiation can affect anything, but remember these are extremely small particles, so it’s not like a bullet blasting through something. It’s at the atomic level.
Host: Right.
Matt Lemke: And so it really doesn’t affect a piece of wire.
Host: Yeah.
Matt Lemke: Right? So–
Host: So, even if it does, this is not something that you need to worry about when you’re designing a spacecraft.
Matt Lemke: Right. We don’t worry about that piece of it.
Host: Yeah.
Matt Lemke: That’s where you get into some of the more long-term effects of radiation of– a board just slowly degrades. Right? So, it hits this molecule and throws it out of the– how it’s supposed to operate. Then it hits another one and eventually your transistors just start not working right. So, they’re– that’s the time duration aspect to radiation.
Host: OK.
Matt Lemke: Which on, you know, this particular mission we don’t have to worry about.
Host: Yeah.
Matt Lemke: You know, a Mars vehicle, something that’s going to be out there maybe for a couple years, we’ll have to start worrying about what we call total dose effects.
Host: Right. Yeah. You’ll have to watch those computers pretty carefully, right?
Matt Lemke: Watch those computers– and really it’s not even just computers, that’s what we’ve emphasized so far–
Host: Sure.
Matt Lemke: But your radio.
Host: Oh, yeah.
Matt Lemke: It’s really a computer these days, right? That’s all digital. Your displays are digital. The life support, the, you know, the controllers for the pumps and the fans. Those are all digital. So, all of those also have the opportunity to be affected.
Host: Huh. OK. I’m thinking about testing. You know? You’re talking about– you’re thinking what could happen, you have these redundancies. Do you have some sort of– I’m imagining a radiation room where you put the computers in and you start blasting away and let’s see what happens and let’s see if the computers can still run with all this radiation happening?
Matt Lemke: Right. So, the best way to do that is to put it in space and test it.
Host: There you go.
Matt Lemke: Because you need these very, very energetic particles. So, we try and do the best we can on Earth. We go to things like Texas A&M University has a cyclotron where it can shoot heavy ions at you and you can control which heavy ions it shoots and how many it shoots per unit time. But they don’t have the same energy as what you can get from the sun.
Host: Yeah.
Matt Lemke: And so, they don’t have the ability to penetrate like all the way through your computer and the beams are very small. So, it’s not possible to take your whole flight computer and put it in a radiation chamber or a radiation beam and get a test all at once. We have to go to space for that. What we can do is test each individual component. So, we can test the processor chip and the memory chip. We take the lids off, we expose the inner silicon to the radiation beam, and then we– we’re running test software that looks for errors.
Host: OK.
Matt Lemke: Right? And so that’s how we know if a part burns up. Right? We put it in the radiation beam and it stops working forever and a power cycle doesn’t fix it. We know this is not a part we want to fly.
Host: Yep. Don’t use that one.
Matt Lemke: We put it in a beam and it does nothing but upset constantly, also not good.
Host: Right.
Matt Lemke: So, we find the right mix between what parts are available, how often they upset. And then our radiation physics people take the cross-section of the beam, the cross-section of the part, how often it upsets, and they give us an on-orbit prediction for what’s going to happen.
Host: OK. So, I guess really what you’re looking forward to now is EM-1, the first flight of the Orion out towards the moon, around the moon, right? That’s where it’s really going to hit some of that– those high energy radiation that you’re talking about.
Matt Lemke: Right. When we flew EFT-1, the flight test–
Host: Yeah.
Matt Lemke: We actually went up 3000 miles, so we went out through the Van Allen radiation belts and back in. So, that was a good first test.
Host: How’d it do?
Matt Lemke: Of how we did in radiation– and we did very well.
Host: OK.
Matt Lemke: That was a very short mission, you know, three and a half, four hours long.
Host: Yeah.
Matt Lemke: But it did get up through the belts. We didn’t have upsets, we didn’t see that, but that’s more of a statistical thing, right? It could have happened, could have happened multiple times, and we would have said our models are still good. So, now we start talking about a mission to the moon.
Host: Yeah.
Matt Lemke: You know, 14 to 40 days, that sort of duration, and your chances go up.
Host: Right.
Matt Lemke: Of having these events happen.
Host: OK. So, that’s– yeah, that will reveal a lot on how these– this avionic software is.
Matt Lemke: Right.
Host: Can we– when I’m thinking about a spacecraft, you know, we’re talking about it going to the moon, right? I’m imagining– especially when it comes to software and processing capability, we’ve come a long way in that field when it comes to the last time we took a human-rated vehicle to the moon and that was the Apollo capsule, right?
Matt Lemke: Right.
Host: What are some of the differences there?
Matt Lemke: It’s really about computer control of the vehicle.
Host: OK.
Matt Lemke: So, when Apollo went, things like the life support system were not computer controlled. Those were all– we call them analog controls. They were just, you know, thermostats and different kinds of electronics to control each individual subsystem. The difficulty with that is when you find a problem, you want it to operate differently. There’s no easy way to fix it. Right? You have to redesign the hardware, rebuilt it, retest it. So, there’s a big advantage to making the spacecraft lighter, more capable, easier to change, and easier to maintain, so let the computers run everything. And now when you find a flaw, all you have to do is change a few lines of code– and I say all you have to do– you’ve got to test it, you’ve got to make sure it works.
Host: Sure.
Matt Lemke: But you can do it without affecting the hardware and you can do it on a much more efficient schedule.
Host: OK. Yeah. So, we’ve definitely come a long way in that–
Matt Lemke: We’ve come a long way and it then gives you flexibility for how you want to operate your spacecraft too. Right? So, now you can do things you never were able to do before. You can go anywhere. You know, Orion’s going to be able to go anywhere around the moon and land back anywhere on Earth, versus Apollo had a very tight corridor of where it could go. So, we have a lot more capability, a lot more changes we can make.
Host: Yeah. Wow. One of the things I’m thinking of is– thinking about the Apollo missions is how short they were. You know? You’re only talking about a couple days. But now with Orion, after the first couple test missions, EM-1, EM-2, we’re talking about a gateway around the moon. We’re talking about long duration stay and sustainability around the moon. That requires length of time and that was one of the things you touched on. So, how do these tests for radiation and redundancies test out over time? Is that something that we still need to discover?
Matt Lemke: We include that in all our calculations. Right? So, the longer you go, if I tell you there’s a one in 10 chance of getting an upset today–
Host: Yeah.
Matt Lemke: And tomorrow and the next day, as you add up the days, your probability goes up that something will happen.
Host: OK.
Matt Lemke: And so the duration– since we’re not worried about total ionizing dose effects, we’re not going to be there that long, it’s just a matter of it’s more and more likely you’re going to see events the longer you’re there.
Host: Of course.
Matt Lemke: Right? And the other risk you take is solar flares. Right? They’re infrequent. When they go, they may not come and get you, they may go some different direction. We have some prediction capability on solar flares and some warning. So, you know, you wouldn’t launch a spacecraft knowing a solar flare was on the way.
Host: Yeah.
Matt Lemke: But you start extending the length of the mission and you start increasing your chance of getting a solar flare while you’re there.
Host: Huh.
Matt Lemke: And so, you know, that’s part of our driving case. That’s what makes our worst-case radiation prediction is we’re depending on a twice the worst-case solar flare we’ve ever recorded. So, back in 1989 there was one, they recorded its magnitude, it’s the biggest we’ve seen. We say, well, what happens if it’s twice as big? That’s what we’re designing for.
Host: Wow.
Matt Lemke: Now, we don’t want one of those to hit.
Host: No.
Matt Lemke: And from your– when you were talking about the effects on humans, you don’t want that to hit.
Host: Definitely not.
Matt Lemke: Right? So, there is a place inside Orion, I don’t know if you talked about it before, but it’s the most-shielded place. Even though the aluminum doesn’t do a lot, the crew can get into the most shielded place. And while they’re stuck in there, you’d like your computers to keep working.
Host: Oh, of course. Of course.
Matt Lemke: Right?
Host: Yeah, because even if they do survive, you don’t want– you still want to be able to fly home.
Matt Lemke: That’s exactly right.
Host: Very, very important. Now, we’re talking about these redundancies, we’re talking about being out in the moon for longer periods of time and being able to deal with that over time. I’m thinking about just besides the technical component, the computer component, just sustaining the vehicle– the power. Is that something we have to be concerned about with radiation? Or, actually, why don’t we just back up even further, how does that even work?
Matt Lemke: So, we have batteries.
Host: OK.
Matt Lemke: Right? Keep our spacecraft going all the way from prelaunch, you know, from the time when we lose ground power all the way until we get on orbit and we can unfurl solar arrays. So, there’s four solar arrays on the surface module of the Orion. So, I don’t know if you’re familiar with the Orion capsule, but it’s got a crew module, which is a place where the crew stays the entire mission.
Host: OK.
Matt Lemke: Then we have a service module below it that contains our power, our extra oxygen, water, fuel, propulsion systems.
Host: OK.
Matt Lemke: That keep the vehicle going throughout the mission.
Host: Yeah.
Matt Lemke: And then we jettison that service module right before reentry.
Host: Yeah.
Matt Lemke: And then just the crew module comes home.
Host: OK.
Matt Lemke: And that service module we call the European service module because it’s being built by ESA and their prime contractor Airbus over in Europe. So, it’s an international program.
Host: Yeah.
Matt Lemke: Part of that service module is four solar array wings.
Host: OK.
Matt Lemke: So, these are four rectangular wings that unfold once we get on orbit, about 19 meters in diameter, so they unfurl, they’re pretty big; 15,000 solar cells, individual solar cells on there, and they generate all the power for the spacecraft.
Host: Nice.
Matt Lemke: And so as long as we’re in a place where the solar arrays are out and we have access to the sun– like we’re not behind the moon– we’re generating power and keeping our batteries topped off.
Host: OK.
Matt Lemke: Now, when we go behind the moon or we’re in a shadow, then our batteries take over. And then, of course, when the batteries– when we jettison the service module to come home, now we’re on battery power as well.
Host: Yeah.
Matt Lemke: So, those batteries have to be big enough to get us back down through the atmosphere and land in the water and then keep the crew safe for up to 24 hours after they’re in the water.
Host: OK. Does this– I’m trying to– I remember seeing an animation about the EM-1 and I think there was– there was one more stage of the SLS, I think, itself that got– that did the translunar injection. And then once it was on the way to the moon, that’s when the solar arrays came out. Is that right? Is that how–
Matt Lemke: We actually put them out ahead of time.
Host: Oh, you do?
Matt Lemke: So, we want to check out our spacecraft and make sure it’s good and healthy before we do that burn to go to the moon.
Host: Oh, that makes sense.
Matt Lemke: Right? And then of course if for some reason the solar arrays can’t come out, if the mechanism doesn’t work, they don’t unfurl right, if there’s some problem, we need to still have enough battery power to do an abort even though we’re in space and get back down while we still have battery power.
Host: OK. So, while you’re still within the vicinity of the Earth, you’re checking out a lot of these systems because once that translunar injection burn goes, you want to make sure everything’s working.
Matt Lemke: You want to make sure it’s working. And then you have to be careful. Now you’ve put these big wings out and you’re going to fire a big rocket from SLS, we call it the ICPS, interim cryogenic propulsion stage– I think I got that right.
Host: All right.
Matt Lemke: But that’s a lot of thrust and a lot of force on the spacecraft. So, we have to put the solar arrays in a safe altitude so that the forces of the rocket engine firing don’t break them.
Host: OK. Is there redundant systems there for power to make sure that we have enough to go in case something were to happen?
Matt Lemke: Yeah. So, the whole vehicle has to be at least one failure tolerant.
Host: OK.
Matt Lemke: Right? So, we have four batteries on our vehicle.
Host: Four. All right.
Matt Lemke: Right, but what if one of them failed? So, we have to make sure that if any component– any one component on the vehicle were to fail, whether it be a flight computer, whether it be a battery, a solar array, that we can still complete the mission safely and get the crew back home.
Host: OK. And that– and I’m guessing the four solar arrays that are kind of pointing out, these four rectangular solar arrays, there’s redundancies there too. For whatever reason, this one’s not getting enough power, but you still have the other ones.
Matt Lemke: If this one doesn’t come out, a wire breaks, the articulation mechanism that lets it point to the sun, if that’s not working right for whatever reason, if one of them fails, we can still finish the mission.
Host: So, how are we working closely with the European Space Agency to oversee that everything’s going to work, we have enough redundancies that the crew module is going to be able to talk to European service module and stuff like that?
Matt Lemke: It’s really– you know, at the working level, it’s just like working with another NASA center or working with another contractor.
Host: Cool.
Matt Lemke: Right? So, we work with our Airbus counterparts, we work requirements, we work designs, we have design reviews, we do testing, that sort of thing. It’s a little different contractual arrangement because they’re our partners.
Host: Yeah.
Matt Lemke: Right? We’re not paying them to do this, it’s a barter agreement between the countries and so the Europeans are building the service module and they bring it over here and we integrate it together and make a whole spacecraft out of it. But really half that spacecraft is being built by the Europeans.
Host: Yeah. So, the power– I– there was one podcast we did a while back, I’m trying to remember the episode number, I think 28. We talked with Jessica Vos about living in the Orion spacecraft for what could be sustained up to three weeks. The mission profiles would be slightly less than that, but everything’s going to work for three weeks. So, is that what the power systems, the avionic systems, are those all meant for three weeks or are they designed to be even longer, thinking about, you know, gateway and missions beyond?
Matt Lemke: Right, they were originally designed for 240 days.
Host: All right!
Matt Lemke: Because back when we started the program, we had the whole concept of– you know, we didn’t have necessarily a destination, so there’s a concept of going to the moon with a lander and then letting the crew module stay in orbit around the moon while the whole crew went down to the surface. And so it may stay therefore up to 240-day missions. So, we really designed it with that in mind originally. The mission’s changed a number of times since then.
Host: Yeah.
Matt Lemke: But really, especially like for power, as long as you’ve got the sun, solar arrays will be charging the batteries, you’ll be good there. We’ll start getting limited on things like water and fuel as you start going longer.
Host: Yeah. I want to go back to the avionic systems, the software for a little bit. We talked about how radiation affects it and how we have these redundancies to think about radiation, but what is actually going on in those systems? What is the avionic systems doing for the vehicle?
Matt Lemke: The avionic systems, they run on a schedule. So, 40 times a second our software executes.
Host: Huh.
Matt Lemke: And it controls all the subsystems. It’s doing communications, it’s getting commands from the ground and sending data to the ground. It is controlling the pumps on the cooling system to keep ammonia flowing and keep the electronics cool. It’s running the fans that keep the air circulating. It is– when we need to fire a thruster to keep our attitude, it’s making those corrections. When it needs to move the solar arrays to point to the sun, it’s doing that. It’s keeping track of where you are, how fast you’re going, you know, what rates are you generating on the vehicle, so where are you at today. It’s navigating. It’s doing guidance for when do I need to do a burn to get there. So, it really handles every subsystem on the vehicle at a periodic rate, just making sure they’re all doing what they’re supposed to be doing.
Host: Yeah. It’s the backbone for all of these different things that make the whole vehicle work. Make it all fly.
Matt Lemke: Exactly. We are very– if you look at Apollo or even shuttle, you saw lots and lots of switches in the cockpit.
Host: Yeah.
Matt Lemke: Lots of manual control. We have very, very few switches in Orion. I forget the exact number, but it’s really only there in an emergency situation or when your computers aren’t quite doing what you want them to do. So, like you have switches to be able to turn off the computers. You have switches to be able to deploy the parachutes in the event the computers don’t work. But, in general, the switches are just there to control power and do some things in case the computer is not doing what it’s supposed to be doing. Everything else is computer controlled.
Host: Wow. All right. So, when the crew is quote on quote flying Orion, really most of the flying is being done by all the computers because they’re executing at– like you said before, on this timeline, on this schedule.
Matt Lemke: Right. The computers can fly the entire mission autonomously. Right? So, we’re flying EM-1 and that will be without crews to check out all these systems and it’ll fly that entire mission autonomously with people in FOD watching it from mission control, seeing if any new commands need to be sent, any corrections to be made.
Host: Right.
Matt Lemke: When the crew is there, they’re sitting in front of a console that has a few switches I mentioned and there’s also three display units. And on those three display units, they can have up to six displays displayed at once where they can get insight into what’s happening on the vehicle.
Host: Huh.
Matt Lemke: So, I think of it as they’re following along with the flight, they’re following along with the software. They have a cursor control device, think of it as a mouse.
Host: Yeah.
Matt Lemke: It’s actually by their left hand– where they can interact with those displays even during ascent. And so if they do need to take control, if they need to make an adjustment, they can do that. They also have hand controllers where if they need to fly something or take over, they can do that as well. But, in general, everything they need to be able to do is done automated and they can just interact with that automated software.
Host: Wow. Yeah, so there– when the crew is flying the Orion, when it comes to the– the avionics software is doing most of it and their job is to just make sure that the avionics is doing its job. That it’s talking to all the right subsystems, that the vehicle is flying normally, and they are trained– I’m guessing– they are trained specifically for watching to make sure if something does go wrong, they know what to do.
Matt Lemke: They know what to do exactly.
Host: Yeah.
Matt Lemke: So, they know their contingency procedures. We even have a lot of the normal procedures they’ll have to execute are all electronic now. So, they’ll actually just come up on the display and it’ll tell them do this step and it’ll actually bring up the screen where they can do that step.
Host: Yeah.
Matt Lemke: Right? And then the crew just says, OK, yeah, it did what it was supposed to do, I like the values everywhere else, enter, go ahead and do it.
Host: OK.
Matt Lemke: So, it tries to step them through– so rather than flying volumes and volumes of paper manual that are heavy and take up area inside the crew capsule, we do that electronically.
Host: All right. So, that’s– I guess that component will be tested on EM-2 when we finally put crew on to make sure that– well, we’ll test the vehicle first and make sure that it’s doing its thing without the crew, but then the crew will be able to understand this whole process and make decisions if something were to go wrong.
Matt Lemke: Right. Now, we’re not flying any displays or hand controllers on EM-1.
Host: Right.
Matt Lemke: So, that’s being done in the labs, but they’re already coming up with all those displays. And actually a kind of interesting thing that’s happening on Orion is the crew is actually developing the displays.
Host: Cool.
Matt Lemke: So, rather than the Orion program, the users, the people that need to interact with it– there’s a rapid prototyping lab in building four south and actually the crew is leading that development effort for what they want the displays to look like, how they want to interact with them, and making sure it’s going to work right. And then once a crew gets each display correct, then they turn that over to Lockheed Martin to go turn into flight software. But they get it all right first and tested first.
Host: Yeah. Yeah. But then, yeah, they actually put the right stuff, the procedures, the processes that the crew understands and recommends into to the flight–
Matt Lemke: Right.
Host: OK.
Matt Lemke: And then they even go so far as to run crew evaluations where they’ll have a mock cockpit and they’ll run crew members– various crew members– through all the different procedures, all the different displays, get their comments, get their feedback, and that way before we spend a lot of money building flight software, we already know we have a product that’s going to be good.
Host: Yeah.
Matt Lemke: Versus a bunch of engineers and people thinking what the crew might like, implementing it in flight software, and then when the crew sees it and says, well, I’d like something different, we would turn around and say, well, that’s too expensive to change. So, we’re kind of turning that process around to make it more affordable by having the crew give their input first.
Host: OK.
Matt Lemke: And get it right before we develop it.
Host: There you go. And then– well, first– like I said before, we have to test it un-crewed first on EM-1 before they actually interact with it, but, thinking along those lines, you know, we’re in the process of actually building the hardware for that mission.
Matt Lemke: Yes.
Host: Where are we in the avionics and power world? How far are we along? How many steps do we have until we have all the components ready and integrated into that vehicle?
Matt Lemke: So, we’ve just finished getting all the components in and tested for EM-1.
Host: OK.
Matt Lemke: Right? So, the vehicle from an avionics perspective, just a couple little units left to install in the vehicle. And our intention is to power on the spacecraft for EM-2 in September of 2020.
Host: All right.
Matt Lemke: So, that’s when we have all our components built, delivered to the cape, installed in the vehicle, and ready to turn it on– we call it initial power on– and make the vehicle start acting like a spacecraft. And so we’re actually already started building some components. Some of the ones that take the longest amount of time to build and test we’ve already started on.
Host: Cool.
Matt Lemke: And then we just have a whole schedule over the next couple years to turn them on as they need to be turned on so they’re ready.
Host: There you go. Is that being done here at the Johnson Space Center?
Matt Lemke: The Orion program office is here, but our main contractor, Lockheed Martin, is in Denver.
Host: I see.
Matt Lemke: And then really it’s being built across the country.
Host: Uh-huh.
Matt Lemke: I think it’s every state, including Puerto Rico, has some Orion business and avionics is spread really in many locations.
Host: Oh, really?
Matt Lemke: Honeywell, UTAS, Ball, GD, SEAKR– and I’m sure I’m missing many others vendors– Harris– across the country are building the components, testing them, and then delivering it to integration labs in Denver to test them out as a system and then to KSC to test them out on the vehicle.
Host: All right. Yeah, so this is a massive effort. Definitely a lot of work to be done over these next few years.
Matt Lemke: Just a massive effort. Takes the whole country to build it– and then when you add in the Europeans–
Host: Yeah.
Matt Lemke: I think there’s quite a few countries in Europe as well participating when they’re building that service module.
Host: Awesome. Are you going to see the whole thing through? Are– is that your ambition to–
Matt Lemke: That is definitely my ambition. I definitely want to see people fly again.
Host: Yeah.
Matt Lemke: Right? So, at least want to continue this sort of job through EM-2.
Host: Right.
Matt Lemke: And then we have lots of new challenges after EM-2, even.
Host: Yeah.
Matt Lemke: Because after that then we want to get into– we call it a production mode– where we can start building spacecraft a little bit quicker, once we’ve tested them out, and that’ll start freeing up resources to build that gateway you were talking about earlier.
Host: Right. There you go. A lot of work to be done, but it sounds like there’s so much– so much to this. Matt, thank you so much for coming on the podcast and describing this wonderful world of what I thought was radiation shielding. I imagined sort of like, you know, one of those movie shields, but really it’s the redundancies and there’s a lot of logic to it on how everything works. So, I appreciate you coming on and describing that for us today.
Matt Lemke: It’s my pleasure. It’s fun to come and do things like this and get away from the normal drudgery of, you know, daily business management and talk about what we’re really trying to accomplish.
Host: Wonderful. Really appreciate your time, thank you.
Matt Lemke: Thank you.
[ Music ]
Host: Hey, thanks for sticking around. So, today we talked with Matt Lemke about radiation shielding and how really redundant systems make the whole thing work. If you want to learn more about radiation, you can check out Episodes 57 and 64. You can check out one of our many episodes about Orion. We have a nice overview on Episode 17. We talk about how the crew will operate on missions up to three weeks for Episode 28, and then we take a ride inside the capsule for Episode 35. Episodes 66 and 69 are ones we did recently on Orion’s heat shield and navigation systems. You can go to nasa.gov/orion to learn more about the vehicle itself. We’ve been going through some of these topics based on an article called The Top Five Technologies Needed for a Spacecraft to Survive Deep Space. Really interesting stuff. Gives a nice overview of some of the stuff that we are talking about in depth here on Houston We Have a Podcast. On social media, we are on the International Space Station, Orion, and NASA Johnson Space Center accounts.
On Facebook, Twitter, and Instagram, use the hashtag #AskNASA on your favorite platform to submit an idea for the show. Make sure to mention Houston We Have a Podcast. This episode was recorded on September 5th, 2018. Thanks to Alex Perryman, Bill Stafford, Pat Ryan, Laura Rochon, Rachel Kraft. Thanks again to Mr. Matt Lemke for coming on the show. We’ll be back next week.