Suggested Searches

The Heroes Behind the Heroes: Part 4

Season 1Episode 91May 10, 2019

In part four of this series, the team of scientists and technical staff turns 19,000 hours of digitized audio into transcripts, all of which can be accessed online. HWHAP Episode 91.

Heroes Behind the Heroes Part 4

Heroes Behind the Heroes Part 4

“Houston We Have a Podcast” is the official podcast of the NASA Johnson Space Center, the home of human spaceflight, stationed in Houston, Texas. We bring space right to you! On this podcast, you’ll learn from some of the brightest minds of America’s space agency as they discuss topics in engineering, science, technology and more. You’ll hear firsthand from astronauts what it’s like to launch atop a rocket, live in space and re-enter the Earth’s atmosphere. And you’ll listen in to the more human side of space as our guests tell stories of behind-the-scenes moments never heard before.

Episode 91 is part four of the Heroes Behind the Heroes series, focusing on how the team of scientists and technical staff turned 19,000 hours of digitized audio into transcripts, and how you can access both online yourself right now.

If you’re interested in hearing more of this historic audio from the Apollo 11 Mission Control Center check out the Explore Apollo website. For more of our Apollo Podcasts, including part one of this series, check out the “Houston We Have a Podcast: Apollo 50th Anniversary” webpage!

Houston, we have a podcast

Transcript

Pat Ryan (Host): Houston, We Have a Podcast. Welcome to the official podcast of the NASA Johnson Space Center, Episode 91, The Hero’s Behind the Hero’s Part 4. I’m Pat Ryan today we’re posting the last episode of our series telling the amazing story of the rescue of the audio from NASA’s Mission Control Center in Houston during the Apollo 11 mission that landed the first Americans on the moon. The earlier episodes recount the tale of Dr. John Hanson’s intent to use that audio for his academic research in communications and speech processing, and how he and his team ran into one hardware issue after another that threatened the whole project, but as Hansen said at the end of Episode 3, his teams determination to succeed was similar to NASA’s attitude when it was trying to meet President Kennedy’s goal of putting a man on the moon.

John Hansen: We set a goal and we may not have had all the answers at that instant when we set the goal, but people worked collaboratively together to insure that they’re going to achieve that, so this was something that we’ve tried to inspire the students that were all involved with this, with that mindset.

Host: When we left our hero’s last time, they had finished digitizing nineteen thousand hours of audio from a Apollo 11 and some other missions of the era, but their job wasn’t done, and so neither is our story. Let’s jump back in and find out what happens next. Here we go.

[Music]

Who can forget this classic opening line, “Wee, sleekit, cowrin’, tim’rous beastie”? Okay, maybe most people forgot it, but probably most people know this line, that’s just from a little bit further down in the Robert Burns poem To a Mouse, “The best laid schemes o’ mice an’ men gang aft agley” Well, John Hansen can attest to the truth of that statement.

[Music]

The University of Texas of Dallas Professor, who is the universities distinguished Chair in Telecommunications Engineering and the founder and Director of the Center for Robust Speech Systems, he had a plan, he and his team of students were going to salvage the audio recorded in NASA’s Mission Control Center in Houston during the first a Apollo moon landing and do a comprehensive study of the language used by a large group of people while they were collaborating to solve a problem. The researchers were going to advance speech and language technology by developing the mechanical systems that could better understand the jumble of human speech, as compared to a single voice. Their plan did not include solving a major engineering problem to get the only available piece of hardware that could possibly play the audio off of analog tapes, get that equipment into operating condition so that they can digitize these sounds of history. Completely off plan, they designed and fabricated brand new hardware components. And then they oversaw the playback of more than sixty audio tapes running some, fourteen hours each, just to get the raw material with which to do the science they came here to do in the first place.

[Music]

John Hansen: Yeah, we had nineteen thousand hours probably in the neighborhood of about twelve terabytes of data, and like anything, that’s a lot of data and if you say, okay, we’ve now digitized it, if I threw that over the fence at you and said, here’s the data, you’d say, well, that’s great, it’s too much, don’t know what to do with it, don’t know how to actually interpret it or use it in anyway, so that’s where—and fortunately, the process of actually building the solution and digitizing it, this was an engineering challenge, but it was something that was basically you could think of as, like, a task, something that had to be done. During this entire process we weren’t, kind of, just waiting around, waiting for audio to be available, we had students and researchers focused on developing the technologies to be ready when the audio had been digitized, and so those algorithms, the basic strategic is referred to as diarization. Normally we, think, well, if you’ve got the audio, just feed it through a speech recognition system and you look, for example at smartphones, you could look at Amazon Alexa or Apple Siri and say, well, I’ve got speech recognition, why can’t I take this audio and feed it through Alexa or feed it through, you know, Siri?

Host: Why couldn’t you.

John Hansen: And the answer is because NASA speaks in acronyms.

AFD: Science, AFD

Science: Go ahead.

AFD: Hey where are you getting your ESAP pointing data?

Science: I’m getting it from the CM

AFD: No I’m talking about the—does either the LEM or the CSM PSAP have a ESAP performance in it?

John Hansen re-wiring and installing newly designed and constructed 30-track read tape playback head.

John Hansen: And those acronyms are not there simply to, kind of, make things sound more technical, it’s because NASA was trying to accomplish, you know, the bulk of the command and control here through audio communication. So if you could say something, you know, in one or two seconds that may have taken three to five seconds to speak, well now you’ve opened up that channel for someone else to talk, and so acronyms were used to primarily to try and be more efficient. The problem is when you look at any commercial base speech recognition technology out there, none of them, absolutely none of them are set up for the type of NASA vocabulary that’s used, that’s one and two; virtually all speech technology that’s out there is intended for one person giving a voice command or saying something to a machine, in all the communication here, none of this was intended to be used for speech technology, it was people talking with each other to solve a problem, so it’s natural, spontaneous speech and the speaking style that we have here is dramatically different than if you’re actually speaking to Alexa or Siri, there’s lots of coarticulation effects and what that means is, you know, if you looked at printed material, if you saw the phrase, “cats an dogs each hate the other” well, if you listen to me carefully, I actually did not say what was actually printed, if it was written there, because I said, “cats an dogs” the arrested A N, not A N D so if I were actually saying it the way I should have been, it should have been cats and dogs, but we don’t speak that way, and so because of that, the speech recognition technology has to interpret from the audio what would be an appropriate text sequence, and so that requires a dictionary or a Lexicon and it also requires a very effective language model that predicts what word might come over a given word.

Host: Your ability to tell the—to give the computer a hint about the language that’s being used?

John Hansen: Exactly, yeah. So when I mention this diarization, well, when you look at any mission for Apollo, they’re typically six to eight days long, thirty channels, you also have to think every NASA mission specialist in the Mission Control Center, they’re not working the entire hundred and, you know, sixty eight or hundred seventy hours straight, right? So you’re going to have typically three or four people that are taking shifts, working an eight hour shift, maybe overlapping by one hour between

Host: At one of those consoles in the flight control room there are, there’s a team.

John Hansen: Yes.

Host: In rotation.

John Hansen: So that means that for an eight hour period we typically would have one person responsible for whatever that loop would represent, and so by the digitizing process, now we want to actually go through and do diarization. So that task first requires that we run something called, “Speech Activity Detection” we want to identify where there’s silence and when there’s speech; the reason for that is if there’s silence, you really want to turn off the speech recognizer. If no one is talking, you don’t want the recognizer listening in because it’ll start spitting out words if there’s some noise or some other distortion that might appear on that, so now you have spurious words popping up and that’s not a good thing. So speech activity detection is done first, after that, you’ve identified blocks which represent speech, and then you feed only those portions into speech recognition, maybe before you do the speech recognition you actually might do what’s called, “Speaker Clustering” and the reason for that is when you’re looking at a loop, you typically have two people talking to each other, so you really would like to actually identify the turns, if PAO is speaking—oh, let’s not—let’s say, flight director, so flight director is speaking and he’s trying to get information about flight dynamics or something like that.

FDO: Flight, it’s FDO

Flight Director: Go ahead

FDO: In the interest of science it would behoove us at this time to take a checkpoint.

Flight Director: Let me check our associates. EECOM?

EECOM: Stand by one.

Flight Director: How about you GNC whenever we get over this present stumbling block?

EECOM: Okay Flight, we’re go.

Flight Director: FDO?

FDO: We’ll get some cooperation.

John Hansen:So he asks a question and then that person responds, so there’s turn taking going on, so we want to identify is it speaker A, speaker B, speaker A, speaker B? And then after that when we run speech recognition we at least have an idea that there’s turns taking place here.

Host: So that it can tell the difference—that there are different persons?

John Hansen: Yes, that there are two people that are talking to each other, yes. So in order to build the speech recognizer, you’ve got to understand that we now need to migrate the technologies that we’ve worked on for more than twenty five years at our center, to something that would work for NASA speak, if you want to call it that. And so one of the things we needed to do is first to get a Lexicon, a dictionary of all the expresses that NASA would use and then build a language model. To do that, we went and we pulled every document about Apollo that we could find, books, news, releases, everything we could find, we pulled a total of 4.2 billion words in order to build our Lexicon and to build our language model. So the Lexicon, you know, the English language has about six hundred thousand words, okay? If you threw some of NASA’s expressions in there you might add a few more than six hundred thousand, I guess, but most vocabulary systems, when you’re doing speech recognition are typically a hundred thousand to maybe a hundred and forty, hundred fifty thousand words is typically what you would see. So we built up the dictionary, or Lexicon. What that Lexicon actually is it’s got the word that the person would have said and it’s has the actual sounds that would make up that word, those sounds are called phonemes.

Host: So that when it hears those sounds

John Hansen: It’s going to replace it with the text of the words.

Host: It knows what the word is.

John Hansen: Yes.

Flight Director:Ahh yeah, why don’t you just keep on going through it. I’m going around for T2 stay/no stay, all flight controllers. T2 stay/no stay. RETRO?

RETRO: Stay.

Flight Director:FDO?

Apollo 11 MOCR

FDO: Stay.

Flight Director:Guidance?

Guidance: Stay.

Flight Director:CONTROL?

CONTROL: Stay.

Flight Director:Telcom?

Telcom: Stay.

[ Unintelligible ]

Flight Director:EECOM?

EECOM: Stay.

Flight Director:Surgeon?

Surgeon: Stay.

Flight Director: Capcom we’re stay for T2.

Capcom: Eagle, Houston you’re stay for T2. Over.

Neil Armstrong:Roger, stay for T2. We thank you.

Capcom:Roger, sir.

[Music]

Host: Lakshmish Kaushik was working as a speech scientist in Montreal when he learned through academic circles about John Hansen’s project and was intrigued. He was interested because he knew Hansen’s reputation and recognized the research significance of the project and he was struck at the prospect of working with NASA, so he left his job and enrolled at the University Texas at Dallas as a PhD candidate working with Hansen on the a Apollo tapes project. It was his job to build the speech recognition system that could turn the digitized audio of the Apollo Mission Control Center Team into text.

Lakshmish Kaushik: Say, for example, if I say “okay eagle,” so it will try to understand, o-k e-g-le, it will try to separate it into its individual components and then try to identify the most basic phonetic composition of the audio. After it finds the most probable phonetic composition of the audio, these units are sent through the language models, which I described earlier, which will try to decode and pose the possible sentences and the words using the phonetic structure out of phonemes that have been decoded by the linguistic models.

Host: At that point, are you, like, playing one conversation at a time to it to put it together? Or do you already get to—are you at a point where you can play at the multiple conversations that were happening at the same time on different tracks and having it sort those out?

Lakshmish Kaushik: No, those things are done in uh, parallelly like, you know we have totally thirty channels in a single—in loop track, and then we have thirty separate audio files, once the system is built by the trans cable system which predicts the speech and then convert it into text, we had an enormous, huge cluster of computers which can convert speech to text parallelly, so we parallelly started this transcription system running across almost around like, running parallelly on fifty computers at the same time.

Host: Fifty?

Lakshmish Kaushik: So we had—

Host: Five zero?

Lakshmish Kaushik: Yes.

Host: Fifty computers?

Lakshmish Kaushik: Yes.

Host: Wow.

Lakshmish Kaushik: It’d be like something called a GP rack, which had almost around ten GPU’s which are very computationally, if we share, which can do millions and millions of computation every second and it had more than a hundred EPU processors, which can crunch the data at a blazing speed, so we used this kind of cluster to convert this speech to text. And mind it, we had nineteen thousand hours of data and this conversion is almost real time. So that means that, okay, if I give them seconds of data, it takes to almost around eight to nine seconds to convert it into text. So in order to convert nineteen thousand hours of data, we had to run more than fifty to a hundred parallel threads at the same time to transcribe the nineteen thousand hours of data, this conversion itself took almost around four to five months.

Host: Yup. That’s what he said; it took four to five months with fifty computers crunching data to generate transcripts of the Mission Control conversations with timestamps to indicate when things were said. That facilitated the researchers work with this trove of audio, just as it will help you when you go to the website where you can read the transcripts and listen to the audio yourself, I’ll give you the web address in a few minutes. It also turns out that the transcripts sped up the ability of the NASA Export Control Office to review the all the material to approve its release to the public.

Karen Walsemann: My name’s Karen Walsemann, I work on the JSC Export Services Team.

Host: Which is also known as Export Control. So what is Export Control and why does anything need to have its export controlled?

Karen Walsemann: That’s an excellent question and one that the JSC Export Services Team and myself and Headquarters and everybody in Export Control would really like to make known, it’s really a twofold thing. The first thing people think about is missiles, you know, you want to control technology so that people can’t get the technology that they have that we have that can then be turned on us and hurt us, missile technology. The other thing is to maintain our excellent superiority in some places where we really have the majority of the knowledge, like in engine design, and so we help when we get with Rocketdyne, Aerojet and these guys and they help us with our engines and provide us with engines, we protect that technology for them and we don’t let that stuff get out unless somebody actually has a need to see it and we have a requirement to send it to them. So we’re protecting, you know, U.S technology, as well as protecting our safety from, you know, bad people overseas.

Host: I guess that you’re trying to protect this information from being released either intentionally or accidentally.

Karen Walsemann: Yes; and a lot of people don’t realize that you and I sitting here, carrying on this discussion, I could release technology, or data into the public dominion that was not ever intended to be released into the public dominion.

Host: Your conversation–

Karen Walsemann: Right.

Host: Might release information.

Karen Walsemann: And so if we have this discussion and it contains ITAR or EAR controlled information, and then you put it out on the web, there it goes, it’s in the public.

Host: Okay. And those acronyms are for laws that are regulate, the release of such information, right?

Karen Walsemann: Those are the two laws, the ITAR is the International Traffic and Arms Regulations, and that’s controlled by the Department of State, they’re more military based. The EAR is the Export Administration Regulations, and that’s controlled by the Department of Commerce and they’re more business based, but there is a lot of technology and missile type technology that is in the EAR controlled by Commerce, space station, for example is controlled by the Department of Commerce. The Orion vehicle itself is controlled by the Department of Commerce.

Host: Karen Walsemann started working at NASA in 1989, during the Return to Flight Effort after the loss of Space Shuttle Challenger, she’s worked in the mission control back rooms on the Robotics Flight Control Team, she’s trained space shuttle astronauts, and designed simulators for astronaut training and has worked on the Orion Program, so she brought a wide range of knowledge of work done at the Johnson Space Center when they joined Export Control and her boss thought she was a natural for this assignment.

Karen Walsemann: So the Export Services Team was contacted and our Center Export Administrator Ari Blum, said, well, Karen can do that, she knows about older programs; and so I met with everybody and tried to get a handle on what was needed and required and we started exchanging information, and that’s how it started.

Host: At this point do you think that you’re looking at just reviewing historical information, or are you still concerned about technology? I’ll be it, fifty year old he technology.

Karen Walsemann: Yeah, it’s fifty year old technology, but it’s still how to maneuver in space, it’s still engines, you know, we’re firing engines and we’re guidance is a big deal, we’re using guidance and navigation and how to control where you’re going from point A, to point B in space, and those things are technologies that are controlled by Export Control. The thing I was looking for though was any very deep details of discussions that could be considered technical data and something we would not want to be transmitted out over the waves and into the public.

Host: Can you give me without spilling the technological beans; can you give me an example of what kind of a conversation you were thinking of that might be present?

Apollo 11 MOCR

Karen Walsemann: Yeah, the biggest thing you’re looking for is when there’s a problem because when you have a malfunction of some kind, then the engineers start talking in very deep detail about how the hardware that failed was designed, and what it was intended to do, and then what it ended up doing and that’s called anomaly resolution and many times, even when we get licenses to provide support to our international partners, we’re still not allowed to them with anomaly resolution and because that will then teach them how to fix their own problems when they have a problem with their hardware in the future, and so you kind of look for those kind of in depth discussions for an anomaly resolution or design discussions on how a piece of hardware was designed, but in verbal conversations it’s very hard to get all of that through without having the hardcopy data that they’re looking at to go through it.

Host: When you say it’s hard in a verbal conversation to get that information through, the through part is throwing me, are you talking about to get any cleared through an Export Control review or to get it conveyed to the person to whom you’re speaking?

Karen Walsemann: Conveyed to the person to whom you’re speaking without having a written document in front of them as well as reference.

Host: That the two of you are looking at and, you know referencing?

Karen Walsemann: And then carrying on a discussion about.

Host: I see.

Karen Walsemann: That’s the better way to put that.

Host: So merely, a conversation between these two people—some two people without, absent the document they are referencing to is less likely to contain anything that you would that you don’t want to be released?

Karen Walsemann: That’s correct, and sometimes that’s not the case, you know, they’re really good at explaining things, but the majority of the time they’re just talking numbers, and unless you’re really good at what you’re listening to, it’s just a lot of numbers.

Host: Or maybe if you don’t— if you already have some detailed knowledge, you would understand what those numbers mean.

Karen Walsemann: And there is a lot of detailed knowledge about the Apollo program already out there.

Host: And because it’s been out for so long, right?

Karen Walsemann: Right; and we didn’t have the controls at the end of the a Apollo program on technology that we do today, the details, and so everybody was all excited about the technology that we had back in the early 70s when everything was winding down and they were all excited about building rockets and showing how the engines worked and all those good things, and so there’s just a lot of technical data out on the Internet, it’s easy to find.

Host: Hansen and some of his students had described to me that they had digitized nineteen thousand hours of conversations, which then is coming to your desk? What kind of plan do you come up with to try to wade through that mountain of material?

Karen Walsemann: My plan was simple, I chose what I considered the three biggies, they started with Apollo 11, of course, what an exciting thing to listen to over and over again. I mean, I listened to the entire mission of Apollo 11, I pick out bits and pieces of other missions that, where I knew they were an anomalies and where they were in the—would be in the recordings to listen to those, and then based upon that level of detail, we extrapolated an Export Control decision for the entire set without listening to the entire set.

Host: Okay. You didn’t have to listen to all nineteen thousand hours?

Karen Walsemann: I think my child would have to had carried on I my legacy for doing that too, I don’t think I have nineteen thousand hours in my world [laughter]

Host: Did you—did either Hansen and his team or the technical people who helped digitized that in the Public Affairs Office, did they give you some assistance or some means of helping get through the material?

Karen Walsemann: Hansen’s team broke it all down into snippets of recordings and then they ran it through a piece of software that would listen to the recording and write down a transcript of it.

John Hansen: In order for NASA Export Control to actually go through and approve the release of this, when we first started talking NASA Export Control, their original approach was, well, we need to listen to this, and we said, well, okay but this is nineteen thousand hours, you can’t listen to nineteen thousand hours. So how do we speed this up? So that was a new problem we had—we looked at. So we decided well, let’s try to streamline this, we were motivated to try and make it easier for the Export Control Specialist to be able to go through this quickly. So we set up a process where once the transcripts were generated and the audio is there, we automatically populated an XL file and in the XL file we have one row that basically just has the text transcript of what’s in that fifteen minute block, and so the person from Export Control could just read what they see in that text block, if they’re unsure of what was being said or they have some concern, in the next cell there’s a direct link, and they can just click and when they click that link, there’s a system called Transcriber, LBC Transcriber tool that pops up, has the wave form, has the text transcript, and all of the links that put the texts directly with the audio, so now they can just click this little highlighted spot that they’re looking at in the XL file and just listen to that, you know, one to three second, five second block. So it really sped up the export control process.

In one month NASA Export Control was able to approve one thousand hours of audio, and otherwise they would have had to listen to all of that, so they listened to a fraction of it. In addition to that, we were really appreciative that NASA Export Control also went and started doing a lot of the, I would say, the improvements of the transcripts because when you run speech recognition, it’s not going to be flawless, there’s going to be some errors and so the NASA Export Control Specialist actually made some text comments and corrections there, but also had a field there that said, well, this spot is particularly important for this reason, so she inserted some comments there that were really helpful for an archiving purpose.

Host: Were there things you found that you had to cut out that you said no, you can’t release this?

Karen Walsemann: It mostly had to do with personal information, you know, they call it PII, Personally Identifiable Information because people were handing out phone numbers, not like they’re valid anymore.

Host: Sure, they’re probably not.

Karen Walsemann: And I said, you probably really don’t want to do that.

Host: So in all of that, you didn’t really find anything substantial or technology releases that, you know, technological information that needed to be protected.

Karen Walsemann: Not really, no.

Host: Did you find anything in there that you found really interesting, really fascinating, things you didn’t know before?

Karen Walsemann: Yeah, that there’s a place in Clear Lake back in 1968-69 called the Chicken House and you can get a sandwich, a french fry and a Coke for a buck sixty five.

Host: Wow, we should go there for lunch now, huh?

Karen Walsemann: I don’t know where the Chicken House is, but every time they were going for lunch or dinner it was, there were only three options and it was the Chicken House and some German restaurant and a burger joint and that was it, that’s the only three they ever came up with.

Voice 1: Go ahead, Network.

Network: Hey, is that the one, the party, the splashdown party down at the Hofbrau Garden back in the back, Thursday?

Voice 1: Alright I understand they gonna block off NASA 1 at Webster.

Host: How do you think of this job, in comparison to the other kinds of jobs that you did, was this special? More interesting? More important?

Karen Walsemann: It was special and because I grew up in Titusville during the Apollo program.

Host: Oh.

Karen Walsemann: My dad was a contractor at the Kennedy Space Center at the time. So, to me, it was very near and dear to my heart to hear all of these things and see all of these things because I got to watch these launches from my backyard or sometimes from the bleachers outside of the Launch Control Center, and so to come back as an adult all these years later and listen to these men and a couple of women go through all of this, was fascinating, absolutely fascinating and my brother is an Apollo nut and so this—I would send him little emails going, you’re not going to believe it, when it’s released I can tell you.

[Laughter]

Voice 2: Powered flight angle 10 decimal 3822; that is a change. Powered flight burn time 444 seconds. Altitude at insertion 60 thousand feet. Velocity at insertion 5535 decimal zero.

Host: Karen Walsemann’s review of the mountain of material led to Export Control approval for public release of the Apollo 11 Mission Control tapes and transcripts and that meant two things; the first thing is John Hansen and his team could use it in their academic research and I’ll get Tatiana Korelsky from the National Science Foundation comment on the quality of that work.

[Music]

Apollo 11 MOCR

Tatiana Korelsky: It was a very interesting development which was never even anticipated before. They decided to use the technology to identify sentiment, to identify negative and positive statements because they were so certain exchanges in those Apollo tapes were so emotional, they were events happening in real time. So people were sometimes upset, sometimes were elated.

Voice 3:[Inaudible] for voice check.

Voice 2:Well good morning.

Voice 3:Ah it’s the man.

Voice 2:It is him.

Voice 3:What have you been up to?

Voice 2:Not much at my age.

Voice 3:Come on. I don’t believe it.

[Laughter]

Tatiana Korelsky: And it was great material to use technology, which is—actually they had to push the technology, because as you maybe understand, to understand emotion you only—you don’t only use words, you also use what we call, prosody, it’s the intonation contour of speech, it’s either high pitch as when people are excited, or it’s so—this prosodic analysis was also done, so that was in addition to what they proposed to do and to tell you, quite frankly, it’s unbelievable what they accomplished. On a small award, it was a small award five hundred thousand dollars for three years and it’s totally amazing to us now what they have achieved on that small small award.

Host: The research that they able to do after they solved the engineering problem of getting the material off the tapes, did it—maybe it’s too soon to know, maybe the research isn’t over, but did they learn the things they set out to learn?

Tatiana Korelsky: Yes, they did, and not only that, they published many, many papers in the best international conferences, which to us, in itself, proves the quality of their research because if they are—all these papers are usually peer review, very very stringently, you know, and the fact that they were able to publish, at the best conferences, actually, and many papers, just tells us about the quality of the research.

Host: And I to want make sure that we understand what the ultimate goal is here, all of this is leading you to develop these language models and through the use of this raw material that you got from NASA, to create, what?

John Hansen: We wanted the text transcript, but we wanted the transcript that actually had for all the audio, we wanted to have the exact time when the speech was produced, we wanted to at least understand that there were turns they were taking between speakers and we wanted to be able to synchronize that across the multiple channels. So think of this as, kind of, like doing the reverse of an audio recording from a symphony, you have, you know, thirty tracks of different instruments playing, you want to digitize each of those individually, you then you want to kind of understand the score of what each instrument is actually playing, and then you wanted to be able to look across these different tracks to understand how are these different sounds across these different instruments working together. Likewise, for NASA, how are each of these different channels and the people on each of these loops solving a particular problem and how does it affect people in another loop.

Host: And that’s what was happening in the example that you used, but you’re using that to develop technology that could be used in what way?

John Hansen: Yeah, so by advancing the speech technology, moving speech recognition out of simply just saying, well, I’ve got one person reading prompted text, let’s say for a news broadcast for TV or radio, to someone speaking spontaneously. Moving away from a single person talking, to groups of people talking collaboratively together. The technology that exists for, you know, the first part there, that’s been developed over the last, maybe, thirty or forty years. The technology for looking at multiple people talking collaboratively, people have been working on that, but it’s a very small portion of the actual research that’s been accomplished and the reason is because there really are no corpora available for the community to use. This will be the standard corpus that the speech and language community uses to look at collaborative multispeaker type engagement for solving problems. The real huge plus here is that these are people solving a real problem, you know, one would argue that the most challenging engineering problem mankind has seen. But also these are natural voices that were being recorded. All the microphones were known, you know, everything was digitized with a particular time code on there. You know, this is the best one could ever hope for, you know, no one would ever think if there’s a natural disaster wait, let’s actually record everyone who’s going speak and let’s make sure we synchronize all the audio, it’ll never happen. So this really is a very rare, but a unique opportunity that NASA has allowed the speech and language community to have access to, and it really will be a game changer. It will ultimately contribute to better understanding of how people work together in teams. You could imagine the Army or the military would to want know this, you could imagine anyone dealing with natural disasters who would want to understand this and they’re going to take the lead and their experience coming directly from how NASA works as a team and that’s, to me, a very, very, good reason why NASA represents, kind of a, you know, a primary example of actually how other organizations should work collaboratively together, it’s not a pitch for NASA, but it’s really what we see from the audio from here.

Host: Now, the second that resulted from Export Control clearing the Apollo 11 audio for use and release, is that Hansen and his team had a go to publish so you and I could listen to it too. It’s all online at the website: ExploreApollo.org

John Hansen: We’re an academic institution so we’re making all the data available.

Host: Where, where, where? If I wanted to go scan it, listen to some of it, where would I go?

John Hansen: So we have a website that you can get access to and it has some of the audio examples there, we had undergraduate students working on this for probably—well, we had several different teams working on this for maybe four years, working on this. One of the spots on the Explore Apollo website, students wanted to—we had like different stories. We said, well, you can’t just start playing this, no one wants to listen to the whole thing. So you have to have bite size chuncks that would be appropriate. The students starting developing those things, and then we test drove those little stories at the Perot Museum during Engineers Week. Having kids that are, you know, from first grade up to eighth grade kind of listening to it. We got their feedback. We had other, there’s another, a group on campus called SEEC, The Science Engineering Education Center, Russell Hulse our Nobel Prize Laureate at UTD oversees that, and so we had that group also review some of the stories and things. So we have small bite size chuncks, so to speak, on the Explore Apollo type website that we set up. Students wanted to have something that would be like random, so there’s a little button that you could say, “surprise me” and it would randomly put you into some spot during the mission that you could kind of listen to, which is an interesting thing. I had to remind the students, look, nothing gets NASA engineers more nervous than the word surprise me, so I said, you’ve got to be careful when you do that, but it was good.

FDO: Network, FDO.

Network: Go ahead FDO.

FDO: Ed?

Network: Yeah.

FDO: [Inaudible] You know that sheet of paper? LLS determination, you know, where you fill in and do all your computation? Right after touchdown.

Network: Yeah you’d like one?

FDO: Yeah would you—no I got one, but I never did get one all filled in ’cause we had so much trouble right after descent. Would you have one of the math aides copy that and send it to me down the P-tube.

Tatiana Korelsky: The site is very nice, it’s very nice, I actually, preparing for this interview, I went and I used the site a little bit. The site has stories about what’s happening on this Apollo tape, you can click and you hear the conversation and you see the running transcript, which is automatically recognized, it’s a very, very, nice site for the public, and especially, and for younger generation. They I think also that this website, although that Hansen might tell you in more detail, that some variant of it was also installed in the Perot Museum of Science, but that is another thing. For kids, it was just really fascinating because they can browse these tapes and see exactly what was said, and who, you know, who is talking, like commanders and the Control Center, very, very, interesting.

INCO: Capcom, INCO.

Capcom:Go ahead.

INCO: I was just looking on page 3-55 and they got a picture of the spacecraft, the angle to the sun, and the attitude of the spacecraft. And it looks like to me about 90 degrees on that day.

Capcom:That was 15 hours of—

Host: Greg Wiseman, the audio engineer at JSC who helped John Hansen and his team digitize the audio for their project and helped post the audio online.

Greg Wiseman: And so now we have it uploaded in bulk to archive.org making it available for anyone who wants it. And since we’ve released it, I’ve been in contact with some documentary filmmakers who are taking this audio and they’re synchronizing it with some recently discovered seventy millimeter film that was shot in the Mission Control room during landing, which is really exciting because for the first time that film will have an audio component, they didn’t record audio originally with the film, but, of course, NASA was recording the audio loops, so they were able to take our recordings and lip sync it with this film, because a lot of the film was shooting fight controllers, so they’ve got the film and now they’ve got their audio; and it’s really amazing, it’s amazing the work that those folks were able to do resyncing all of this. So all of these things are coming together at the same time. I think it’s going to provide a new perspective and insight into Apollo 11 and I think, as we get closer to the anniversary of the moon landing, with the release of all this new material, it’s really going to propel this monumental event back into the forefront public consciousness.

John Hansen: We’ve distributed, I think, a few of these to a couple of different groups. There’s one group that’s developing a movie for CNN and an IMAX movie using some of this audio, I think, yeah. So one of the interesting challenges from that effort this one company, NASA had a videographer that would walk around with a video camera and actually videotape, you know, Mission Control space, people, working and stuff, but the video recorder was actually just video, it didn’t have any audio, so they had these silent films of all these people. They may take, you know, five seconds or ten seconds of video here and then they’d go somewhere else and take five or ten seconds. So all that video is actually available, this one company had access to it, but they have any audio, but they could see someone was talking, you know, on their headset, so they were trying to read the lips, and look to see, okay, we think they said this and then they would try to go back and see—

Host: Search.

John Hansen: Search and find that, but they weren’t being too successful, so they reached out to NASA, I think, Greg Wiseman they contacted, and Greg said, well, go find John Hansen up there in Dallas and maybe he can help you out. So we talked with them, and we sent them—we got permission from NASA, but we sent them the full material for Apollo 11 and then Apollo 13, and the way I see is it they’re basically playing Frankenstein. What they’re doing is they’re using video that was recorded from a silent video camera, and they’re linking that to audio that we digitized, you know, forty eight years later, and stitching them together, and it’s really cool because there’s some examples where you’re listening to an audio, which is the audio that we pull up, and every once in a while you’ll see a video portion of the person actually speaking and then it just goes to audio again, and so what they’ve done is they’ve just stitched this video and connected it to the audio. And it’s really interesting because it gives you, like, a window into what Mission Control space actually looked like when they were talking. When we, kind of, had some of these news releases, we started getting contacted by, you know, someone saying, yeah, my father worked for NASA or my next door neighbor, he worked for NASA; and so right now one of my students is actually doing something called, “Finding Waldo” if you know what Waldo is, you’d find a picture of him. So someone will say, yeah, my uncle or my grandfather or my father worked for NASA so we asked them, well, can you send me a picture of what he looked like at that time and what he looks like now. And what we’re going to do is we’re going to go back and we’re going to actually go back and we’re going to actually go across all nineteen thousand hours and try to find examples of when that person spoke and then on our archive, we’re going to have their picture and then we’re going to have a little, you know, folder with all of the audio clips that that person spoke. So now someone will be able to actually say, hey, my grandfather worked at NASA, here are examples of him talking to Neil Ar—well, he had to be Capcom if he was talking to the Neil Armstrong—but here is, you know, him talking with other specialists at NASA at the time, and so this actually, we hope will show a little bit about the people that worked behind the scenes that really are, kind of, like the unsung heroes that made sure that everything was going to be successful. Very much true in Apollo 11. Apollo 13 was probably NASA’s one of the biggest accomplishments they were able to achieve, you know, at least when I looking at Apollo, so we’re kind of happy this notion of the heroes behind the heroes, we think it’s something that rings true.

Host: They’re going to kick us out of here, this has been terrific.

John Hansen: All right let’s go. Let me make one more statement.

Host: Yes.

Apollo 11 MOCR

John Hansen: So one of the—you might ask, well, why would we to want try and put so much effort into digitizing and making this technology available? We really wanted to try and emphasize that people at NASA at that time were trying to accomplish an enormous engineering challenge. When we think about NASA and we think about the Apollo program, we tend to gravitate towards the astronauts. The astronauts are to be admired, you know, they risk their lives, they achieved enormous accomplishments for, not just the United States, but for mankind, but when we talk with students one of things that people often forget is for someone to make it to the moon and back safely, you had to have a team to people working collaboratively together to solve some of the most challenging engineering tasks that you could ever imagine, so we kind of used this expression, “The Heroes Behind the Heroes” these are the people that sacrificed everything to try and make sure that NASA was going to accomplish this.

[Music]

Host: In 1962, just a couple of dozen miles up the road from where I’m sitting today, President John Kennedy predicted that exploring space would be hard and he set an ambitious goal for the nation.

President John Kennedy: Because that goal will serve to organize and measure the best of the our energies and skills because that challenge is one that we’re willing to accept, one we are unwilling to postpone, and one we intend win too.

Host: John Hansen and his students and a group of irregulars gathered for the task, made it possible for you and me to travel in time fifty years into the past to hear the men and women of that America live up to the challenge. You could listen to all unfold for yourself today at: ExploreApollo.org.

[Music]

Thanks for sticking around to see if there was anything more after the music ended, I mean, what more Could there be to say after four full episodes, right? Well, just this, we decided to take a different approach with these few podcasts for a couple of reasons, one is because we felt the story was worth it, but mostly, it’s because we can, the podcast format allows us to spend as much or as little time as we choose on topics that interests us and that we think will interest you, and we have the freedom to try out different styles of storytelling to achieve our goal, capturing your attention with the stories of what’s going on past, present and future, in American’s space program, and we hope you’ll keep coming back for more. The Heroes Behind the Heroes episodes of Houston, We Have a Podcast, were produce by Greg Wiseman and me with editing and audio engineering by Greg, with help from Alex Perryman thanks to the guests who shared their parts of this story, Tatiana Korelsky at the National Science Foundation. Larry Vrooman and John French who helped bring SoundScriber back from the dead. Karen Walsemann of the Export Control Office at the Johnson Space Center. Tuan Nguyen and Lakshmish Kaushik of the UT Dallas team who digitized the tapes and built the speech recognition system. My NASA media production services colleague, Greg Wiseman and Dr. John Hansen the leader of the whole effort. Also, thanks to Norah Moran and Gary Jordan for helping us pull all of this together for you. Check out the whole series and all of our podcast episodes at NASA.gov/podcast and when you do, please check out the other cool NASA podcast that you will find there also, like, “Welcome to Rocket Ranch,” “On a Mission,” “NASA in Silicon Valley,” there are many more and they’re all worth your listen, they’re all available at the same spot where you can find us, NASA.gov/podcast. We’ll be back with all new episodes starting next week.

[Music]