Oral-History:Gene Golub

About Gene Golub

Gene Howard Golub, Fletcher Jones Professor of Computer Science at Stanford University, was one of the preeminent numerical analysts of his generation. Born in Chicago in 1932, he was educated at the University of Illinois at Urbana-Champaign. He had been at Stanford since 1962 and became a professor there in 1970. He had advised almost thirty doctoral students, many of whom have themselves achieved distinction. Gene Golub was an important figure in numerical analysis and pivotal to creating the NA-Net and the NA-Digest, as well as the International Congress on Industrial and Applied Mathematics. He died in 2007.

Golub was awarded the B. Bolzano Gold Medal for Merits in the Field of Mathematical Sciences and chosen as a Member of the National Academy of Sciences (1993). He held several honorary degrees and was scheduled to receive an honorary degree from ETH Zürich on November 17, 2007. He was also an IEEE Honorary Membership Recipient for "improvements in signal processing, modeling, control, and other areas through numerical methods for least-squares estimation and matrix singular value decomposition."

In the interview, he discusses his research, his contributions to numerical analysis and Stanford, and his relationship with the National Science Foundation.

About the Interview

DR. GENE GOLUB: An Interview Conducted by Andrew Goldstein, Center for the History of Electrical Engineering, July 18, 1991

Interview #109 for the Center for the History of Electrical Engineering, IEEE History Center, The Institute of Electrical and Electronics Engineering, Inc.

Copyright Statement

This manuscript is being made available for research purposes only. All literary rights in the manuscript, including the right to publish, are reserved to the IEEE History Center. No part of the manuscript may be quoted for publication without the written permission of the Director of IEEE History Center.

Request for permission to quote for publication should be addressed to the IEEE History Center Oral History Program, IEEE History Center, 445 Hoes Lane, Piscataway, NJ 08854 USA or ieee-history@ieee.org. It should include identification of the specific passages to be quoted, anticipated use of the passages, and identification of the user.

It is recommended that this oral history be cited as follows:

Gene Golub, an oral history conducted in 1991 by Andrew Goldstein, IEEE History Center, Piscataway, NJ, USA.

Interview

INTERVIEW: Dr. Gene Golub

INTERVIEWER: Andrew Goldstein

DATE: 18 July 1991

PLACE: Telephone Interview

Overview of Research

Goldstein:

You probably remember from my letter that I want to talk about the research support you received from the National Science Foundation. I have some questions about it. Let me first ask you some quick questions that should offer me greater detail about your work than I can get from the abstracts. What specifically were you trying to accomplish with the work that you did during the ‘70s?

Golub:

I was really interested in developing numerical algorithms for solving least squares problems and data for these problems. There was a lot of fallout as a result of that, and lots of other nearby problems were solved and analyzed. Basically I was just working on linear problems.

Goldstein:

Did you have specific applications in mind when you developed these problems and did you propose them?

Golub:

No. Well in a generic sense, but not in a very specific way.

Goldstein:

Did your work find application?

Golub:

Yes. That is one of the hallmarks of my work. That is, generally people find it useful and that pleases me greatly. That stimulates me tremendously because then often there is feedback. I do something and then they come back and sort of ask for something else.

Goldstein:

Can you give me some examples of what specific techniques you developed, and what their application might be?

Golub:

They are often technical. I mean, they are really matrix algorithms that people use in solving certain Eigen value problems where you have linear constraints. So, what is your main interest?

Goldstein:

For this project, we are trying to understand the role that the National Science Foundation had in stimulating the development of computer science. This particular chapter is focusing on research in the United States. We have broken up research into several sub-categories. Numerical analysis is one. Graphics is another. Artificial intelligence, computer theory are others. What I would like to get at is the nature of the work that was supported. Some talk about other organizations who were supporting similar work — where the NSF fit in, supporting the work, promoting the work. You know, in some cases they may have aggressively solicited research in particular areas. That is the sort of thing that I would like to find out.

Atmospheric Research

Goldstein:

I noticed in one of the abstracts that you were working with Klein and his project was to find the normal modes of the Indian and Atlantic Ocean.

Golub:

Yes.

Goldstein:

Can you give me some detail about that project or similar ones?

Golub:

Well, no. I could just give you a little history. Allen at that time worked for NCAR, National Center for Atmospheric Research, in Boulder. I gave a talk there and he was working on a problem connected with a man by the name of George Platzman, who was at the University of Chicago. So, NCAR — Are you familiar with the organization?

Goldstein:

No I’m not.

Golub:

It’s actually a National Science Foundation supported organization. They do work connected with meteorology typically. This guy seemed to have approached Allen, and then I got involved because I was at that time working on an algorithm for solving Eigen value problems. I’m really impressed by all the papers I did on this. So, then we kind of worked together and, well, Platzman's problem was a little different so we had to handle it. We had to generalize from previous work that I had done. Then everything came out all right. Now there is of course some question whether the model that Platzman was working with was really the correct model. But for the model that he was working on, it was really the appropriate one.

Goldstein:

Did they approach you for your assistance?

Golub:

Well, one way that I work is that I’m a peripatetic. You know, I do a lot of traveling. I was just visiting and giving a talk in Boulder and then Allen, whom I knew from other circumstances, and I, got together and talked about this project. That is how it began.

Goldstein:

I know that some of your other colleagues were in Tennessee and New York. How did you become involved?

Golub:

Either by mail or seeing people at meetings or giving lectures at another institution. I think sort of a personal forte of mine is to go around and think about an idea, and then often I get other people to work on it. You know, work out all of the details. I have been very fortunate in that.

Approach to NSF Proposals

Goldstein:

Now in cases like this, would you sit down with your collaborator and develop a proposal for the National Science Foundation or would you simply collaborate on papers that were supported by money that was given to you?

Golub:

Here is what my real game plan is. I submit a proposal which is a general proposal and it concerns problems that I have been working on. But I feel that once I’m supported I'm free to work on those problems as well as other problems too. Though very often I just work on nearby problems as well as the problems that I write in my proposal. I have never checked with anybody, but I assume that NSF just wants to support people who are doing research.

Goldstein:

So did your proposals have specific research goals?

Golub:

Audio File

MP3 Audio
(109_-_golub_-_clip_1.mp3)

Yes they did. They stated certain problems that I was going to do. But NSF was not very specific, at least in the old days, about what you had to achieve. Sometimes when you reapplied for a grant, they would ask the reviewer to see whether or not the person actually achieved what he said he was going to do. I guess, my view is, that when your grant is supported, it’s really almost like a hunting license. You should work on the problems that you suggested, but also any other research that is relevant to the general goals of your proposal.

Goldstein:

It sounds to me, then, like what you are saying is that you didn’t interact very extensively with members of the foundation review board. You had proved that you were productive, and they continued to support your work?

Golub:

Yes. My own vision by the way, is that NSF should not be too specific. The other agencies are more of that sort. DARPA and government and military agencies.

NSF Compared to AEC

Goldstein:

Yes that is an important question. I noticed in some of your papers that you also received support from the Atomic Energy Commission. How did their style differ from NSF?

Golub:

Well, it’s more personal. They don’t have as many people there. They have generally had one person in the math sciences program in the past. So there is more of a one to one relationship. Its not filled by rotators. Often you have rotators at NSF and that is not the case at the Atomic Energy Commission.

Goldstein:

Did the AEC take a more proactive role in steering research and trying to accomplish specific ends?

Golub:

Well, they would say certain things would not be desirable. The guy who was there last wasn’t very interested in software. Now, I have somebody who is interested in software. So, you see, sometimes the controllers, so to speak, can focus.

Goldstein:

I would expect that. That is one of the things that I am interested in: what sort of emphases were placed on NSF supported research, and what did NSF want to see during the ’70s?

Golub:

You know, I've never gotten the word from them. I just would write a proposal, submit it and most times it would be supported. Most of the negotiations would be over the budget rather than any of the details.

Support for Stanford Graduate Students

Goldstein:

What were the budgets like? Did it support graduate students who were working with you? Travel?

Golub:

Often one or two students.

Goldstein:

Anybody notable?

Golub:

You mean any notable students?

Goldstein:

Right. I’m just curious if any of the students who worked with you have gone on to distinguished careers themselves?

Golub:

We had a wonderful array of students at Stanford. We have a numerical analysis activity. Maybe I should give you a little history. Do you know about the computer science department at Stanford?

Goldstein:

I think I know something about it. You can start from scratch.

Golub:

One time it was rated the best department in the country over all departments, whether it is English, or French, or anything. So it really had a sterling reputation. The department was founded by a man called George Forsythe who died in 1972. He was a numerical analyst. Through him and various other circumstances we really had top-notch students. The students that we had in that period, not all of them did their Ph.D. under me, but I might have supported them for a year or so. They occupied very good positions. They weren’t supported, necessarily, through NSF. Like the professor of Numerical Analysis at Bergen in Norway, an ex-student of mine. The numerical analysis group at Bell Labs, which is well known, with the exception of two people, everyone got their degrees at Stanford. We have a tremendous influence at places like Bell Labs.

Goldstein:

How did Stanford come to this position of preeminence? Was that through NSF support?

Golub:

I don’t know. I mean it certainly was important that we had that support. I’m glad that we could support students. There was also a time where many of our students got NSF support. Now as it turns out during this next year, two of our students in our new program — it’s called “Scientific Computing and Computational Mathematics” — will have NSF fellowships. We are really happy about that.

Goldstein:

Has the level of difficulty in obtaining NSF support increased or decreased over time?

Golub:

Well, we probably get relatively less money than we did before. During this golden period, we probably had one of the world's most eminent numerical analysts with us. His name was James Wilkenson . He was a Fellow of the Royal Society and really a very famous and wonderful man. He and I had a proposal together, and that was funded and that certainly helped us a lot. We were able to support a couple of students.

Goldstein:

I want to ask you another question about your specific research interests. I noticed in a paper that you did with Tang, who I guess was a student of yours —

Golub:

Yes, Wei Pai Tang. He was a student here then.

Goldstein:

Right. It was on Bandermann Coefficients. I am not familiar with this bit of math, but apparently you say that the algorithm that you were using, because it used Velakty composition, was particularly well suited for parallel computation?

Golub:

Yes.

Motivations for Research

Goldstein:

How often would you pursue a particular line of research because of hardware capabilities?

Golub:

Anytime I sort of try to construct an algorithm, I think, “Well, is this an algorithm that will be useful on a parallel computer and how can you handle this?” I think in those terms all of the time.

Goldstein:

What would you say stimulates a research question in your mind? Is it computability or applicability?

Golub:

Applicability and computability. Whether I can figure out a fast and accurate algorithm for solving a problem.

Other Numerical Analysis Researchers

Golub:

I don’t feel I’m giving you enough information. How many people are you contacting here?

Goldstein:

Well, it depends on their research area. I think there are four or five people that I want to talk to in numerical analysis.

Golub:

Who are the others that you are talking to?

Goldstein:

You are the first I have called. At Stanford, I may opt to talk to Herriot.

Golub:

He is retired. He may be at home.

Goldstein:

When I say Stanford, I am referring to his home institution when he was working. Wayne Cowell at Argonne, David Young at Texas, yourself and Herriot are the four.

Golub:

Let me just give you a little run down. May I tell you about these people?

Goldstein:

Sure.

Golub:

They will probably tell you things about me too. Well, David Young is really somebody who did a very important piece of work. His thesis is sort of one of the landmarks of numerical analysis. He’s been at the University of Texas for a long time. I think he is about sixty-six or sixty-seven-years-old and he continues to be very active. Its funny, he sort of continued along the line of his original research. Although he has done other things, that is really what he is famous for. He’s a very nice and gentlemanly man.

Herriot actually never has been a very significant scientist at all, but he worked with George Forsythe. So I think the original grant was in the name of Forsythe and Herriot. Then Forsythe died unexpectedly. So, it was really Herriot that had the money. During the period of the ’70s, if Herriot wrote three papers that would have been a lot. That was just my impression. He had a big grant, but it was really there because of Forsythe.

Goldstein:

All right. The other name that I had was Wayne Cowell at Argonne.

Golub:

He was an organizer, but he really didn’t do that much research.

Goldstein:

Who would you name as more significant names?

Golub:

In numerical analysis?

Goldstein:

That is right.

Golub:

That NSF might have supported? John Rice is a very well known figure. He is at Purdue University. He has had a lot of impact in numerical computing. He is head of the Computer Science Department.

Goldstein:

When you say a lot of impact do you mean in terms of setting research agendas?

Golub:

That is right.

Goldstein:

Or significant results?

Golub:

Cowell had bright young people around him who developed the Setzer-Lindt pack, an ice pack that they developed. He was not what you would call a research person at all. He may be retired, as a matter of fact. Carl Debower is really an important figure.

Goldstein:

Actually, his name didn’t come up too much in the NSF grants.

Golub:

But he was supported. You see, he was at the Army Research Center in Madison, Wisconsin. He was supported by the Army, largely. Maybe some NSF money. He is really a great figure in numerical analysis.

Development of Numerical Analysis

Goldstein:

How would you describe the development of the field since Forsythe? What have been the principal research questions? Where have people taken them?

Golub:

Well, the great development is, of course, we have much more powerful computers, and so then people attack larger problems. For instance, they have gone from solving partial differential equations in two dimensions over very regular domains to solving partial differential equations in two dimensions over irregular domains. That was the motivation for the Finite Element Method. If you could have very complicated regions, you could use the Finite Element Method. Now they have gone up to three dimensional problems. So the three dimensional problems require more space and more time to solve. So as we have gotten more powerful computers, we are able to solve larger problems. That motivated people to go on to do certain kinds of research. So, large scale problems are really what is being done.

Goldstein:

Are new mathematical techniques developed to approach these larger programs or are they more like new algorithms?

Golub:

Well, both. The Finite Element Method, it’s been around for awhile, it’s a mathematical technique that really developed in the ’70s and ’80s. People understand it better, and variations of this method have come along and all kinds of things.

Goldstein:

How does NSF fit into this?

Golub:

The NSF has two pockets where it supports numerical analysis. One is through computer science activity. A lot of my research is of that nature — you know, constructing algorithms and designing algorithms. Then there is also the mathematics branch of NSF which does a similar thing, but they are of a more analytic nature. I think that the money that has come to me has come partially out of the math budget [and] partially out of computer science.

Goldstein:

Do you deliberately direct your proposal to one or the other?

Golub:

Well, most often to computer science. It’s appropriate for either area to support me. As I said, often they support me jointly.

Goldstein:

Is that their decision? I mean, they look at your proposal and decide that it merits support and then they figure out where it is?

Golub:

I think that is how it works. You would have to ask them. Have you talked to people at the National Science Foundation?

Goldstein:

No, but the principal investigator in this study, William Aspray, has spoken to them.

Golub:

I see. How did you get involved in this whole activity?

Goldstein:

The National Science Foundation is supporting this study as one part of a series of studies that they are doing on history. This is the one on computer science. Now, you say that one of the trends in numerical analysis has been the ability to handle larger problems. In looking over the grant titles, it’s hard for me to pin down trends, because some of them are brief and sometimes vague. I get the feeling non-linear studies are more common than they have been in the past?

Golub:

Yes. I think people are now moving on to that.

Goldstein:

Is that again due to increased computing power?

Golub:

Yes, and I think people feel they understand the linear problems well enough. Except there's still a lot to be done. I say that because it’s my own area of interest.

Goldstein:

Are you aware of any funding source that is trying to direct this research?

Golub:

On linear problems?

Goldstein:

Yes.

Golub:

Well there has been a lot of interest. I forgot who is actually pushing this.

Change from "Passive" Funding

Goldstein:

I think that is all that I want to ask you about. I may look over my notes, and develop some more specific questions. But I want to thank you for taking the time to talk to me.

Golub:

I personally feel very pleased with my relationship with NSF. At least as long as they've been funding me.

Goldstein:

It seems like it’s not terribly interactive.

Golub:

That is right, but most of those agencies have been that way. There has really been a change though, a notable change. It used to be that they were more passive, and now there is more effort to stir things up. Maybe that was necessary. Of course, since I was relatively successful in the way that things went in the past, I’m not very interested in seeing things stirred up too much. I mean, I think in the past the model was pretty good and a lot of good research was turned out.

Goldstein:

I see that a few times you identified your home institution as the National Bureau of Economic Research.

Golub:

I was a consultant there. I was included on some of their grants.

Goldstein:

What sort of work were you doing there?

Golub:

Well again, they had linear problems that I worked on.

Total Least Squares

Goldstein:

Tell me if this is correct. The impression that I got from looking over your papers was that you were working at solving Eigen value problems for matrices with an enormous number of elements?

Golub:

Yes, that is partially it. A lot of my work has to do with large, large problems.

Goldstein:

When you say partially, what am I missing?

Golub:

There are linear problems and then there are some nonlinear problems. For instance, there is a paper by myself and Charles Van Lone called “An Analysis of the Total Least Squares Problem.” That paper is now a classic. It had been done in part before. I mean, some of the ideas. Then Charlie and myself had sort of extended those ideas and really were able to investigate it more successfully.

Goldstein:

When you say classic, is it often cited or simulated?

Golub:

The statisticians use a different word. They use areas and variables. But the words “total least squares” are taking over.

Goldstein:

I am familiar with that and I did not realize that that was the source.

Golub:

Yes, and it was my co-author, Charlie, who invented that phrase, “total least squares.” So that paper in 1980 was really the first time that it appeared. Now I’m going to be going to Belgium in a few weeks and we are going to have a conference on total least squares. The Army’s European office has some money. They were happy to sponsor a meeting. I am really excited. I’m fifty-nine-years-old, so I don’t get so excited about these. I was in Belgium recently and I looked at the schedule and the program and I thought, “Boy, this is really going to be a great meeting.” The word, as I said, originated in that paper with myself and Charlie Van Lone. And Charlie was the one, I should say, who thought of the idea of calling it total least squares. How did you know about total least squares?

Goldstein:

Just from my training in school.

Funding from AEC and DARPA

Goldstein:

It seems like most of your support has been from the National Science Foundation or the Atomic Energy Commission?

Golub:

Yes, but then I had a falling out with the guy.

Goldstein:

At AEC?

Golub:

Yes.

Goldstein:

For personal reasons?

Golub:

Well, I don’t know. Just a very difficult person to deal with.

Goldstein:

I am curious if there were other organizations who were interested in supporting your work.

Golub:

Well, I have had money from the army.

Goldstein:

Okay. Did you ever approach DARPA or work with them?

Golub:

Well its interesting. In the past, DARPA was not very interested in supporting people in numerical analysis. Actually, now, I am getting my first DARPA grant after all of these years. They have had a change. At one time they really didn’t want to support mathematics at all. Now there has been a really big change there. They have a pretty active program. So I’m getting some support. It’s working on signal processing. But it is really connected to the same sort of linear problems that I’ve worked on in the past.

Goldstein:

Was there an understanding in the numerical analysis community that DARPA wasn’t interested and AEC and NSF might be more sympathetic?

Golub:

Yes. It was only a few years ago that DARPA got interested in mathematical problems. There was a woman there by the name of Helen Wushnusky. I don’t know if you have ever heard of her?

Goldstein:

No.

Golub:

She was at DARPA. Somehow she made a big thing of mathematics at DARPA. She left and she’s at Lockheed now. Since then they have actually had much more prominent people working in their mathematics program.

NA-Net and Netlib

Goldstein:

Is there anything else that you would like to say in summation?

Golub:

I have done a lot of things, because I guess that I was partially supported by NSF. They used to support me in part during the academic years. Well a few years ago Jim Wilkenson asked me, “Who could he send messages to by electronic mail?” And I said, “Oh, well here is a list.” I had about ten names at that time. But then I sort of built up a list and then I fixed it up with our systems people that you could send a message to anybody in the numerical analysis community whom I had an address for, by just writing na.lastname. So if you wanted to send a message to, let’s say David Young, you would send it to na.d@na-net.ornl.gov. Or if you wanted to send a message to Carl Debower, it would be na.debore@na-net.ornl.gov. It used to be na-net.Stanford, but we changed it.

Goldstein:

Is the host machine at Stanford?

Golub:

It used to be, but I was doing everything myself or a student and I would be doing everything. So I got tired of that. Now it has been moved over to Oak Ridge and it’s partially supported by the NSF at this time.

Goldstein:

Has that been significant in terms of stimulating interaction?

Golub:

We have had over 2000 names who are all over the world who are on this. People in, you know, Australia, India, Sweden and so forth.

Goldstein:

The comparisons that spring to my mind are either with ARPANET or CSNET.

Golub:

Well, it’s an artificial network. I mean there is no physical network, it’s just network. We call it na.net. And then once a week a digest is sent out which lists problems, meetings, deaths. You know, sort of keeps the community together.

Goldstein:

Can anyone use it?

Golub:

Anybody could use it.

Goldstein:

Do you need to subscribe specially?

Golub:

Yes. You don’t have to subscribe, but if you want to that is fine and you get this digest about meetings and so forth. I could send you the information.

Goldstein:

No, I’m less interested for myself, and just curious.

Golub:

I think it’s the only group of scientist that have such a network. Well, I started it all, really. Now it has gotten sufficiently big that I pleaded with my pals at Oak Ridge to take it over.

Goldstein:

You just started independently or did you submit a proposal?

Golub:

No I have never gotten any support for it. In a way, you might say NSF gave me some time to do research and that is one of the net results of it. There was no specific money and maybe that was unfair to them but on the other hand it’s a contribution to the whole community that's been very great.

Goldstein:

When did you say that you launched it?

Golub:

About seven or eight years ago. It’s evolved and changed and so forth. Then there is something called Netlib. Have you heard about that?

Goldstein:

No.

Golub:

A former student of mine, Eric Gross, and Jack Dangara who is now at Oak Ridge. What they did was they took all of the software that they could in the public domain and they put it in Netlib which is also on na-net.

Goldstein:

It sounds enormously useful.

Golub:

It partially came out of my attendance at an NSF meeting. I was on the advisory board for the Computer Science Division. They were just developing CSNET and they said, “We have these processes which could be used for other things too.” I think I mentioned that to Eric Gross and to Jack. And then they just went ahead and did it at their own institutions. They didn’t even bother with the processes that CSNET was using at the time.

Time on Advisory Board

Goldstein:

Just to help me round out my chronology, when were you on the advisory board?

Golub:

1982 to 1985.

Goldstein:

Thank you.