Oral-History:Ayanna Howard

About Ayanna Howard

Ayanna Howard grew up in Pasadena, California. She then attended Brown University and received her bachelor’s degree in Computer Engineering. Howard attended the University of Southern California for her doctorate degree where she wrote her thesis on enabling a robot manipulator to sort out waste for hospitals. Currently, Howard is the Linda J. and Mark C. Smith Professor and Chair of the School of Interactive Computing at the Georgia Institute of Technology. She also holds a faculty appointment in the School of Electrical and Computer Engineering and serves on the Board of Directors for the Partnership on AI and Autodesk. She has also served as the Associate Director of Research for the Institute for Robotics and Intelligent Machines, Chair of the Robotics Ph.D. program, and the Associate Chair for Faculty Development in ECE at Georgia Tech.

During and after her time as a graduate student, Howard worked at NASA’s Jet Propulsion Laboratory (JPL) where she became interested in artificial intelligence and human-robot interaction. Dr. Howard was a Senior Robotics Researcher and Deputy Manager in the Office of the Chief Scientist at NASA's Jet Propulsion Laboratory. After the space shuttle accident at NASA, Howard left the Jet Propulsion Laboratory to join Georgia Tech faculty in 2005. In 2013, she founded Zyrobotics, a company that utilizes her research in order to develop therapy and educational products for children with disabilities.

Dr. Howard’s career focus is on intelligent technologies that must adapt to and function within a human-centered world. Her work, which encompasses advancements in artificial intelligence (AI), assistive technologies, and robotics, has resulted in more than 250 peer-reviewed publications in a number of projects from healthcare robots in the home to AI-powered STEM apps for children with diverse learning needs.  To date, her unique accomplishments have been highlighted through a number of awards and articles, including highlights in USA Today, Upscale, and TIME Magazine, as well as being recognized as one of the twenty-three most powerful women engineers in the world by Business Insider and one of the Top 50 U.S. Women in Tech by Forbes. Howard has been involved with IEEE and the Computing Research Association – Women (CRA-W).

In this interview, Ayanna Howard discusses her past and current research, her interest in integrating robots with humans, and her collaboration with children, NASA, colleagues and students. She also gives advice to prospective students who are interested in robotics and talks about the issue of diversity in field of engineering.

About the Interview

Ayanna Howard: An Interview conducted by Peter Asaro, IEEE History Center, 22 January 2015

Interview # 791 for Indiana University and IEEE Center, The Institute of Electrical and Electronics Engineers, Inc.

Copyright Statement

This manuscript is being made available for research purposes only. All literary rights in the manuscript, including the right to publish, are reserved to Indiana University and to the IEEE History Center. No part of the manuscript may be quoted for publication without the written permission of the Director of IEEE History Center.

Request for permission to quote for publication should be addressed to the IEEE History Center Oral History Program, IEEE History Center, 445 Hoes Lane, Piscataway, NJ 08854 USA or ieee-history@ieee.org. It should include identification of the specific passages to be quoted, anticipated use of the passages, and identification of the user. Inquiries concerning the original video recording should be sent to Professor Selma Šabanović, selmas@indiana.edu.

It is recommended that this oral history be cited as follows:

Ayanna Howard, an oral history conducted in 2015 by Peter Asaro, Indiana University, Bloomington Indiana, for Indiana University and the IEEE.

Interview

INTERVIEWEE: Ayanna Howard

INTERVIEWER: Peter Asaro

DATE: 22 January 2015

LOCATION: College Station, Texas

Early Life & Education

Q: So if you could just start by telling us your name, where you were born and where you grew up and went to school.

Ayanna Howard:

My name’s Ayanna Howard. I grew up in Pasadena, California. I went to undergrad at Brown University and graduate school at the University of Southern California.

Q:

And when did you first become interested in robotics?

Ayanna Howard:

I became interested in robotics in middle school, so it was sixth grade. I was interested in sci-fi, so sci-fi was all the rage, and they were starting to show women in positive lights, so Wonder Woman, and my favorite was Bionic Woman. And so I knew when I saw it I was like I wanted to build a bionic woman and, in fact, if you looked at it, she was cool. She was bionic. The engineer was a little nerdy, but that was okay. So I wanted to be a doctor, so I was going to be a medical doctor and I was going to create the bionic woman. And then I took biology and absolutely hated the cutting, <laughs> and so I switched in terms of-- in fact, there was a teacher that said, “You know, well, you really want to create the bionics. You don’t necessarily want to do the operation.” So I knew then that I was going to go into engineering, computer science, something related to robotics.

Q:

And what did you study at Brown?

Ayanna Howard:

I studied engineering, so technically computer engineering, but Brown has a liberal approach to engineering, which basically means you take a little bit of what’s considered mechanical, a little bit of electrical, a little bit of computer engineering, and then you decide based on your courses what you want to take in junior and senior year. So I was general education of engineering, but I consider myself computer engineering.

Q:

And what was the first robotics project that you worked on or did you work on any robotics projects?

Ayanna Howard:

Yes. My very first robotics project was actually my sophomore year in college. I worked with a professor who was a robotics guy and I did manipulation, so at the time worked on coding a robot arm to trace around objects to then figure out what the object was, which at the time was like, oh, you just program it. You approach. Didn’t realize that was actually cutting edge at the time.

NASA’s Jet Propulsion Laboratory & Graduate School

Q:

Great. And then what made you decide to go on to graduate school?

Ayanna Howard:

So I knew I was going to go for a masters, and this is because after my freshman year I started working at NASA JPL, and so everyone at NASA either had had a graduate education, either a masters or a Ph.D., so I knew I was going to at least get a masters; in fact, that was all I thought I was going to get because I actually wanted to have a real job and make some money. But when I went into grad school it was just so different than undergrad and it was so fun. I mean it was like, oh, there’s a robotics course. Okay. Oh, I’m doing robotics research. Okay. Oh, there’s a robotics person coming to give a talk. I mean it was perfect, so I decided to stay.

Q:

What were you doing at JPL then before grad school?

Ayanna Howard:

So I worked at JPL also while I was there. Basically I worked there for 20 to 30 hours and also was a student, which I don’t recommend to anyone.

Q:

What kind of group were you working with there?

Ayanna Howard:

So at JPL that’s when I got interested in artificial intelligence and learning and interacting with people. So I was a controls person, so I learned about feedback control and adaptive, and I was controls. But with NASA JPL what we started to look at was neural networks. How do you take this data and do data mining, all these interesting things. And so I consider myself an undercover computer scientist because of that. Did not take a formal computer science course until grad school basically because I said, “Hmm. I’ve been doing a lot of programming. Maybe I should actually have some formal education on this.” But self-taught computer science because of the whole NASA experience.

Q:

Great. And any projects in particular that you remember?

Ayanna Howard: So when I started off what I worked on was actually pretty boring; converting Fortran to C for some of the satellite data, so that was my very first project, and then went on to work on robot manipulation, before I left actually worked on Mars Exploration Project, so designing advanced technologies primarily to model how people navigate and how scientists interact, figuring out how do you take that and code it on a future Mars rover.

Q: And who were some of the people you collaborated with in that period?

Ayanna Howard:

So some of the people were-- one was my advisor, Homayoun Seraji. He passed away. And then other people who were in my group, there was Isa Nesnas, Eddie Tunstel, worked a little bit with Richard Volpe, who is actually now the section manager of the robotics. Larry Matthews was the project manager at some time. Curtis Padgett supervised some of my projects. Yeah, I think those were the major ones. Jacob Barhen, who actually went to Oakridge, he was one of my first supervisors.

Q: I need to just have a quick-- I was curious who the professor was at Brown that you were--

Ayanna Howard:

Wolovich. Wolovich.

University of Southern California & Robot Vision and Object Manipulation

Q: And then so you decided to go to University of Southern California?

Ayanna Howard:

Yes, mainly because they had a really good relationship with NASA. In fact, there were trailers at JPL that adverti-- the courses, and this was before the whole MOOCs and things. So you could go up in the trailer and actually take the course remotely. So that was just like a no-brainer. I could actually work at JPL, still take my courses at USC, so USC was just great for that, and they had some good professors.

Q:

And who were the professors you worked with there?

Ayanna Howard:

So actually my first advisor was Ken Goldberg, so he had started off at USC. In fact, he was my first research advisor. And then my second year there he decided to leave, so that was a transition point. You go or I had already passed the qualifying exam, so that’s a big thing. Do you go and have to retake everything, or do you just stay, find another advisor? And so we started off, he was going to co-advise me, and my primary advise became George Bekey. And so eventually what happened, George Bekey became the advisor, and so I graduated with George Bekey.

Q:

And what was your thesis project or projects?

Ayanna Howard:

So my thesis project and-- no, I didn’t do a masters. At USC you can request a masters, so I requested it, which is basically you have enough units, yeah. So my Ph.D. project was on figuring out how to enable a robot manipulator to sort out waste for hospitals. So the concept was at this time this was when there was a big concern about things like HIV and needles, and the hospitals hadn’t gotten into the security. So right now you go to a hospital; needles go into this metal bin. Everything is safe. So back then it wasn’t quite the same, and so my task was how do you have a robot go in and figure out what’s the bag of needles versus the bag of fluffy sheets versus trash, and how do you sort this waste out so that humans don’t get hurt? So learned things like vision because you have to figure out the shapes, put in some of my control in terms of deformation, so if you grab some of these things it deforms based on what’s in it. Can you learn that characteristic, and then how do you grab it? So it deforms that means you have to have different points of inflection in terms of grabbing it, so where should you actually grab it? And so I had two arms, two manipulators that would go and they would basically-- just like a little kid would go and squeeze it a little bit, come up with a model of, okay, what was most likely in it? Given that this is most likely in it where’s the best point to grab it: Underneath, two sides, pick it up, things like that. So that was my thesis project.

Navigational Algorithms & Terrain Assessment Systems at JPL

Q:

Great. And so you kept working at JPL after you completed your education?

Ayanna Howard:

I did. I basically on Monday I was 20-- at that time I was actually 30 hours. By Friday I was 40 hours, so it was not even a working at. It was just I just changed the number of hours that I worked.

Q:

And did you collaborate with other graduate students or faculty at USC during that period?

Ayanna Howard:

So I had my lab mates, so we weren't ever on any of the same projects but they’re your lab mates. So you see them every day, you cry on their shoulders, they come to your defense, and so my lab mates were-- in fact, some of them are still in academia. So there was Stergios Roumeliotis, who’s at University of Minnesota now. There was Jim Montgomery, who works with NASA. There’s Alberto Behar, who was at NASA that just passed away a week ago. There was-- I’m trying to think who-- tall guy, Mike McHenry, <laughs> who was at NASA. He left. Then I have Tony-- what’s Tony’s last name?

Q:

Lewis?

Ayanna Howard:

It’s Tony Lewis at Qualcomm. <laughs> Tony Lewis, who’s now at Qualcomm.

Q:

He was my boss <inaudible>--

Ayanna Howard:

Oh, really?

Q:

--robotics, yeah.

Ayanna Howard:

Okay, see? Small world. I’m like I couldn’t-- I was like, what’s his last name? <laughs>

Q:

So, okay.

Ayanna Howard:

And he’s doing very well at Qualcomm.

Q:

Yeah, building those brains used in cellphones.

Ayanna Howard:

Yes.

Q:

And so then what kinds of projects did you start working on following your Ph.D.?

Ayanna Howard:

So the ones I was talking about, so immediately after the Ph.D. I kind of started-- because now I had a Ph.D. I must know what I’m doing, and I started managing my own jobs and writing grants. And so my first big project was designing navigation algorithms for robots based on trying to figure out what’s in the terrain. So at the time it was terrain assessment. So if I see a terrain can I figure out what’s rocks, what’s cliffs? Can I tell the difference of the texture of the ground, so if it’s sand versus hard ground because you want to change your navigation profile. So I was incorporating vision along with learning along with fuzzy logic to classify the terrain, and then put that into your controller so that as you navigate you could use this information to change how you might rove on the surface. So that was one of the very first projects. And then some of that same technology started and was transferred to trying to figure out how to land safely on Mars. So you’re landing you want to do the same thing. As you’re descending you look at the terrain. You have a very large area that you want to descend and you don’t have precise information until you get there, and so how can you use that information about the terrain and from your sensors and come up in basically real time an assessment of what’s the best place to land and how do I get there. So in that aspect some of it is maybe the safest place you can't get to safely, and so what’s the next safest place? And so, again, combining things like terrain assessment, vision control in that case as well because you have to understand things like you also have measurements on you’re there now. What’s the “wind speed,” for example, and how does that equate in terms of where you can navigate to? So worked on safe landing and then-- so those were the two big projects that related to the Mars Exploration Program, but then some of the side projects I actually enjoyed that I did was doing things like designing a hardware sensor. So one of the things I’m interested in is-- so all of this was software based, so this learning and this navigation, can we somehow encode some of this logic, very, very specific and make a sensor? So one was a traversability sensor. You know, we’re doing all this terrain assessment. Can we just take that code and put it on a sensor and it basically gives you a value from zero to one. So that doesn’t take away from all of the other logic that’s going on, all of the other information that has to be processed. So I started dabbling a little bit in terms of designing sensors, designing little modules of AI components that you could put on, say, like an FPGA, so different things, so just kind of exploring. So those are my what I call pet projects.

Q:

And did some of those wind up on any of the rovers?

Ayanna Howard:

None of my stuff ended up on any of the rovers. <laughs> So why is that? So what happens is the way that research works is you have maybe four or five competing ideas, and it’s not necessarily-- not to say mine wasn’t the best, but it’s not necessarily which is the best. It’s the one that is validated as the most safest at the time that they need it. And so if you have a mission that’s, say, 10 years away they need it. They need to know that it’s going to be 90 percent reliable within the specifications basically 10 years before it happens. And as researchers we don’t even compete that way, so you just keep going and then someone’s like, “Hey, are you guys ready?” You’re like, “No, I need some more testing.” And then they go to the next researcher. “Oh, are you guys ready?” And so eventually that is what happens. And so with my stuff I left before-- so my mission would’ve been MSL, and so I left right when it would’ve been put in, but if you notice, MSL did things like change where it was going based on terrain assessment. So not to say anything, but those ideas and those thoughts were definitely started to put into the missions.

Joining Georgia Tech

Q:

Great. And why did you decide to move on from JPL?

Ayanna Howard:

So I decided to leave JPL because there was a shuttle accident, and at that point NASA basically-- I won't say zeroed out research funding, but they pretty much halted anything that was beyond the immediate horizon. And so at that point I had to decide do I just rough it out, figure it out, or what is it that I really liked? Did I like JPL and NASA or did I like research? And I decided that research was more important, robotics research, and so I decided that I go where the research is. And at the time academia was where research was still prevailing. You didn’t have to fight as much in terms of the NASA money, and you had other fights but you could still do research. So that’s why.

Q:

And so what was the period of time that you were at JPL?

Ayanna Howard:

So, actually, my very first summer was the summer of ’94. I became full time, so a full time employee, 40 hours in ’99, and I left in 2005. So I was going to be a lifer. I mean everyone was so surprised when I left. It was like, “You’re a lifer. Where are you going?” But it was a good decision.

Q:

And where did you go?

Ayanna Howard:

I went to Georgia Tech. So I went to Georgia Tech in 2005. I started there in July, and it’s been a great experience. It’s been different but definitely a great experience.

Q:

And which program department are you in there?

Ayanna Howard:

I’m in the school of electrical and computer engineering but also part of-- we have a robotics institute, so I’m also a part of the robotics institute, and I think they call me associate director of research, which means I do something with research with the institute.

Robotics Program at Georgia Tech

Q:

And was the robotics institute already there when you came in?

Ayanna Howard:

No. So when I started there was no robotics center. There was no robotics institute. There was no robotics Ph.D. program. So I was I would say the first of trying-- and so Georgia Tech has always had robotics, has had good robotics people, and so when I came, and one of the reasons that they hired me is they were just starting to think about how do you synergize the folks here? How do you grow that robotics edge? And so I was one of the first kind of strategic thoughts about how to do this. So I was on the committee that-- I was on the Ph.D. committee, so I was there pushing it through the system, so I was there for the first class. I was the second director of the Ph.D. program, so was there. The institute, we are the ones that interviewed Henrik Christensen, so I’m sitting with the other faculty interviewing who’s going to be the director of this new center. So, yeah, I was there before it existed. And I came I think at the right time at the beginning of the growth, which is nice because you have a lot more control about where it’s going.

Q:

Who are the other roboticists that were there when you arrived?

Ayanna Howard:

So Magnus Egerstedt was there. He still is there. Wayne Book, who just retired but he’s still there. Ron Arkin, who’s still there, Tucker Balch, Frank Dellaert and Aaron Bobick. He’s robotics, more heavy on AI, but he’s considered a-- now he’s totally robotics but at the time he was AI. Like, oh, yeah, this robotics is interesting. So those were the main folks, and I’m sure I’m forgetting people, but those are the ones that were actively engaged at the beginning.

Q:

So what were the big challenges to setting up a Ph.D. program in robotics in an institute that’s very sort of interdisciplinary?

Ayanna Howard:

Yes, interdisciplinary is the key. So for the Ph.D. program the hardest challenge was that you have-- and even in interdiscipline you have different cultures. So computer science has a different culture than engineering, and in engineering electrical has a different culture than mechanical, than aerospace, than BME, biomedical. So those are all the five colleges. And so how do you position a degree where you have different characteristics, different culture, even different requirements that the faculty are used to. So it’s really trying to convince the faculty because academia is really driven through grassroots. It’s the faculty that get together and they push it through. Typically unless it’s a strategic initiative it doesn’t come from the presidents as “thou shalt” because faculty, we can say no and we have every right to and we do sometimes, so it has to be grassroot. So I think for the Ph.D. it was making sure that we had a good story, making sure we had a reason for this, and making sure that our graduates were the best that they could be and that they-- and we forced it very carefully and we’ve had to change things to make sure that they are interdisciplinary. So every robotics Ph.D. student comes out they’ve touched some aspect of engineering as well as computer science, and so they’ve programmed but they’ve also taken control, for example, or they’ve taken mechanics, and so we force that. And maybe they stay in computer science when they graduate, but they understand, oh, there’s other parts of the robotics equation and they’re not limited, which makes I think what’s valuable about the Ph.D. but also makes it a little challenging as well.

Earth Rovers at Georgia Tech

Q:

Great. And so what are some of the projects that you’ve led or worked on at Georgia Tech?

Ayanna Howard:

Oh, so Georgia Tech--

<laughter>

Q:

It was good at the start of it.

<laughter>

Q:

<inaudible> question we didn’t have in there.

Ayanna Howard:

So at Georgia Tech one of the differences between being in academia versus at a research lab is that your projects are your own. So pretty much whatever is in your head you work on. So when I first started at Georgia Tech I had to figure out-- what I knew was NASA. What I knew was exploration, so how do you do that when I don’t have a rover in my Georgia Tech lab, so that was the first challenge. So looking at the problems, because I know scientists, and so just figuring out what is it that I could do as an academic and still have an impact on exploration. So I actually built an indoor Mars yard and interacted with scientists and earth scientists because that’s what-- I’m here on earth. I can send rovers to places on earth, so grants, grants, grants. Grants, grants, grants. So my very first project was sponsored by NASA by the earth science division, and we were sending rovers to glaciers to understand climate change. And this was right at the beginning when there was the whole Gore book and things like that, so it was a nice place to be. It’s like, oh, rovers. We do it on Mars. We could do it here on earth. So we actually went to glaciers in Alaska and Colorado. There’s a glacier in Colorado. So we built rovers, basically hacked toy mobiles, so basically the little mobiles but child size. We hacked those, made them autonomous, put cameras on them, put intelligence on them, science instruments on it and we would-- so on the technology side is how do you create this integrated system low cost with all these sensors that’s autonomous. On the science side it’s how do you get this data and make it in a form that’s representative. So how do you create a sensor map or a data map of information that’s also localized exactly to where you need to be because scientists need very high precision, which we need in rovers, too, localization, so it was a nice blend. So that was my very first project from scratch, so it was the first time I actually had to build something from an idea. Very challenging, but at the end we had a great asset that could be used. And unlike NASA, though, it’s like when the funding runs out you either continue or you decide to do something else, so build a little bit more. But then translated that a little bit I decided what is it that I really enjoy in life, and I love NASA, I loved exploration, but it was so hard doing that, and at the end-- I mean we had cool media. We were in USA Today and things like that, but at the end I had to think about, if I could do anything-- because that’s what I can-- you can do anything in the world that you want. What is it that you want? And so I decided to revert back to my thesis and go back into the healthcare. And so toward the end of when I was working on the glaciers-- and I still do some exploration, so we send now robots undersea. In fact, I had a student come from Antarctica sending an undersea robot to do some of the mapping. So still do that, much smaller scale, but started working in healthcare and children with disabilities. So designing robots and associated technologies, assessment methods to enable robots to do therapy in home with kids. So, again, anything that you want, what is it that you want, and that was what came-- now if you had asked me exactly then, okay, what is it? It’s like I don’t know, so I’d just start to do I’ve worked a little at this, you know, kind of the ideas in my head, and that was what really hit home.

Human Robot Interaction Therapy for Children with Disabilities

Q:

And what was the first project? What did you build?

Ayanna Howard:

So the first project, we call it Playing with Toys, got a small grant, worked with actually Charlie Kemp on that one because at the time he was healthcare, not kids but he was healthcare, and so I used his equipment because I’m not sure what I wanted to do. But Playing with Toys was can I model, so can I look at as an adult or a therapist and model how you’re playing and interacting with toys and take that information and have the robot play in the same way. So that was the project, and so that’s when I got into this whole-- so I realized I’d done human robot interaction all of my life but was never classified as HRI because, you know, HRI is pretty young. But I’d always worked with scientists and learning from people and trying to include that knowledge. In fact, my lab was called Humanized Intelligence. It’s like in one of my papers. It’s like we’re doing humanized intelligence. And this is when HRI started and I was like, oh, I guess I sort of do HRI but from a different perspective because it’s, again, from that classical controls perspective and we’re using humans more as-- and I think of it as if you have a control law the human could be your input or it could be your disturbance factor. And so looking at the human as that point is how I looked at the equation. Now my perspective is totally different, but at the time when I first started was just like the human is just a box in my controller, and it can be my input or it can be this thing that’s an anomaly and I have to figure out how do I navigate and it’s just a disturbance and positive or negative, put there are some boundary conditions. It just seems so reasonable and so logical at this point, and it totally fails in the real world. So that was basically the approach, and so for a system that’s when I learned about IRBs and doing human subjects because at NASA you just ask your friends to come and, “Okay, I need you to code up this. Okay, I’m gonna take your data.” And in academia it’s like, oh, no. It doesn’t quite work that way. And so first system-- and I totally enjoyed it. I think I enjoyed having to learn something new, and so-- I mean we’re the expert and a lot of times what we’re learning is just a little deviation of what we already know. So you read another paper. It’s like, oh, that’s an interesting perspective. I’ll quote that up. But here was a case where it was entirely new. I had to understand what is this aspect of real learning from people. What is this aspect of play because I wanted to do play for therapy. So I started interacting with clinicians and therapists and actually sitting in therapy sessions with clinicians and kids to figure out, okay, what is really the problem? I have this idea in my head, looks good on paper, we got funding for it, but what is it like in the real world? And I think I enjoyed the learning something new and then being able to take that. And it was just like being back in your thesis but without the pain, so like all of the joy of discovery in doing that. And so when we finished that project I was like this is what I want to do, and so since then it’s been amazing. It’s been amazing. You see these kids. Their eyes light up. You’re actually seeing some aspects of improvement in terms of therapy using your technology. We’re now commercializing some, so I have a startup that’s commercializing some of this lab stuff, and it’s just an amazing thing to actually see that all these years what you’ve been doing can actually help someone besides Mars.

Perspective on Working with Children

Q:

Was it really challenging the first time to work with children and a robot together?

Ayanna Howard:

Oh, yes. <laughs>. You learn so much when-- so the first thing that I’ve learned is when you’re designing technology introduce it to your target demographic before it’s ready because-- and, again, this is lessons learned, so now I know. It’s like it doesn’t have to be ready. You just need to see if you’re on the right track. So I remember our first thing we were going to do was using vision. So we’re going to use-- it was the Kinect, so we’re going to use the Kinect to grab some of the data, the structure because we needed to come up with kinematics. So we had this full system. It worked perfectly with all of our human adults, which were students, and it was great. And then we took it into the first-- and of course the lighting conditions. We had already figured that out. So we go into a home and we have a child, severe disability in a wheelchair that’s sitting there very patient, and we cut on the system. There was no child in our imagery. It’s like, oh, maybe it’s the lighting. So of course you have this child in a wheelchair. We’re like, “You know, we’re gonna move you around.” We moved furniture around and all of these things. Still didn’t work. So here we are, our first subject, child with CP, very, very, very generous, nonverbal but very generous, very excited. And so we actually got it to work. The dad picked him up, put him in his lap and suddenly we saw the child. We’re like this makes no sense, right? So, again, now he’s playing the game and we’re having this little- so worked out. So then of course you’re like, okay, so why didn’t it work? So we go back, and it’s little things of the fact that typically you have typical kids. You have adults. None of them are in wheelchairs. We have them sit in chairs. A chair looks actually different in a camera than a wheelchair. Something very simple that you wouldn’t know until you-- and then there was other things that quite didn’t work the way we intended, and you don’t know until you actually have whoever it is that you want to use this system there testing it out. So, yeah, we’ve learned. That was just a little thing, but we’ve learned quite a bit. Not everything works but it’s okay.

Q:

And do the kids get overexcited about the robots, and does that interfere with your control variables or anything?

Ayanna Howard:

So the kids definitely get overly excited about the robots, but I will say that that’s okay because it works. And so what we find is we actually want the extremes because the extremes-- if our system works at the extremes it’s going to work for everything in between. So a child who’s excited who doesn’t listen, who doesn’t actually follow the nice trajectory like, “Okay, we want you to follow the robot exactly. We’re gonna measure it,” and the child doesn’t and the system still does something right, then that’s a good thing because that’s going to be your extreme. And so one of the things that my lab practices now again is early introduction of the technology to the child demographic and there are no rules, i.e. there’s a rule in terms of the protocol, but there’s no rules on what the student does. So we give general instructions like, “We’d like you to do this,” but there’s no correction like, “Oh, can you redo this?” because this is what it’s going to be like in the real world. Our systems now, they work. I mean, our latest, we have a robot that interacts with kids, children with autism as well as difficult kids, with a tablet and we’ve taken that robot to about maybe thirty different places, over one hundred individuals and it works every single time. I could just bring it here. I can call and say, “Hey, let’s bring the robot, let’s cut it on.” The only thing, that sometimes the battery wasn’t charged and that’s the max, but we make sure that it works all this time because again, that first instance with our first child, you don’t know how-- you’re just like, “Oh, my gosh.” And they’re so patient and the parents are so patient because it’s something that is needed. So it’s not just a nice, novel idea or it’s like, “Oh, this is a beautiful equation.” No, this is actually a need that is wanted and desired in this target demographic and so as a researcher, you can actually make a difference with your robotic technology. So parents and kids are totally patient with robotic researchers, when it-- even when it doesn’t work quite right.

Q:

And these are primarily diagnostic or therapeutic?

Ayanna Howard:

So we don’t do diagnosis because we are not medically trained, but we do provide the data so that clinicians can actually come up with an assessment, and we do therapy. So there’s both parts of the equations. So we’ve done some evaluation. In fact, we have a paper out that shows that our evaluation that the clinician can use is equivalent to their standard measurements. So they typically have basically tools where they can do diagnosis and so-- and they actually have to do and they do measurements and do all these things and we have shown that our you just have this follow directions are not and we give the clinician the same type of data for her or him to make the evaluation as the standard clinical tools. So that’s one and then the therapy is in the home, can I engage the child in moving so that they can do repetition, repetition, repetition?

Communication Device for Children with Cerebral Palsy

Q:

So tell us a bit about your startup and what that experience was like.

Ayanna Howard:

So another learning experience, brand new, it’s like you have to learn everything. I think that’s what excites me, which means in ten years it’ll be something else. So NSF has this program called I-Corps, which is basically NSF gives academics a lot of money and again, three years, maybe four if you extend it, but you basically-- three years, four years, and then as academics, we’re on the next grant and then we’re on the next grant and hopefully we can have a continuation, but sometimes that’s just not possible. But typically, most academics, it stays in the lab. So you do it and we get some beautiful papers and our citation index is high and everyone knows your name, but if you go out and you pick off just some random person, it’s like, “Oh, do you know what X has done?” They’re like, “No, but I saw that robot on TV” and they’re like, “Yeah, but that’s fake, that’s not real.” And so what, basically NSF was like, “We need to get these academics to get this stuff out of”-- because it’s great stuff. I mean, there’s so much stuff I see on TV out there, I’m like, “That’s crap, you know that so and so is doing this,” but it’s not getting out of the lab. And so the I-Corps is basically take academics and start looking at your research and your technology to see, here’s something that can translate.” And it’s not about academics, go quit your daytime job. It’s not about that. It’s just, can you translate the technology? So participated in that and our technology, and it wasn’t even a robotics technology although we are now transferring one of the robotics technologies, it was basically an assisted technology to be able to interact with the robots. So these are children with CP. So primarily, it was children with cerebral palsy with motor disabilities, how do you communicate to your robot? And we’re using tablets, how do you communicate to the robot through a tablet? Now if I have a motor impairment, pinching and swiping is not necessarily-- it’s difficult, very difficult if not impossible. So how do I provide that? So we designed a device that allows a child to use their joystick or use switches or whatever access point they have, whether it’s their head, to control their tablet and they control their robot. So that device is what we went through the I-Corps program and so from there, what we found was yeah, it wasn’t about the robot. People were like, “Oh yeah, that’s cute,” but the tablet, that interaction, being able to-- the thing was, “My child can use this to navigate YouTube?” “Yeah.” But no, here’s the robot, right? So what we learned is that that interface was actually something that was a good thing, so that was the first technology that we then translated. So process is how do you design a technology that works perfectly for you in your lab, and outside but you’re still-- there’s a grad student or a professor that’s still there, so if anything doesn’t work, like “Oh wait, just a sec, let me reboot” or “Oh just a sec, let me do something.” How do you take something like that and provide it so that someone basically opens up their box from Amazon, they plug it in and it works every time and like, “Oh, this is obvious”? That’s translation is probably the hardest thing to do, is making it so that it’s so robust and it’s so-- it’s just like your iPhone. You don’t expect your iPhone to come not charged. It’s just you get it, you cut it on, you transfer your number and it works. If it doesn’t work, then it’s like, “Wait, this is not reasonable.” So how do you get research to that stage? It’s a different way of looking at things and thinking about things.

Startup Company: Zyrobotics

Q:

What was the name of the company?

Ayanna Howard:

Zyrobotics.

Q:

So do you primarily do the software or you’re doing some of the manufacturing?

Ayanna Howard:

So we do it all. And when we say-- we do the manufacturing. So we have an electronics manufacturer, but we did our-- the whole bomb or we did the deliverables, the CAD and so basically they have a list of the parts and so they put it together now. The very first few, we actually built in-house with a very talented tech that could solder and then we have some basically software both for the device and we have some educational apps as well, math apps, and again they’re all designed to be-- so a typical child can use it, but the purpose is to enable children with disabilities as access to these games and math education and cause and effect for therapy and aspects such as that, and then this robot is actually coming on the scene.

Q:

So how many people did you start with and how has it been growing?

Ayanna Howard:

So right now, we have seven. It started with volunteer folks. <laughs> So it started with myself and I have another founder who is actually a person who has translated some Georgia Tech-- he’s worked with some of the Georgia Tech faculty, so he’s basically a serial entrepreneur. And then my grad student, who’s now-- she’s now my post doc, but she graduated. The device was her thesis and that was basically who we started with. And so now we have seven employees, so we have a CEO, we have a full-time programmer, full-time graphics person. So we have kind of the team that makes it move forward without me.

Q:

So you can give back to more research.

Ayanna Howard:

I can give back to more research, although I’m so-- it’s like I’m still actively there because one of the things we’re doing is-- so the whole interface, which was there was this robot that was part of this that kind of somehow got disappeared, so now we’re actually maturing that to translate it. But again, talking to the targeted-- it has a different focus than what I originally anticipated but that’s okay.

Q:

Is it something you can talk about?

Ayanna Howard:

So it’s a robot coach. It’s a robot coach.

Impact of Zyrobotics on Students

Q:

Would you do another startup?

Ayanna Howard:

Yes, I would. I definitely would. So one of the things that it’s done, even for my students, is that they look at their research differently. In fact, sometimes I have to scale them back. It’s like, “No more, let’s write this up so you can graduate.” But my students are actually, when they look at their research now and when they’re doing their subject trials, they’re actually looking at it for societal value, as well as, “Is this a good chapter in my thesis?” So it’s starting to change how they look at things as well, how they look at research, which I think is fabulous because that means when they finish and they graduate, they’ll start thinking about, “Oh, I’m designing this new gadget, I’m not going to keep it in the lab, I’m actually going to make it come out and have it be of use to someone else besides myself and my team.” That’s nice. I really like that. Again, I have to scale them back sometimes, like “No, no, no, okay put that underneath, let’s work on this part and that will be good product, but let’s not push that right now.”

Q:

Is that something that you’re able to convey to your other students who aren’t necessarily working on this project?

Ayanna Howard:

Yes. So I have one student. So again, I’ve continued on some of the more exploration, so I have one student who’s working on human robot trust for emergency evacuations. So definitely relevant and the way kind of starting is, is like, okay, when you start it before for I-Corps, it was like, “Okay, this is what I think emergency evacuation is.” After that, I was like, now he’s gone to talk to the Georgia national rescue agencies, he’s gone to talk to the Georgia Tech police. He’s actually been gone now to talk to the target to say, “Does this make sense? How should I be thinking about that?” Whereas before, it was, “Oh, this makes a good”-- and he’s got-- even before that, I mean he’s gotten a ton of papers, so it’s good stuff. But now he’s really thinking about how do you make it relevant? So that’s one example. The other student I have who’s doing underwater, he’s doing underwater human robot interaction with the goal of helping underwater divers, say, who are doing scavenging, scientists who are doing things at the coral reef. How do design a robot that can assist them? And so he’s actually gone to divers and started to communicate with them. So yeah, it’s creeping in.

Growth Path of Students & Colleagues

Q:

Who are some of your PhD students or post docs that have gone on to other things?

Ayanna Howard:

I have one that I share. So Sekou Remy, who’s faculty at Clemson, he’s doing basically crowd based robotics for education. I have another student, Douglas Brooks, who’s at SwRI. He can’t necessarily talk about all the stuff he does, but it’s in robotics. I have another student who’s at Lonnie Parker who’s at Newport. So he’s at the- NUWC underwater, naval, underwater, naval something something. It’s in Newport, Rhode Island and so he’s doing things in control and some aspects in underwater concepts, trying to get more into underwater. I have another student who was at NASA, Brian Smith, he’s the one that was shared and then he recently-- and don’t quote me because I don’t remember. He recently left but I don’t know where he went. I don’t remember where he went, but he left maybe a year ago. Yeah, maybe about a year ago. Steven Williams who is at a startup, robotic startup, either in Boston or Pittsburgh. I don’t remember what the company’s called. If I forget anyone, they’re going to kill me. <laughs> And then Hae Won Park, who’s actually my post doc, but she’s also co-founder of Zyrobotics. She’s actually interested in academia, but she’s torn. She’s like, “I’m going to go into academia because I think it’s just the fabulous job,” but then she’s like, “But I have Zyrobotics.” So she’s a little bit torn but we’re trying to push her out of the nest.

Q:

Who are some other people you’ve collaborated with?

Ayanna Howard:

So Charlie Kemp, who’s at Georgia Tech, collaborated with Magna Zeggerstet, who’s a control person. Another faculty is Patricio Vela. He’s a controls manipulator guy. He’s heavy into vision. So collaborating with him on looking at metrics for kids and how do you extract that with standard cameras and not the Kinect. So we’re so reliant now on Kinect but it’s not the answer to all. There are still cameras. We were using cameras before Kinect. There’s some value there. So his domain is in the camera space, stereo cameras, things like that. Other people I’ve collaborated with, more on students, Frank Dellard, who’s a SLAM guy. He’s a hardcore slam vision person. He did GT SLAM, so that’s the big thing now, is Frank Dellard. Outside, I’ve collaborated of course with my previous advisor, George Becky, did some collaboration, NASA folks, I collaborated at the beginning, Isa and Larry and Isa-- Adrienne Stoyka is another one who’s neuromorphic robotics. Yeah, <laughs> which is yeah, he does some neural stuff in robotics and he does things like BCI and just kind of hybrid sort of stuff. I’m trying to think of who else. And then a bunch of scientists, but yeah, I think those are the robotics folks.

Funding for Projects

Q:

And your grants have come primarily from what sources?

Ayanna Howard:

Well, big one right now is NSF, National Science Foundation. Some, I would say surrogate NIH. We’ve gotten some money from foundations that are funded through NIH, so not directly through NIH but foundation funds, grants. So that’s for really the studies. So not necessarily development, but for some of the studies and the assessment and that part. Yeah, primarily NSF and then these foundation grants.

Q:

When you were at NASA...?

Ayanna Howard:

That was NASA, yeah that was NASA and even the first couple of years, that was NASA funded, so that was one of their large grants, which if you can get a NASA grant, it’s quite nice. It’s like two NSF grants.

Other Activities

Q:

Maybe you can tell a little bit about your involvement about in the IEEE robotics motivation society or the HRI community.

Ayanna Howard:

So what I would call public service and service, so primarily for IEEE, it’s been associate editor of X. So for example, I’m the-- in IEEE SMC, they have a human-- they call it human animation but I’m the human robot person. So anytime the papers come, I have to find reviewers for anything HRI that comes to the SMC journal. And then for RAS, the one that’s coming up, I’m running our first ever PhD forum. So we’ll have a PhD forum at the conference this year, which will be good, and that’s because I’ve ran forums at AAAI, focused on robotics. I’ve ran the PhD forums at actually AAAI for two years, yeah AAAI for two years, and then I’m running the PhD-- I’m in the PhD forums-- the PhD forum for CRAW, which is focused on females, so getting more females interested. So I’m co-chair of that as well. So education is really, really the name of the game. Other things in IEEE, I’ve organized some of the tutorials, tutorial chair and I have to look at my CV over the years and they’ve been in different-- so it’s not just RAS, it’s been-- so there’s RAS and SMC. So I think both of my homes, RAS more so but SMC as well. So SMC has a number of roboticists. So this is before HRI. So what happened is when HRI was in this gray area, IEEE RAS was like, “What is this HRI stuff?” and this is before the HRI conference. And so you would publish some things like RO-MAN, which was a big venue, or SMC or BioRob and things like that, so you kind of had the RAS family but then you had to find something else to get-- now we have HRI, so it’s good. So you still have that little bit of a family. So other service besides the associate editors, organizing tutorials. Yeah, it’s a bunch of different things.

Diversity in Field of Robotics

Q:

So you mentioned one of those was trying to get more women in these PhDs. What’s been your experience as a woman in robotics and how do you see that growing?

Ayanna Howard:

So I will say in robotics, there are more women in robotics in academia than there is in-- because my home school is engineering controls, so there’s a big difference there. But if I look at-- there’s still not enough and it’s because I can-- if I can count pretty much and know all of the female roboticists that are in tenure track, then we don’t have enough because I can say, “Oh yeah, there’s da, da, da, da, da, ten.” After a couple of hands, I’m done, but I know all of them, not enough. And so it doesn’t make sense because if you look at biomedical engineering, they are right now about half-- in terms of who’s coming in, and this is even in grad, it’s about half and half. And so there’s this whole thing, well with robotics, it’s like navigation and localization and controls, this is not interesting, whereas BME, it’s a social value. But out of anything, robotics is really-- I mean, that is the social value of anything and so there’s a disconnect. There has to be a disconnect because if I look at the students that are in my class, they’re doing robotics, if I look at the BME folks, what are they-- I mean, half of the females are doing something in, “I want to design something that can be injected into and is”-- I’m like, “That’s robotics.” “No, no, no, no, it’s medical.” It’s like, “No.” So there’s a disconnect about the message we’re saying and giving and so I think it’s like robotics of anything has societal benefit. You can go out and say, “Okay, what is a robot to you?” “Well, like robot X.” So everyone understands that, so why is there a disconnect if I look at the people who are creating this and yet what is the expectations? And so I think there’s a disconnect and the reason is, is because they’ve done studies for this, they’ve shown that-- and even as I learned this from the disability community. If you have one type of viewpoint, looking at the same thing, you’re going to get the same answer, which also means you’re going to exclude everyone else that is going to be part of that. And so you really need the different viewpoints in order to come up with a solution that is really universal and so the reason why-- it’s not just to have-- and I always say yeah, it’s not just because we need more women. That’s not the point. The point is is that you need a representation that reflects the society, period. If you’re creating technology for society, you should create technology by folks that reflect society. So if there were no women in society, then I would not argue for having women in robotics, but the fact is nature kind of gives this divide as fifty-one, forty-nine or some kind of thing, so it should at least reflect close to that. So that’s really why getting more females-- and it’s just getting that different perspective and they’ve shown this in the medical world, how they had technologists and the things that came out when they started introducing females, you were like, “Oh wow, it’s actually better.” Yeah, even though they may have said, “Oh, you need this for this,” it was like, “Oh, but you know what, everyone really wants that.” Disabilities, curbs on the sidewalks, everyone uses it, your grocery cart or your suitcase, well guess what, it really wasn’t designed for typical, it was designed because people had wheelchairs and it was likes, “How do I get from this curb across the street?” But yet everyone is just like, “Oh yeah, curbs, this makes perfect sense.” It’s the same thing.

Q:

So how do we get more women in robotics?

Ayanna Howard:

I think in something like this, giving the message that the robotics is really about society. It is creating-- it’s not just about the best SLAM algorithm, although that’s important and it’s not about the best vision algorithm that can detect information that’s important. But it’s also about creating robots that can be used to help the elderly or that can be used in education or that can be used in the surgical operation room, which it exists but really getting that message out that yeah, it’s a good thing. And you do need creative thinking and you do need designers and you do need programmers and you do need engineers to make it happen.

Q:

Diversity issues more generally, so not just about women.

Ayanna Howard:

So if you look at just diversity in general, so robotics is part of the larger problem of STEM engineering, which by default is just we have a hard time. It’s not just about its diversity, it’s diversity on gender, it’s diversity on minorities, it’s also diversity on terms of domestic versus international. It is a problem kind of in the U.S. definitely and so I think that message of societal value addresses a lot of those issues. I think the concept of it’s not just engineering, it’s not just about equations, although that’s part of it, but it’s about making an impact and making a change and I think that message will affect- I mean, and they’ve shown studies. So for example, Hispanic students typically have a very strong family culture of value. Talk about how robotics and engineering can address these things. In fact, they have some programs that engineering for Native American communities, how understanding engineering can help your community. But it’s usually one or two people that give you this message and they’ve shown changes but it’s just one or two people and I think as a society, we need everyone giving that message and not just me, it’s like “Oh, but it’s because you, you just want people that look like you.” No, I think everyone has to be part of that equation because I don’t think we can keep going like we’re going and come up with beautiful technology until there’s more diversity of thought and it really is-- so it used to be I was like, “Yeah, there’s not enough women and there’s not enough minorities,” it’s about the diversity of thought. And I will tell you, if I’m in a room full of females, you still need diversity of thought, period. It’s just you shouldn’t have a majority of one coming up with solutions that are going to affect the entire world. So it really is, if I have a bunch of folks that all are making over one hundred thousand dollars, you’re going to have a thought that’s based on that and then you’re going to have these very expensive systems that nobody can afford. So it’s really about the diversity of thought and making sure that that is always the message and you have good societal benefit when you do that. I’m sorry, I got on my high horse. <laughs>

Q:

There’s a perception problem of robotics, but there’s also potential institutional issues of allowing that diversity to express itself, so in term of what kind of work might be valued or that kind of thing.

Ayanna Howard:

Yeah, it’s a systemic problem. I mean, there’s been recently with the tech startups and there’s this big hole, “Here are the numbers” and “Oh my gosh, this is really bad” and then you have people going, “Yeah. But the pipeline’s broken but not even that, it’s like when you enter the pipeline, then it’s just like, “Why am I here? This is horrible. Why do I have to fight when I go to work? I’ll do something else and make more money.” So it is a problem. I think what’s nice now, which I hadn’t seen in the past, is that people are now open to the conversation. It used to be just like, oh, people in backdoor rooms, but now people are open to the conversation at least, which is a-- that’s a difference. That is a difference and it’s not a-- I grew up when they were talking about affirmative action and then it was just like, “Okay, we’re going to force it.” And so it was like, “Oh yeah” and then as soon as that was-- it was like, “Okay.” But now, it’s like there’s no forcing function, there’s no “Well, you have to have X percentage.” People are saying, “Hey, we’ve got a problem” and that conversation, I really expect to see change, not like the past where it was forced. I think because people who are in leadership positions are actually saying, “This is a problem and I am not female and I’m not minority and I’m saying it’s a problem.” I think it changes the conversation, definitely.

Advice for Perspective Students

Q:

What’s your advice to young people who might be interested in a career in robotics?

Ayanna Howard:

So if you’re interested in robotics, this is the time and this is the place and so imagine if you were at the place where you could have been employee number one at Google or employee number one at Microsoft or employee number one at Apple, that’s where we are now with robotics. We are at the employee number one through one hundred and so robotics, I think right now, is going to be-- is going to take off. I think we’re right before that little elbow where in fifteen years, people are-- like now, cell phones. I mean, if you don’t have a cell phone, it’s like, “Is something wrong, did you lose it?” I think it’s going to be that kind of thing where people are like, “Oh, where’s your robot X? You don’t have a robot X? Why not?” I think that’s where we are in the next-- and it’s not like fifty years. I think it’s in the next fifteen years. Now it may not be my robot maid, it might be my robot car that only gets cut on during rush hour, but everyone has their robot car during rush hour, so now it’s no longer rush hour. It’ll be things like that where you’re not even going to think that this was novel and then our kids are going to grow up and be like, “You didn’t have cell phones? You had to use a payphone with money?” I mean, they’re going to be like, “You didn’t have a robot?” So I think that’s where we are. So it’s a really good time to be in robotics, to be part of the inventors of this new world.

<off-topic conversation>