The world-famous NAO humanoid robot just got an upgrade! Check out what is new with the most widely used platform in robotics and how this can impact your classroom or research lab!
Video transcript below.
Peter Bowman:
Hello everybody, thank you so much for joining today's webinar. My name is Peter Bowman, I am your host for today's webinar; Nao Robot in Higher Education and Research. Today's webinar is brought to you by Robot Lab. We have several people here from the Robot Lab team that is collaborating to bring you today's webinar. Again, I am your host account manager higher Higher Ed specialist Peter Bowman. Also joining me today is Elad Inbar he’s joining us from Dallas Texas, while I am in here in San Francisco California.
And then we also have Maria Alejandra, who is joining us from Bogota Colombia, so it is an international effort today, thank you so much for all of your efforts. Maria, today's webinar would not be possible without you. Then we also have Doctor, our very special guest Doctor AJ Aluri from West Virginia University. I would like to start by humbly apologizing to Dr. AJ Aluri for referring to West Virginia University as the University of West Virginia in the initial invite email. That was my personal mistake in a rushed effort to put that invitation out to all of you. So again, my humble apologies to Dr. AJ Aluri, but we certainly are excited to have him here today, and bringing you his exciting research, and I know that he's also excited to tell you about all of that. And then…. Alright. For today's webinar, we do have an agenda today. We will be covering a host of topics.
We've just kind of introduced our webinar; this is the third installment of the Robot Labs Summer Webinar Series, this is again the now higher educational research, there will be one more installment. Unfortunately, we have had to delay the final installment of the Robot Lab Summer Webinar Series that will be on quadruped robotics, but there will be more on that to come. Today, we're going to be talking about again, the NAO Robot, we will be covering Nao as an autonomous agent, in human robot interaction, in different forms of research, in autism therapy research as well, more specifically, we will also again be joined by Dr. AJ Aluri, and he will talk about his research and his experience with the NAO robot over the years.
Our CEO Elad will be talking about the SDK and API for further NAO robot, he'll be going into not only choreograph which is the most popular thing used by people looking to program NAO, but also other available SDKs as well. At the end, we will have a Q and A session with not only our Robot Lab team, but if you have any questions for Dr. AJ Aluri, alluding to his research, then feel free to also go over that when the Q and A session time comes. Aluri I'd like to pass today's webinar over to you so you can talk a little bit about Robot Lab.
Elad:
Thank you Peter, thank you everyone for joining us. So, just a little bit of background overview about Robot Lab. We were founded over a decade ago, I started the company actually back in Israel. I'm Israeli if you wonder about my name, my accent. And moved everything to the US in 2011 and a little known fact; my current journey with robotics actually started because of now. As a child, I always loved robots and build them and play with them. But robots were toys for geeks for too many years. You had to solder them, work on electronics and everything. It's not for everyone.
And around 2010 or so, the first version of NAO was publicly available. This just started a whole new journey for me. Basically brought my old passion back, and basically started a Robot Lab in a way to help educators and researchers in bringing to life all these concepts without working on the electronics. This is something that you'll see today in this webinar. Now it’s so versatile and so capable that it's really, really hard to compare it to any other platform that is out there, and we'll talk about that later. We are based in both California, San Francisco and in Dallas Texas, as Peter mentioned. I'm personally in Dallas, the rest of the team, we have team here.
The rest of the team is in California. We are working with tens of thousands of schools; we are working with everything from kindergarten level, all the way to college level, and working with schools and universities globally. We have served almost every school district in the country and almost every university in the past decade plus. So, pretty good chance that we are already a vendor in our university. Our educators community is pretty strong and growing; these are all educators and researchers that are passionate about robotics, about bringing robots to the public and very much aligned with our mission statement and our passion which is, making robots smart and useful and that's what you guys are also trying to do, and I want to thank each and every one of you for your contribution, for this efforts because, we live in 2022, and I always say we are already 20 plus years into the 21st century, so it's about time that we get robots and all of these technologies into the hands of our students that will graduate in the 2030s.
So this goes without saying. And last but not least, what makes us so special and why should you even listen to us about NAO robots? We are a sophomore robotics exclusive partner when it comes to both NOA and pepper robots. We are working with them as I mentioned for over a decade, we have close to 6500 robots deployed out there, so you know 6500 NAOs are out there, our team is responsible for everything from pre-sales all the way to repairs. We have technicians that repair the robots if something happen, our professional development will help you go through that and start your research or your teaching career with that and this is what our team is passionate about.
This is something that no other company out there have, including software robotics the manufacturer and I'm very, very proud of the team and all the efforts that we put behind these amazing robots. So, that's a little bit about Robot Lab, and what we do. Let's jump into a couple of very interesting research fields that I think can really, really emphasize where this robot with flexibility and the accessibility of this platform can help you boost up your research. This is a picture from a video, we decided not to play videos during the webinar because the quality won't be good but we'll share with you the videos later on. This is a picture from a research that one of our customers, a professor from UC Bakersfield is doing.
This is a research that is designed to help disabled people to interact with the world using robots. You can see the student here is wearing one of these brain wave helmets that are designed to capture all the brain waves that are going on in our brain when we think about things. And the research is basically designed to capture different thoughts, so the student need to focus on something, let's say you look at the screen over there, we have different pictures, different videos. one of them is the opening a door. So, the student needs to focus on that action and eventually, you see on the bottom right, the robot goes to the door and open the door. Can reach out to the door and open it.
If you want to grab something, you can focus on another action. You create another pattern in your brainwave, and the robot can actually do that. And this is something that is a very, very hot topic right now in assisting healthcare and aging population, because we don't have enough caregivers. I mean I'm sure you all read about that in Japan but we're nearing this point in the western countries as well. Especially because of ( 9:54) it depends on how you want to look at that. A lot of people decide that they don't want to work in assisted living any more because of all the restrictions, and everyone needs to be, these are the most sensitive populations, and the restrictions on the employees of the assisted living facilities are so great that people just don't want to work there any more. So, finding ways for technology to help these people even if they have dementia, if they can't function, if they can't even… let's say they are on a wheelchair or something, and they can't reach out to other objects, and have different types of robots and different type of technologies helping them in their daily life is something that is really, really important. Especially now that these residents are living assisted facilities and without any help. So, this is one of the examples. Everything was done using the now API and SDK which I'll share with you later and I'll show you later how it works but, design just to show you the one of the ways that you can engage and interact, create engagement interaction between humans and robots.
Peter Bowman:
Thank you so much for that information Aluri. Another really exciting way that now is being used is as an autonomous agent. This is a hot button topic in higher education right now right? Robots as autonomous agents and there's no more appropriate way that we can explain or feature NAO as an autonomous agent than the robocup soccer competition right? This is a global competition that's between universities from around the world. That take these NAO robots program. Then to play soccer you have to write a dissertation in order for your team to even begin to start to qualify for robocup and it's robocup's stated goal that they would like for the robot athletes to actually compete against human athletes by the 2050 world cup which is an ambitious goal, I would say right? But I think that this is something that lots and lots of people would like to see and it would certainly capture the attention of everybody if this was something that they were able to achieve. Cup is a competition that brings great prestige to not only the winners, but also just the participants because, there is a limited amount of people that are selected to participate in robocup and so, this is something that Robot Lab is very excited to support. We have a very close relationship with the president of robocup and the competition as a whole. This is something that we continue to endorse and support a lot. I know that yeah…
Elad:
One more thing on the robocup, if you can go back one second. So one more thing on robocup that is very, very important to understand the robots, you see them here playing soccer but the robots are totally autonomous. No one controls them or anything like that. That's basically the autonomous agent problem. And you can apply the same problem to anywhere, from autonomous cars that drive on the highway, need to make decisions by themselves based on either other humans or other autonomous cars without communicating with them. Just by getting cues and clues about what's going to happen. And this is a very, very hot topic right now. Using NAO robots as a medium to solve this problem to augment that problem is a great example. And one thing that is very, very important the way that the collaborative work between universities from all around the world works here; each year, universities from all around the world compete here. The winning team at robocoup here is basically releasing every year. They release their code for everyone to start the next year competition from that level. So every year, the whole industry, all the participants are getting better and better because they're getting access to the best performing team code, which is something very, very unique when you think about the collaboration between universities, we are servicing almost all the members of robocup. All the universities and we know them on a personal basis so we can definitely help you and introduce you. I'm sure you choose to participate in this amazing competition.
Peter Bowman:
And furthermore, a lot just kind of further elaborate on what it is that you're talking about. I mean think about all of the decisions that you, as a human make during a soccer game right? Soccer, or football in most parts of the world, it's the world's game right? So a lot of you are probably familiar with the sport. The amount of decisions that you have to make, whether you want to shoot the ball, pass the ball, try and steal the ball from a competitor. And the fact that we're able to make these robots do that is a truly amazing feat.
Elad:
Personally, I have one choice to make which is, go to rest, take a cold coke. I'm not the player but yeah, I totally agree.
Peter Bowman:
Absolutely. And thank you for that Elad And just moving on Elad I know that you, one area that is particularly a passion of yours is NAO as a research tool in autism therapy.
Elad: Yeah. Another research field that has picked up a lot of attention in the past few years is the ability to help children with different…. let's call it the spectrum, right? It doesn't have to be totally autistic or anything in between which is basically the ability to help them convey their thoughts, their feelings, through a robot. Something that I heard from one of the researchers that really stayed with me for a long, long time is that, when you hear about how to treat a student, students or kids with autism, even adults, it's really, really hard to penetrate this shield that they created uh from the outer world. This shield is basically designed to protect them from unexpected behaviors. You know we are humans right? We have body language, we are moving all around, we have different tonal voice which stresses them out. They prefer the predictable known feeling and feedback that they get from the world. I'm sure you all heard the stories or watches videos of someone with autism just bumping his head in the wall without stopping. The reason they do that is that, it gives them the confidence, because they know that the first hundred times they bump their head in the wall will feel exactly like the next one, so they know what's going to happen. Communicating with adults is something that is very unpredictable and it causes them to shut down and basically close all the communication. However, communicating with the robot creates this predictability because, the robot will say the same thing, the same tone, it will make the same motion, it will always, always do the same behavior exactly the same way, the same pace. And this opens up kids we have heard from researchers that kids that have never talked to their parents eventually opened up because the robots was leading the communication. We have videos we can share that with you. Parents who are stuck in tears, he never said a word to us and eventually, the child is able to talk, is able to convey what he feels and what he wants. It's through a robot because he trusts the robot. This research is taking two prongs; one is to help students open up to adults, to the world by teaching them everything from feelings, to right and wrong and everything round that. And the other approach, which one of our researchers, one of our customers in a UC Northridge, Professor Ali Kazami. He took his… basically help the therapist to learn how to interact with the autistic child. And what she told me is something that really boggles my mind. The therapists are going through two or three four years of studies without basically seeing or touching a single child. They cannot because they're not licensed. And the first time that they'll see a child is the day that they basically got their certificate. And this is something that is wrong because we all learn from our experiences and from practice. So what she created, she and her team, which they created is basically a software for the robot to act as a child with autism. So, if you don't like something or maybe the robot doesn't like something, it can go into tantrum and the therapist students need to basically calm down the robot. And you need to know when to touch the student or the child, when not to, what to say, when to say that. And this is using the wide array of sensors that the robot has cameras, microphones and so on. So, the software is designed to help the future therapists to under understand how to deal with student in internship. How to respect the boundaries, the distances and so on, in a way that help them become better therapists later on when they get their license eventually. And again just another example; how the same platform can be used in a different way.
Peter Bowman:
Fantastic Elad thank you so much for that. I would like to just further elaborate on that by saying that one of the biggest things that we pride ourselves here on Robot Lab is, being platform agnostic and being able to offer different solutions for specific areas of need right? And so specifically, relating to autism research, we do offer several software solutions to assist in this type of thing, and so, if you are interested in a consultation associated with this type of research, feel free to reach out, we have a solution for you. To go into another way that NAO robot is being used that's in a human robot interaction. This has been highlighted by the fact that global international brands have chosen the NAO robot to be at the forefront of the robot age in hospitality, right? So, we're talking about brands such as; the Hilton Hotel, IBM, Marriott Hotels, have actually used the NAO robot in their hotels to greet customers, talk to customers, answer frequently asked questions etc. And this is a very exciting thing because, you know robots, they're not here to replace humans. This is a very common misconception that's thrown out there by people that are sometimes afraid of this advent of the robot age, right? This fourth industrial revolution that we are faced with today. Robots are ultimately here to assist humans. We are hundreds of years away from the possibility of robots replacing humans. I assure you, the decision making and just a personal touch of humans, they're not going anywhere, but the fact that these international brands in hospitality such as; Hilton and Marriott and more have recognized the advantages of using a humanoid such as the NAO robot at their properties is something that's very exciting and that leads us into our very special guest, Dr. AJ Aluri.
Elad:
Just taking on the human robot interaction. In all these applications, we are connecting the NAO to a bigger brain such as; Google dialogflow, IBM, Watson Alexa and so on. We want to bring knowledge that is out there in a very personable way through a robot. And this is something that all this research is all about. How do we bring this knowledge that is available in all these chat bots in a way that is easily digestible and consumable by people, which is speech? That's how we communicate, right? So robots, and especially now, is a great tool to be the face in front of a chatbot that you develop on the background.
Peter Bowman:
Absolutely. Thank you so much for that elaboration Elad I appreciate that. But that leads us directly into our very special guest, Dr. AJ Aluri, from West Virginia University, he's a professional professor there and an internationally recognized scholar, speaker and thought leader in the fields of hospitality innovation. And so with that, Dr. AJ Aluri, please take it away.
Dr. AJ Aluri:
Thank you peter. I hope you all can hear me. ( 24:36) and Peter, thank you so much for having me. And I always had a great time working with your team and glad to do this. Thank you everyone for attending this webinar. I know you're taking your valuable time out of your busy schedules right after the long weekend. So I'll make this a more insightful and more exciting for your time being here. Let me share my screen very quick, I do have like a few slides that I want to share. Let me make this a full screen here.. yeah! hope you all can see my screen right and hear me clearly? Good! Awesome! And again, my name is AJ, I'm a founding director of the WVU HIT lab, it's hospitality innovation technology lab at john chambers scholars advanced economics, and this is my hashtag as well as my.. if you want to tag me, here is my handle, the social media handle for all my channels.
So, I'm gonna make this very straightforward in terms of all the insights that peter and Elard gave you guys. I think that's really awesome that you know the potential for it. But I'm going to talk about two major trends that's happening, post-covid and how we're using NAO in the lab. I'm going to share with you research insights with data we collected between 2019 and during the second phase of Covid, which is 2020. And again, a few insights that we are doing more than what we've done in the past. We also used a brainwave device moto insight in order to collect data, we have a lot of data on that as well. We haven't published or presented that yet but, I want to do that sometime in the future. Let's get started. Here are the two major things I think we need to look at; this is an early study done during early 2020, first phase of Covid.
What we saw was the business adoption accelerated like seven years ahead during just 2020. So in other words, a lot of businesses have adopted technology which would take like seven years just happened in the year 2020. and in North America that number is six years. And this is a study done by McKinsey & company, and they collected that across all the businesses, most of the businesses, thousands and plus. And they found out that North America is six years ahead of the game, in terms of acceleration. If you go back to consumer adoption though, the consumer adoption has gone for three years. So, it's accelerated three years in terms of digitization of customer interaction. So, it really took us three years ahead of the game when it comes to consumer interactions as well as the technology adoption. And the North America is the same of three years.
Asia Pacific is the only one that had acceleration up to four years just during covid. I mean I'm talking 2020. And I think most of these numbers probably stayed the same, if not increased in some aspects of it for instance, most of them that did digital, the apps in order to do delivery and pickups have probably reduced a little bit retail side. But when it comes to the service industry, from interaction standpoint, it actually increased for example; mobile apps. Mobile check-ins have increased significantly after 2020, right? So, let's look at few things other than technology adoption acceleration. We're also looking at staffing issues, right? According to US travel, a total of 2.3 million jobs are lost so far. This is a study done in May 2022 at recent numbers. 1.4 million of these jobs are basically generated business travel and business travel is not back yet. Leisure travel really picked up very well, compared to pre-pandemic, but business travel is not there yet, and international travel is not there anywhere close. And then, the research shows that it could take another three years 2025 2026 to get back international travel.
So, 97 percent of the service respondents of an AHLA research found that there is a staffing shortage, right? The most critical area is housekeeping, we all know that, but there's also a lot of challenges with concierge services. People, there's only front desk that's only limited staffing, they don't have a concert service anymore, they've taken away a lot of jobs, only few people are efficiently working and being productive in the hotel industry. So the service industry in general, if you look at it, this is one of the areas where we see the trends before and after Covid 19. What we're looking at is from a consumer side of it, there's the pent-up demand, there's no question about that. There are still people not travelling to their favourite location, international for instance. Not going on business trips on international.
There's a pent-up demand of all of that. But on the other side, there's also high wage growth across all positions. I mean hospitality, on average now has gone up 20 dollars for ( 29:26) which is the highest so far. And I'm talking on average in the US, but I think in some places, it's more than that. So, a lot of my students, obviously young people, young students are looking for flexible job opportunities, right? So we talk about new travellers which are digital nomads that are really ( 29:41) during Covid people who wants to work remote. There's a lot of focus on safety, but also smart devices.
There is a huge shift on automation when you look at service industry. And because of this new shift in automation, there's going to be new carrier pathways and opportunities. In other words even though you are a product-based, service-based company, the chances are you want to build your technology platform and build your products and services on top of it. In my research, my philosophy is you want to be high-tech in terms of your culture and foundation which is basically good at face-to-face interactions if you're in hospitality and tourism, and build your high-tech framework which is your technology framework on top of high-touch. So high-touch is going to be very important in terms of your culture and how you do things from a foundation standpoint and build on top of it. And that's where now comes in now comes in; now comes in where you built a great foundation on high touch and you built a high tech framework on it in terms of automation and so on and so forth.
So let's look at now at WVU HITLAB; we launched HITLAB in 2018; lynx was our first humanoid robot, we just got it just to test to explore the robots in hotels and we quickly understood that we need a programmable service robot; we need a robot that actually does more than just controlling from an app and having only a few features that's complete autonomous; which is I think a lot talked about it; that's very important for if you really want to go further in terms of it, so we quickly did an upgrade purchased NAO in 2019; we started programming and coding; and we conducted a study 20 times between 2019 and 2020, and I'm going to share quick insights; and what we did with now is just exciting.
Here are a few insights very quickly for all of you, when we hosted this at the Marriott Morgantown at Nemocolin Woodland system which is a five-star diamond property very close to WVU; most of the people, you know, in these properties have experienced the humanoid robots beforehand, we took this robot in concierge service; sometimes we also use it in the lobby, in other words we also use it in the front desk as well. So we rotated across these three areas but as you can see 94 percent of them said it was their first experience interacting with the humanoid robot in a hotel and six percent of them had in the past, which means, you know more people are getting this experience which is great and post coverage, I think more and more people will see it.
The other thing that we saw is “do you like now? versus a black box; like you know a alexa or any small box, you know, and 88 person preferred NAO, you know, they preferred to have a humanoid robot instead of just a black box, you know, so which is really cool to see NAO does play a major role. We also looked at whether they preferred a male or a female voice interestingly, you know, 38 percent of them said there's no preference at all, you know, which is very interesting because we thought most people would prefer a particular, you know, voice. 33 percent said they want a female voice, 29 percent said they want a male voice, so there's not much of a difference it all goes back to your preference and your customer base and I think it's interesting that you know there's no clear preference on it.
Like I said most of the people wanted the NAO robot in the lobby or front desk right not in the room because of privacy and so on and so forth, restaurants sometimes maybe if you are very engaging in terms of creating experiences in a team restaurant, and so on and so forth, event centers again depend on what kind of event you're doing right.
So the other thing we looked at is preference of a robot size, you know, what kind of site they would like, you know, in the industry; we looked at 59 percent preferred now and 28 percent preferred somewhere in the middle and only 18 percent preferred live size, you know, a big size like human sized robots. So again there are a lot of cool things here, and more and more people like to see a robot that's more fun and more versatile like now in here and this is a very important slide for me, apart from all the things I shared here.
So we created; we worked with the hotel as well as the resort; we looked at what's happening around, we looked at the customer base, we created personalized services/ experiences, but we also created what we call a deal of the day, you know, and we work with the restaurants and coffee shops around them and as you can see, 88 of them that engage with the robot, by the way we collected data with around close to 200 people but so many people; we created experience of so many people that only few people wanted to do the survey right. So, 88 percent of them had wonderful experience engaging with NAO; which is really cool and they were satisfied with the hotel experience because of NAO; it's 78 percent and obviously they want to recommend the hotel in the reviews or word of mouth; that’s 78 percent; that's all great but go back to where it's important for our industry, right? You know, 40 percent of the people are willing to pay more when value is great and this is where you need to be really creative in terms of how you create value when you're hosting a service robot like now, you know, because it's programmable you; should be able to do that and then the other thing is, you know, we created a deal of the day and we would have now actually talk to the guest and at the end of the conversation, now would go back and say “hey, thanks for talking to me, do you want to get a deal of the day today”? And they would be like “whoa, what is the deal of the day”? You know and they would go back and say “hey, we have a Starbucks all winter packages, you know, discounted 25 percent because of your company, “you know, because I had a great time talking to you. If you are eating at the bourbon prime, you can get an entrée for, you know, free or you get an appetizer free when you buy an entrée, so on and so forth.
So we created the deal of the day and 35 percent of them said they would actually go purchase the deal of the day after they had conversation with now, and that is where I think you are seeing it written on investment, and when you program now, the impact of now in the service industry is really great in the sense like you are, you know, and I'll tell you in the next slide.
What the pandemic has done is it has really brought us new opportunities you know in terms of thinking innovative, you know, in terms of clearly bringing out a lot of people are thinking that way; they want something new when they go back to hotels they don't want the same cookie cutter kind of experiences anymore, you know, and that is one reason why the focus is, should be on personalization, right? We would create personalized experiences for guests, you know, I'm not going to talk about much about photograph because Ellard is going to talk about quickly on photograph, but we were able to create personalized experiences, like we would identify colors; we would see colors, for example, if I'm in front of now and I would say “hey, great I love your blue bowtie”, you know, it would do the personalization, right, then and there, you know, if you're wearing a bow tie like me, you know, or you would say “hey, what a great t-shirt”, you know, “I love your colors”, you know, so on and so forth.
You know, we would do personalization to start with, and by the way, Gen Z are attracted to now like crazy; they would run to now and they would bring all their family back and they would bring their grandparents back they would bring their friends back; so they always come back to now repeatedly which is something exciting because, how many times you would go to concierge services back again and again, you know, you will go to concierge only when needed and you would be like you don't want to go back again, right? But with now in the service industry the good thing is, the young generation is going to come back, they're going to bring their family and grandparents back if they are with them, you know.
The other important point is you need to focus on where now would fit with your touch points, you know, is it arrival, is it a check-in, is it that concierge, is it a specific experience that you want to create. Like I said, if you have an event, you can program now for that specific event but again focus on the guest touch point and go back to your goals and objectives, you know, and then that is where I suggest having a framework; like a script, very clearly; you have the script for this particular event, front desk, concierge, and these are your goals; and end of the day, if you really want a return on investment, you would go back to a goal of saying “I want to have 35 percent of them increase our return on investment or have a revenue of this much generated”. If you have that framework, a clear game plan for your ROI, now is really perfect for doing it, you know, and we prove that for both the properties that Marriot Morgantown as well as Nemacolin.
The other good part about now is that we are still in the process of creating the software and we are almost close to really understanding how to customize and personalize when people download a software and put it on now. And it's great that Emielard mentioned that there's 68,000 nows deployed so many educational institutions are interested, so why not have a software that's really specific to service robots that we can always download easily and then just change your, you know, brand name and business name and so on and so forth.
The other thing that peter talked about is future job pathways; there's a lot of opportunities in the sense like, you might think “oh, not all the staffing that the jobs that are lost are coming back but the new jobs that are really will come because of now and automation and service robots, and one of them is experience managers and these are the managers that would connect the data that would look at what's happening in a hotel in an event in a particular education institution at the particular time and they would create experiences for those of customer base. And again, it's also segmenting the customers making sure you have a clear plan for each of the segments and how you want to create experiences for them and these experience managers would play a vital role in the future and by the way if you want to really pay more value. If the flexibility is needed, somebody can manage a now robot from home as well or as well as in the back of the office instead of being in the front of the office so the jobs in terms of what people are looking for, in terms of flexibility can happen with now too.
The other thing I will definitely you know let you know is maintain the battery of knob every now and then during COVID, you know we use now as much as we can but always maintain the battery because now wants to always you know have a charged battery every 90 days once in a while so use as much as you can because there's a lot of cool things you can do; this is a conference that I attended a few weeks back at international street and now was one of the you know we hosted now at the big stage and everybody had a blast; it was the showcase for the, you know, it was that you know it was the pick of the day at the conference everybody had fun doing it. I programmed the now to talk to all the attendees, and you know, it was just fun to keep now and create these experiences.
The other main thing that I can tell you is the versatility of now is not only from a programming standpoint it comes with a pre-programmed which is really cool you can actually get it started with pre-programmed code; coding already done but you can also build on top of it, you know how you we want to create those personalized experiences and that is one reason why, you know, I appreciated what I did with now so far, but I think there's more we are doing in terms of our software as well so hope that gave you guys a quick few insights from our research and what we're done with now and here is my contact information if you want to know more. I only gave you a few information but we are also looking at data from our brainwave device which is inside we collected we're going to look at those five different emotions of people while they're interacting with now.
The other one that I also saw at the conference; one of the conferences was elderly people, senior living people love interacting with now; it's not only the autism side but also you know I had one elderly woman that came to me and she said “AJ, you guys need to program this to take it to senior living”, and she was very serious; she's like “there's a lot of need, there's a lot of need, you need to engage with these elderly people because they are so lonely and these kind of robots work perfect and there's a European company that's done this where they took the now to actually early senior living as well. So there is another area of that, in terms of autism, service robots, and so on and so forth. So, yeah here is my LinkedIn QR code and then my Facebook QR code here, and my website actually; it's my official website but I’ll be publishing more on these; some of the insights. But if you have any questions, I'll be happy to answer your questions. Again, thank you; again thank you for your time.
Peter Bowman:
And thank you so much Dr. Aluri for all of that wonderful information; I really, truly enjoyed listening to all of it; some of the things that, you know, really jumped out to me, just coming from, you know, a person that's for a lack of a better phrase here, right. I'm in sales ultimately as a lot of us are, and to hear that, you know, the robot had a close rate on something like a deal of the day at, you know, 30; I think it was 35 percent, that's very impressive. So, you know, and that's something that's transferable too, right? You spoke a lot about your use case but that type of data is also transferable to other use cases you mentioned that you had somebody from senior living that was also interested in this type of research and this type of work, you know, I think that that's valuable information for lots of people that are joining today but again thank you so much; I really appreciated your time and I can't let you know how much we appreciate your collaboration and your efforts, and not only to us here but also to your students at West Virginia University right. This is very exciting, so again, thank you.
Dr. Aluri:
Thank you Peter and by the way, both our team in the HITLAB team, the student team; they love working on now, everybody wanted to work on now, you know, they're like “hey, can I have now for a few days”, you know, so one of the students recently almost, you know, requested from me “AJ, I want to code now, can I have it for one week? I want to take it home, you know, so but we have a space in the business school that we use it. But again, great working with your team at RobotLab, you know, I look forward to actually sharing some of the correlations we are trying to figure out; some of the extended; we just only explore to study with it but we're doing more on trying to understand causal relationships as well as the rope, you know, brain ware that we collected data we're going to look at more about collecting more data, you know with real-time interaction with the robot which we have a clear focus on a framework; research framework and I'll be happy to present that in the future.
Peter Bowman:
Quickly, prior to starting today's webinar, you showed us a device that you actually use; can you show everybody that?
Dr. Aluri:
Yeah, this is the emotive insight; this is the one I think Elide was talking about the complete brain ware that they use; this does actually fly emotion; this is one what we did when people are interacting with the robot NAO; we had now and people usually wear this and then they interact with now and we program now with so many cool stuff which is actually five different performance metrics that they look at and those are basically, you know, excitement, engagement, relaxation, interest stress, and focus. So these are the five performance metrics it measures in real time; so if somebody is stressed, then how NAO can actually reduce the stress? Somebody is excited when they're actually interacting with the NAO, you can actually measure in real time. Some of the people that we had were more relaxed when they spoke to NAO; in other words, like they have, you know, too much going on at the conference, and they came to NAO when they were interacting, they were more relaxed; and sometimes they are more focused as well, you know. So, I think we want to look at that steady in the future but this is the brainwave device that we used actually to measure their performance metrics while interacting with NAO.
Peter Bowman:
Thank you so much for that Doctor, and you know, I think that's very important to highlight kind of leading into our next topic like what we were talking about, you know, soft bank Aldebaran; they originally put out the NAO robot to be a development platform, right? And that's all the intent there was behind it, they built this amazing piece of technology and they wanted to put it in the hands of people like Dr. AJ Aluri and like yourselves joining us here today, in order to see what all of you; what all of us can do with this technology, and with that, Elad, would you like me to go back to sharing the power point or would you like to share your screen to go into talking about choreograph in the now SDK.
Elad:
Yeah, I can share my screen here. Give me a second, so here we go. You see my screen? Everything good?
Peter Bowman:
Yes.
Elad:
Okay, cool so this is basically a choreograph, which is the quality entry level a programming interface for NAO. As you can see, it's a drag and drop interface, it's a visual interface; you don't need any prior, you know programming skills or anything around that in order to actually start working with the robot. Everyone can work based on a virtual robot that we have here on the side. I'm sure you all can see that, and we can basically just you know drag a box over here, let's say it's an animated say, I can change the, you know, the voice shaping we talked about male voice or female voice basically, you know, high-pitched, low-pitch; we can change the speed and everything.
And we can also set what the robot is going to say which is “hello everyone”; of course this is in French, let's do it in English, by the way, the robot can speak and understand 22 languages, I don't have all of them installed on my computer. But just for your information, it's absolutely possible; so let's do it here, “hello everyone” and I don't even need a physical robot next to me, let me just move this one a little bit below, you can see. I can actually see the bubbles over here, just showing me what the robot is going to say; if I want the robot, you know to move somewhere, I can just set up how far we want to go on each axis; if I want the robot let's say to you know ask a question or have some kind of a dialogue, I can drag this one over here and as you can see, I can ask a choice, let's do it again in English, do you like chocolate? and the robot will ask “do you like chocolate”? And it expects an answer yes or no, and based on the voice recognition that is built into the robot, the software will trigger either one of these output boxes and basically we'll get if I go back to the root level, we'll get out of this box, we’ll get our choice whatever we said, is it yes or no or what not.
So, it just shows you how flexible the platform is; really you don't need any prior experience with robotics or programming in order to get started, however for the more advanced people here, we can actually double click each one of these boxes and we can customize the python code behind it. Okay, so over here, for example, we'll have all the code behind it, this is based on the SDK that is coming with the robot, you can see here, we are creating a proxy to something that’s called text to speech, the AL represent the Leveron; the manufacturer; so AL text to speech, that's the library; so we reference that and then we can actually you know make the robot say something. If I double click the move one, you can see that we are creating a reference to AL motion, where is that? Here.
We are creating a proxy to the motion library and over here, we can actually control the robot based on that, so any python code, any version of code that you have can work with that, you don't actually need to put your python code in choreograph, you can run it in, you know, any editor, it can be even notepad as long as you import the library into your project; the robot can support also java; you can code the robot in java which is, you know, might be a different level of programmers and you don't have java in this in this piece of software; you can compile your code and have it run on the robot directly; so the robot can be totally autonomous, if the robot has its own Wi-Fi card, then you can connect it to the internet with your code; so you don't need your computer to basically remote control the robot; you can take your code, deploy it on the robot, shut down your computer and the robot is autonomous from now on.
Now this is for the basic version of now that we are offering. Recently, we added a new version of now which we call the now AI edition and to that we actually took it one step further and instead of letting you hard code everything, and program, and fight with all the connectors, and you know making sure that your code is actually compatible and works with everything, you actually get a whole different interface that allows you to create interactions with the robot. Behind that you're getting a connector to a Google dialog flow, so for those of you who don't know dialog flow, it's basically a chat-bot that is designed to extract the context behind the spoken words and I'll give you just a quick example let's say again, your, you know, a hotel and you go to the concierge and you want to ask, you know, what's your opening hours or until what time you're here, we have 50,000 different ways as humans to ask the same question, right? You know until what time are you here? When will you be back? Can I come tomorrow morning? Will you be here after breakfast? Are you open, you know, during the weekends and so on and so on. But all of them lead to one, you know, a piece of knowledge; the context here is, what's your opening hours? And the answer can be, you know, we are open Monday to Friday between 8am and, you know, 5pm and over the weekend between, I don't know, 9am and 1pm, or whatever you want to do that. We can actually randomize that and create some different answers so it won't sound very robotic, but the point here, we don't want to program the robot to identify each of the 50,000 different ways that humans can ask the same question. So, you can see here as an example; this is a basically part of a dialogue that was created to, you know, one of the robots; we decided to call him Antonio just because you can rename the row, it doesn't have to be now and as you can see here; you can decide what the robot understands of the left side and what the robot is going to respond with, that's on the right side.
And you can just create multiple responses, multiple answers when it's in there in brackets. It actually can be a keyword that is attached with the before or after a thing. And you can see here for example, Antonio give me a key, Antonio, can you give me a key, can you give me a key Antonio, and the answer is “sure, I can give you can your keys and the animation of like you know kiss, like the Italian kiss; the animation will be played by the robot. You can ask for a little bit of information “who made you? By whom were you made? Who's your mother? Who's your father? Who manufactured you? and so on. All of these will provide an answer which is SoftLabs robotics built me and brought me to life in France blah-blah or this is a randomized; thing we can play different answers to provide different interactions.
And obviously as Dr. AJ mentioned, we want to see analytics; we can see the full report of, you know, who interactives are, you know, what's the age group? and what's the gender? And, you know, what they said? and everything, we can get full analytics on that. We don't have time to dive into that but this is just to give you a quick you know overview of how you actually get all this you know good stuff that we showed you into reality. How do you take your passion and make it happen in a way?
Peter Bowman:
Awesome. Yes, thank you so much for going over all of that. Elad, I want to also kind of elaborate on what you said by just telling everybody I actually have a now AI edition right here to my right and, you know, it just shows that RobotLab; we're not just happy bringing the level of experience to, you know, what already exists, but we're continuing to innovate, right? We're continuing to bring you new materials, new tools to execute what it is that you want to do. Up here on the screen, there is a contact information for both our student CEO Elad Inbar, there at the top, as well as myself; account manager again here at a RobotLab; I am in charge of higher education; I have both my cell phone number and the office number displayed there as well as my email address, so, if we're going to go into a Q & A session here. We’ll commit to going a slightly overtime as we are coming in to 10 o'clock here, so we'll stay a couple extra minutes but if too many questions do come in, then we will ask that we direct some of those questions via email to myself mostly again Dr. Aluri, if you'd like to reach out to him and ask him questions about his research; I know that he's open to that as well; he's displayed his his contact information but Dr Aluri, if you'd like to also kind of put your information here in the chat, then please feel free to do so, and so that way folks can have another avenue to reach out to you.
Elad:
Yeah, on that note, a few questions were answered right on the chat but I wanted to elaborate on a question from a Richard; Richard asked if we can actually help you get the robot in a library setting and have the robot, you know, somewhat you know interact with different items that you have at the library either, you know, your catalog or you know 3D printers and everything, the answer is yes, and this is something that can be done really easily using the AI version, you can actually recommend books, you can actually create all kinds of recommending, you know, genres, if you like this, you might like that, all of this is pretty easily done again without coding at all; you saw the interface over there. So, that's that, then Dr. AJ, there is a question for you; if your lab created the code and personalized the code for the hospitality experience or was it out of the box?
Dr. Aluri:
And I answered that Elad and we actually created and personalized code, we actually did a code for each of the hotels like when we went to a resort, we looked at what their requirement was for the particular day, we would look at what are the people who are the people in the event and then we would actually customize all that with programming through our, you know, we did choreograph and we also did quick interactions as well, you know, so we did both of them simultaneously. Like I said, our HITLAB team were like experienced managers, they were trained what to expect that day, what is our goal? And then you know you need to train the people who are managing that day and we were able to custom build some of these features apart from the what came in the box.
Elad:
Awesome, thank you. Jackie asked about the autism and the therapist so the one that I mentioned is Dr. Eli Kazami; she's from UC Northridge; we have a few other researchers; we'll send you the details later on; Maria will take your details and will send you all the introduction to the others. Fernando asked which operating system the now robot works. What's a native one? So, the now runs a version of Linux that is proprietary to software robotics they modified a little bit of the operating system just to match all the, you know, support all the sensors, and motors, and everything, but it's essentially Linux and it's a, you know, pretty open environment you can install whatever you want on the robot. Budget if someone is with you; yeah any advice?
So, serial, a question about the budget; good question, there are two answers to that one; I don't know if in your school you have access to what's called the cares act funding, it's still available; even the president said that he really encouraged schools to apply for this funding because apparently only third of the schools applied and got the funds; there is still millions of dollars available and this is especially for what they call closing the gap, helping students to move forward and, you know, study all these, you know technologies, and, you know help them again get ahead of the curve, so if you have access to that, I mean check with your administration or, you know, reach out to us after this webinar, we'll provide you all the resources for that; the same with Perkins; we are just the beginning of September now and Perkins funds is available for everyone, these are specific funds that are available for career and technical education, and now computer science research in robotics fits perfectly in that.
And the last advice that I have on that is, if, you know, for some reasons these are not available for you, we have worked with many schools and universities, and basically allowed them to tap into different buckets of funding, so let's say, you know, the robot is, you know, thirteen thousand dollars and you have five thousand in this grant in two thousand and that grants and eight thousand in different grants, what we can do, we can actually break down the purchase orders to multiple, you know, buckets of funding and allow you to tap into all of them in order to get the raw one so you're not limited to just one funding source for that.
Peter Bowman:
And I would like to further elaborate on that a lot; a lot of the professors that I've partnered with over the past couple years have great success in applying for NSF funds as well. So, NSF grants; I would just like to throw that out there as another resource for everybody.
Elad:
Cool; I see Hiba asked a question about learning AI, what's about algorithm used to learning? So, in the AI edition of the robot, we basically did all the legwork for you; so what we did we created a connector between NAO and Google dialog flow, and basically let you focus on the content, on the, you know, questions, the answers and all of that, and basically offload the algorithm, you know, the understanding, the context, understanding to a Google brain and this is what Google does best right? I mean when we type our search term on Google, Google understands what we want, so you don't need to code that, you don't need to do that, if you want to do that yourself, you can actually get the transcript of whatever was said to the robot take it to your algorithm, then you can plug it to any neural net or whatever you want. Literally every service that is either available out there or you create it yourself, you basically get the text, get the content and, you know, let your engine run and provide the context, provide the answer to the robot; the robot itself doesn't come with the AI, it's a platform; it's basically the, you know, face of your system; everything behind it.
Peter Bowman:
And Miriam; I see your question as well; I'm not sure exactly which code you're referring to but you see my email there peter@robotlab.com please reach out and I'll be more than happy to get you whatever information I have available, and I'm very excited to see that we have somebody that looks like that they're at a school district very small writing from my perspective but thank you so much for your email, yes I will send you a couple of books probably, one for the NAO robot and then one for the AI edition as well, so that you can consider both options you've asked.
And I see another question here; has or will NAO be getting python 3 SDK; I know it's just been released, and has been python 2 in the past. Elad.
Elad:
Yeah, so NAO supports natively to python 2.7, I know it's not maintained anymore but there are wrappers that you can create that basically allow you to communicate with the robot the focus in the software development was shifted from python into a java into a, you know, more AI; higher level interfaces like you know the one; I showed you with inter connection to a dialog flow and so on. So, the python specifically to run on the robot is 2.7, if you want to run it from, you know, any other service; you can basically run it with python 3 or any other version just wrap it with the 2.5/8.7 executables.
Peter Bowman:
Thank you so much and this is going to be the last question that we're going to be able to; I have time to take again; thank you so much for everybody's time today: it says “is the AI capable NAO available only with the latest version of now or can I upgrade my older version to take advantage of the new feature. So the now AI edition is a new version of the robot that's being released; if you would like to upgrade then that is something that is possible, please reach out to me by email at peter @robotlab.com and I would love to facilitate that conversation with you, Mohamed thank you so much for your question again, we do have to wrap up today. Maria; she will send me your question and if you leave your email here as well, then I can respond via email. Again, thank you so much for everybody's time today; we've taken up a little bit more than an hour of your time at this point and so I sincerely appreciate everything thank you Fernando I appreciate everybody again, you know, our time is valuable and you've chosen to take an hour out of that time to spend it with me and I really appreciate it. Thank you everybody and I hope you guys all have a wonderful rest of your day.
Elad:
Thank you bye-bye.
Dr. Alure:
Goodbye everyone.