Today we’re diving deep into the futuristic world of generative AI video tools. Picture this, you create an AI Avatar that looks and talks just like you and then use it to produce captivating video content. Sounds like sci-fi right? Well, buckle up because we’re bringing this cutting edge technology straight to your ears with our special guest Joshua Xu, the founder of HeyGen. Join us as we unravel the secrets behind creating hyperrealistic AI avatars and learn how HeyGen is revolutionizing the video marketing landscape.
Joshua graduated from Carnegie Mellon University in 2014 and joined Snapchat since then. He founded HeyGen in 2020 Dec. Their mission is to use AI to reinvent the video creation experience. They believe visual storytelling = creativity + creation. HeyGen aims to make the “creation” cost nothing, liberating everyone’s creativity.
In this episode we cover:
- A short intro to Joshua and what led him to create HeyGen
- How technology changed over the last few years that has made AI something that has a genuine use case in video marketing.
- What AI Video Generation is
- What AI Video allows for creators and business.
- Why use AI? Isn’t real video better?
- Does AI Video Generation put traditional video producers at risk?
- What HeyGen does exactly.
- The challenges that arise from this sort of technology.
- Custom Avatars raises a whole new set of questions around ethics, legality and ownership, what issues does Joshua see and how he thinks these issues should be overcome.
- What’s next for HeyGen.
Links mentioned in the show:
- Try HeyGen for free https://www.engagevideomarketing.com/heygen
If you found this episode of value I’d love for you to reach out and let me know on Instagram @engage_ben or email [email protected]
Rate the Engage Video Marketing Podcast on iTunes
Transcript of the Interview: ** Note: the following transcript was generated by AI and therefore may contain some errors and omissions.
Ben Amos:
Joshua Zhu, welcome to the podcast. Thanks for being here.
Joshua Xu:
Yeah, hi. Nice to meet you, Ben.
Ben Amos:
So in my introduction there, I explained a little bit about Hey Jen and what the platform does and a little bit about yourself. But I’m interested to hear from you, what’s your backstory? What got you into AI and the technology that you work with today and video?
Joshua Xu:
Sure, sure. Yeah. I think so I have been always passionate about AI. So I did my master’s degree, you know, in Connecticut Mellon back in robotics, like figuring out, you know, building the intelligence for robot. And once I graduated in 2014, so sorry, 2014. So basically I joined Snapchat as a pretty early employee there. And I worked there for about six to seven years. pretty much I get a chance to work on a lot of different interesting stuff, like say building out the advertising system, figuring out the optimization algorithm for our ads, building out the recommendation engine, and also I get exposed to work on a lot of the AI, enable camera experience as well. So yeah, that has been something that I have been passionate for, you know, more than 10 years, and recently I think probably starting around Like three years ago, we just felt that, you know, the AI can actually start to create content and that got me even more excited. That’s why we found the company, um, specifically work on that problem, use AI to create content for people.
Ben Amos:
Yeah, I mean it’s definitely an interesting space that for many people probably… haven’t really paid much attention to artificial intelligence. It’s been something of a technology that you kind of know what it is for the last number of years. You kind of have heard about it before. It’s something that’s talked about, something that’s coming, something that’s being used in like scientific laboratories and robotics and things like that that you talk about. But really over the last, I would say one year really, I think it’s become, I mean obviously we can talk about chat GPT and things like that that have brought it into the mainstream language. but AI is something that in the last 12 months has just really become, I think, a viable tool in content creation, in marketing, in language processing, in everything. So from your perspective, who’s been working with this for 10 years or so, what is it recently that’s led to AI as being something that’s a genuine use case in business?
Joshua Xu:
Yeah, sure. I think the biggest thing, I would say the reason rising of chat GPT, like you said, it got brought to the mainstream and people start knowing that, oh, AI can do actually do these powerful things for us. I think that has been huge. I think pretty much like that even for myself, I use chat GPT to do a lot of the copywriting, not only for marketing, but also sometimes for internal communication as well. It has been great. You know, I think on the other end of it, I think the stuff we are particularly care about is the visual content creation, right? So if you look back, I think, so mid journey or stable division can actually start to create, you know, some very realistic image right now and help, help people to do, you know, the marketing imaging or the, or poster and stuff like that. I think. What has been really changed in my opinion in the whole generative AI space in the past few years? One was that, you know, I’m not sure if you remember back in 2018, you know, Snapchat released the first version of the more generative model can actually turn a user into a babyface or Disney style.
Ben Amos:
Yeah, yeah, yeah.
Joshua Xu:
That actually, yeah, so that actually was the very first model, in my opinion, can prove that. AI can create something that doesn’t exist in the world before. Right. So that really proved the capacity of the AI, what it can do. And, you know, in late, in early 2021, I remember the time that OpenAI released Dali, so they can actually use AI to create the image and the recent raising of the chat GPD thing, combining all together, it really makes like creating a video or generating a video possible in the space.
Ben Amos:
Yeah, it’s interesting. When we think back to early Snapchat filters and so on, that really took just the general culture by storm at the time because it was doing things that other, just I guess like Instagram filters, right? It was doing things that just seemed impossible.
Joshua Xu:
Yeah.
Ben Amos:
And
Joshua Xu:
Yep.
Ben Amos:
most users probably didn’t understand. It was because of the development of AI generative technology behind the scenes that was making these fun little filters so realistic. and possible. And then obviously as you say that builds upon the technology just gets better and so on and so forth. I’m interested from your perspective, what’s the benefit when you’re developing technology like this to turn it into fun filters and things that just the general public use without knowing that they’re using AI? Is there some kind of thinking behind let’s make something fun so that people actually use it, is that kind because I don’t really understand this space anywhere near you like you do, so I’m interested. What’s the thinking there?
Joshua Xu:
Yeah, I think it’s a good question. So like I’ve been a really a hacker style engineer in the past 10 years for myself. I really enjoy building new stuff, hacking new stuff. So I think as an engineer or a hacker, you always enjoy, you build something, get used by someone else. Even that you build a toy is being played with by some kids, right? I think AI is especially magical in my opinion, because… most of the people, they don’t understand, or they don’t need to understand this is AI, but it actually has a technology that can show off the magic. In my opinion, especially on the content creation space, if we build something that’s interesting for people to use, I think other than fulfilling the desire to see people actually using it, I think there’s another aspect of it is that you will see great creativity come out of, come out from this. create technology, you know. So we have been always, you know, very, very curious, like looking at what people would do with it. Basically the way we provide the technologies you are providing a tool, right? But many different people will have different use case, different user with, they have different creativity. They want to turn that into actual visual content. I think that part of it really, you know, is fascinating.
Ben Amos:
Yeah, it is fascinating and I think AI is going to, as we move forward, be underneath this layer underneath so much of what we do without even realising that we’re using it. And it already is in many cases when we think about social media algorithms and things like that are all run on artificial intelligence models. For the listener who has heard about AI generative video, or video generation through artificial intelligence, which is what Hey Jen, which we’ll talk about your platform in a moment, is really all about, how would you describe to someone who is completely un-technical, what is AI video generation or generative video?
Joshua Xu:
Hmm. I have a short answer. Basically, you could imagine the short answer would be like this. You have a dedicated assistant. He always sits next to you. And then you can tell him that what you want to make, what are the content, what is your creative ideas. And then that assistant will help you to fulfill the idea and bring the video in front of you. That is the proper short answer on it. I think the longer answer, I would say AI video There’s really two aspects from it. If we look at the video content today, so the first content, the first type of content is something coming out of the camera. We use a camera to film the footage. And then later on we will have some editing stuff add on top of it, right? So we have some, you know, animated sticker or image background, image asset, video background or transition and stuff like that. And those are the two elements of content in the video today. So, If we look at it on this angle, AI can actually create both aspects. It can not only create the asset as an image or video, it can also create something that we record from the camera so that we don’t have to do the camera filming again. So that’s sort of my answer to it.
Ben Amos:
Yeah, and the other aspect as well that AI can generate is the audio, right, the sound as well,
Joshua Xu:
Yeah.
Ben Amos:
which is, yeah,
Joshua Xu:
Yep.
Ben Amos:
absolutely. So, you know, as technology’s got better, it starts to become more possible, right? I guess the question would be, like, and it’s still early days, I think anyone who’s using this sort of technology understands that, you know, everything moves at a really fast pace and the technology gets better and it improves its ability to create these sorts of frames of video and so on. But why do you think it’s better for businesses in some cases to use AI or generative video rather than real video?
Joshua Xu:
So, I will put it this way. So I think real video is definitely great. You know, I think the problem is not at the real video itself. The problem is that most of the people or business, they don’t have enough time or budget to produce that many of good quality of real videos. I guess the problem is at the efficiency and the cost. So, and that’s why AI can help because it fundamentally is not trying to create something. you know, different from the real video. That’s something we want to approach to, right? I think that the AIS role is actually pretty much like reducing the cost to bring the possibility that people can create more videos.
Ben Amos:
Yeah, the way I see it is the demand. for businesses to create video content is continuing to increase, to be on multiple platforms with multiple videos, multiple messages, perhaps even targeted at multiple different audiences with different messaging in it. So it can be very overwhelming, which means most businesses don’t do enough, they don’t take enough action. And I think
Joshua Xu:
Mm-hmm.
Ben Amos:
using AI tools like Hagen and using them well for the purpose that they’re designed for, isn’t going to take away from the produced videos where they have a place and an important place in a marketing strategy, but it empowers that person to do more and to do more versions of the same content as well. I think, again, still early days and I think as the quality improves it will become much more of a viable tool for many businesses, but I think it just possibilities for people.
Joshua Xu:
Definitely.
Ben Amos:
So you guys at, say, Jen, or maybe it’s just you, Joshua, you talk about the idea of visual storytelling as being, I think, this combination of creativity and creation, right? So,
Joshua Xu:
Yeah. Yeah.
Ben Amos:
and that’s what makes up visual storytelling. And your mission, and that’s in your bio as well, is to actually make the act of creation cost nothing, to remove any restrictions or any barriers to creating so that people can, I guess, focus on the creativity. So that sounds like a pretty bold claim, right? The idea of making creation cost nothing. Can you talk to me about that? What’s the thought process behind that?
Joshua Xu:
Hmm. Yeah. So I think a bigger, you know, if you look back like 10 years ago, right? So pretty much, I think, so probably more than 10 years ago, you know, when mobile internet is not a thing at during that time, really, you know, there’s really small amount of people can actually create great quality of content. And during that time, you have to have, you know, a TV device or a professional camera. But once the mobile cameras get invented and a lot of mobile apps get invented, so we can see that almost everybody can start to create very high quality content using the mobile camera. So as far as I see, the AI generation technology is as big as the invention of the mobile camera, because it can bring down the creation cost even lower. So yeah, just imagine a day. Me myself also runs the video marketing and marketing team in my company. We, we are still, you know, we were in a place that we spent only 20% of the time thinking about what should be our video strategy. By the way, Ben, I enjoy, you know, listening to your video strategy podcast a lot, but we still, you know, we can only spend 20% of the time thinking about the strategy, but almost spend 80% of the time actually making the video, right? So it doesn’t quite make sense to me. The ideal cases should be that, you know, we actually spend 80% of the time thinking about what needs, what video needs to be made and what is our strategy. And they only spend 20% of the time making it and tuning, like you said, having multiple version, having a lot of idea get tested out. So that really is the, you know, vision that drive us to build a video engine for the business. Ultimately, I think our vision for the company is to build a video engine for everybody so that. Every time you have a new idea, you can use video to visualize that, and that can communicate better. As a first step, we want to build a video engine for business, and that’s what we have been taking action in the past few years.
Ben Amos:
Yeah, I mean there’s so many parallels in history, right, of things that new technologies, new tools that came into common use that… made things easier, that made things faster, that made things cheaper, that at the time that they were released, the existing ways of doing things felt like they were a threat. You think about the printing press, you think about that made books into the hands of everyone, you think about the horse and cart and then the car, the motor car coming through, and cheap content creation with smartphones and that hasn’t put, the reality is that hasn’t put professional video producers out of work. In fact, as more content’s being created, there’s more and more videographers, video producers coming to market. There’s more demand for those professional producers. I think AI video generation is the same sort of thing. a threat to professional video producers. In fact, those that embrace it will actually find that they can do their job better, faster and cheaper. I think it’s an exciting space.
Joshua Xu:
Yeah, I think we should just imagine that the AI or the new technology is a new tool. Like I’m not that type of, you know, super bullish on the HGI thing, but I think at least for now, we should always consider AI as a tool that help us to do our job better. And like, if that can be done in a better way, and then we can just like express our creativity in a much more efficient way and better way.
Ben Amos:
Yeah, absolutely. So let’s talk about your platform. So Hey Jen is your tool, is your software that
Joshua Xu:
Mm-hmm.
Ben Amos:
you and the team developed. So tell me, for people that haven’t come across it before, what does it do exactly?
Joshua Xu:
Sure, sure. Yeah, we can imagine that Heygen is the tool that enables the video marketing for every business. So pretty much, I think the vision we want to achieve is that to turning that 20 and 80 percent the other way around, so that people can put more time to actually thinking about what’s the video strategy, right? So we actually started with… Basically, we want to build a video engine for the entire business. So every time you need to create something for video marketing, sure, we should come to HeyGen and that’s easy way to make a video. And we started with creating a spokesperson video. Basically, so for example, if I’m recording a product tutorial or product release or promotional video for my company or for my service, I don’t have to show up in the camera again. I could have my avatar. as an AI-sposed person that you can just use text or audio to drive it and it can speak more than 40 different languages. So in that way, we help to reduce a lot of the cost of content creation and people can actually be unlocked to do a lot more video marketing content in the different social platforms.
Ben Amos:
Yeah. So. just to add to that. So I’ve been playing around a little bit with HeyGen very early, but for listeners who are interested, I definitely recommend you go and check it out and just have a play as well. You can jump in for free and have a little try of the software and see what it does. But basically, currently, anyway, and I know there’s bigger plans for HeyGen, currently it’s really about creating talking head style videos that can be used in marketing and in business. But instead of the talking head being a real human, it’s using artificial intelligence and generative video to create that avatar, that talking person on the screen. And I think one of the interesting things is that you can actually create an avatar that looks and sounds like you, which is really interesting and raises a whole bunch of questions as well. So that’s the kind of thing. So can you tell me, like, for the people that are early adopters of Hey Gen, what sort of things are they doing? What sort of videos? I mean obviously it’s very wide and varied, but what sort of content’s being created on Hey Gen right now?
Joshua Xu:
Yeah, so they really probably definitely we are, we build a horizontal to horizontal too. There’s a lot of like, you know, different use cases. So I can name a few of them. Like first of all, there’s a lot of customer making product tutorial videos. Basically, usually we will see, you know, there’s some company have a brand ambassador sitting in from the camera to make those type of video and do the screen recording. And right now what you can do with the HGN is that pretty much you upload the screen recording and then we will have the avatar speed up for you. And if you guys check out on our YouTube channel, there’s one playlist that’s created entirely with my avatar and my AI voice. So we just put on some of the visual elements on it and the product tutorials done. And there’s also a lot of use case around creating the personalized video messaging. Like for example, when we want to do the email outreach or like sending out, you know, a lot of the sales which our email or messaging, right? We can use the AI tools to create a lot of like the, you know, different personalized video for different customers. And there’s also a bunch of use case around building up some internal content, like for example, learning and development, onboarding, internal training and stuff like that. Yeah, there’s really a lot of happening inside the platform. People are making use of it to fulfill some different kinds of video marketing or video messaging purpose.
Ben Amos:
So what do you see as the benefit to using avatar kind of presentation on camera, which… is not a real human and many viewers will recognise it’s not a real human, at least at the way that the technology is now, but it adds a human element to the content. So what do you believe is the benefit to using AI avatars to present information rather than AI just voiceover or even just a regular voiceover which is easier to produce than video content? know, so what’s the benefits?
Joshua Xu:
Definitely. I guess the question would be, I would try to ask a different way, even for ourselves. So we are basically doing the interview right now. What’s the benefit of having both you and I actually show up in front of the camera, having a visual face of ourselves, because that actually creates a better engagement between the content and the user, that’s for sure. But I think the other question is that whether this quality of the content is good enough for people to actually recognize that it is artificial or it is the real content. That’s another question because that’s something we are trying to solve and really we have been putting a lot of effort on that. But I would say in a way that if we can present the real human in front of our audience and customers that really create a lot of connection and build a bridge between the content and customers. And we have seen a lot, you know, we are comparing a lot of campaign or experiment before just comparing by using the voiceover only versus the voiceover plus the avatar, you know, the other one always perform better in terms of engagement or click through rate and stuff like that. Yeah.
Ben Amos:
Yeah, yeah, because people… people interact with people differently, right? And
Joshua Xu:
Yeah, yeah.
Ben Amos:
to echo what you said there around the quality of the avatars that your AI is generating in HeyGen, that was one of the things that just blew me away when I first saw that. Now, obviously, when you are looking at it and you’re kind of knowing what you’re looking at, then I think at least at this stage, it does appear to be AI, context of a piece of content. Many users, I think, even with the technology you guys are working with right now, is incredible. It is often very difficult to tell that it’s not a real person. And I believe… that’s probably going to get better very quickly. Would that be right? Like as far as you know, in the way that things are developing and the technology is improving, I think that the few things that maybe are holding it back from full realism, it’s probably not too long until that is overcome. Would that be correct?
Joshua Xu:
Yeah, definitely. So you can imagine the way we have done a lot of tests before. Like if you are a person that doesn’t know about this technology before, and I just present two contents, one is real, the other one is not real, or created by AI, right? So from our previous tests, 90% of people really cannot tell which one is actually making it from the AI. That’s first. And secondly, I do agree that, you know, every time when the technology gets mutual in a business, the last 5% problem is always super hard to solve. Yeah. Even, yeah, we have been working really hard on it. And some of it is that how to, how do we, it’s a question about how do we design the product in a way we can hide the limit on the technology, but actually, you know, it’s both the advantage of using that and create value for our customers. those would be the questions I think have been top issues that we will be working on in the team.
Ben Amos:
Yeah, it’s really interesting to see where. where this technology is going to go as it improves, and I’m sure it will improve very quickly as well. But I also just want to say again, just how blown away I am by the quality that it is right now is like very usable right now, no doubt. I think when we think about this idea of creating avatars that have a likeness to a real person, so you’ve created an avatar that you use in videos for Hey Gen, the team at Hey Jenna currently putting together an avatar that is my likeness as well and trained on my image and my voice. We haven’t seen it yet, but I’ll share with my audience as well when we see that. Obviously, there’s a lot of benefits to that sort of technology to scale up content creation. But it does raise some questions. I think we have to address avatar of myself, I can type words into Hey Jen and get it to say something that I’ve never said. So how does Hey Jen get around that idea, or not get around, how do you think about that idea of potential ethics and ownership and so on of these avatars’ likeness?
Joshua Xu:
Yeah, definitely. So it has been a top, one of the top questions that we have been thinking about every day. So like part of it, I think, so first of all, the ethics or ownership problem is super important, like even for our company internally have been taking really care of it. So I think we have some strict policy inside of the company, for example, let’s say we create an avatar for you, Ben. So… we will make sure none of our employee can actually get access to your avatar, even for our internal employee, unless we get your authorization. That’s first. And second, we make sure we build a lot of like security and data privacy policy around of it. And, and the third one would be mainly, you know, every time we want to get the, um, before start making the avatar, we will make sure we always get the user consent. to make sure it is himself that he wanted to create this avatar. Yeah, there were a lot of stuff that needs to be done in this space. Yeah, I would say we are still working on it. And meanwhile, I think the general generative AI does raise a lot of questions about the assets ownership or issues around the ethics. You know, yeah, I think it’s… I’m personally optimistic about the future because I believe the regulatory department will make sure it gets regulated. And meanwhile, as a company that has the most advanced technology, we’ll make sure we can serve a good example to leading that industry standard, making sure we can actually protect our user data and ownership.
Ben Amos:
Yeah, thank you for responding to that question. I know it is a tricky one. And it’s definitely, you know, this whole space of artificial intelligence is moving so fast that I know companies that are trying to stay ahead of the technology
Joshua Xu:
Yeah.
Ben Amos:
and genuinely provide valuable resources and tools and software and so on for users are also moving very fast with this as well. So, you know, I think it’s an interesting space. There’s no clear answer, right? on the individual, on the user to understand the… the situation, the ethics, the potential, rather than the companies to be 100% responsible for that as well. It’s an interesting one. Watch this space. I’m interested, just as we wrap up here, Joshua, you’ve talked and you’ve hinted a little bit about the future for HeyGen as being the complete solution for marketing content for businesses. What’s What are the next things that we potentially, you probably can’t talk about at all, right? But you know, where are you going with Hey Jen and what sorts of things do you think we’ll be able to do with your platform moving forward?
Joshua Xu:
Sure, sure. Yeah. So like I mentioned earlier, the ultimate goal for the platform is to get involved into a video engine for everyone or for business. Right. So right now we can think about the angle like this way. So the spokesperson or the avatar video is like the A role of the content. And then we haven’t been able to add a lot of B role elements to it yet. So but it’s coming very soon in the next one or two quarters that we are adding a lot more generative capacity to generate those B-roll elements. For example, video background, image asset, animation, sticker, and stuff like that. Making sure we can, on one hand, we can provide the end-to-end video generation experience, not only the avatar, but also other elements. On the other hand, it’s that people can actually finish a lot of most of the work on HeyGen by not needing to go to some other software to edit it again. So I think that’s probably will happen in short term. In the long run, we will be able to, you know, I think we envision a future where the product can take the idea of creativity from the user or from a business, then we immediately turn that into video. That would be probably something we’d work towards in the long run.
Ben Amos:
Yeah, that’s very interesting. It’d be interesting to see where it goes. Thank you for sharing a bit more. But just in closing here, Joshua, if people are interested and they’re intrigued and they haven’t yet played with Hey! Gen, can you tell me a little bit about how people can get started? You do have a free option for people to dive in, and then there are some paid options as well. Can you talk to me about how people can get started with Hey! Gen?
Joshua Xu:
Sure, sure. The easiest way is definitely go to our website, www.hn.com, and we provide free memes so every new customer can actually enjoy one minute of free usage on the platform to give it a try how you like about it. And then if your usage is more than that, we provide paid options as well. And also, I think we have been very active on our social platform as well, especially on YouTube. I recently… publish a new YouTube playlist, like I mentioned, it’s the product tutorial content that made with my avatar. Definitely give it, check it out and see if you want to build something for your business as well.
Ben Amos:
Yeah. Awesome. We’ll have all the links to Hey Jen and the YouTube channel and that YouTube playlist in the show notes for this episode at engagevideomarketing.com slash 272. So this is episode 272, so slash 272. Joshua Zhu from Hey Jen, thanks for joining me on the show here today. This is a really interesting space that I think everyone in the world of video marketing is going to be well, the next foreseeable future. So thanks for joining me.
Joshua Xu:
Yeah. Thank you, Ben. Thanks for having me.
