In today's episode, Ben and Michael dive deep into the intersection of education, AI, and innovative instructional design. Luis Garcia who is the President of PETE, delves into automating instructional design, content development, and assessments, shedding light on the evolving educational landscape and the pivotal role of evaluation and learning. Ben shares invaluable insights on leveraging chat GPT and generative AI to streamline documentation creation and evaluate knowledge, drastically cutting down processing times.

Together, Luis and Ben discuss the positive reception and transformative potential of AI-driven micro-courses, text-to-speech features, and customized training tools in education. They also touch on the intense training involved in fields like nuclear reactor operation and the need for effective onboarding processes. Michael contributes by emphasizing empathy and strategic pacing in international business projects, while also summarizing instructional strategies and organizational tips for rapid learning and growth.

Join them as they explore the crucial role of innovative AI technologies and personalized learning tools in reshaping education and business training, featuring insights from top industry professionals and thought leaders. And don't miss the chance to learn more about Pete and Collectiva. Get ready for a compelling discussion about enhancing learning outcomes and the future of education with AI!

Michael Berk [00:00:05]:
Welcome back to another episode of adventures in machine learning. I'm one of your hosts, Michael Burke, and I do data engineering and machine learning at Databricks. Today, I am joined by my co host.

Ben Wilson [00:00:15]:
Ben Wilson. I do quarterly roadmap planning at Databricks.

Michael Berk [00:00:20]:
Today, we are joined by Luis Garcia. He started his career in academia and then moved into software engineering. Curriculum development. And then over the next 20 ish years, he held

 Luis Garcia [00:00:28]:
a variety of positions

Michael Berk [00:00:28]:
at SAIL. Currently, he's, the cofounder of Collectiva, an organization offering fractional executive services for game tech startups. And I know this is, an area that Ben is passionate about, so we'll get into that for sure. But before we do that, Luis, you served a variety of offices, at least departments in Brazil, Venezuela, and the US. And I was curious how you approach doing business in non US countries.

 Luis Garcia [00:00:59]:
Oh, boy. That's a a little bit of a loaded question because, is is there's some allure into the national business. So it's great to interact with other cultures in other countries. But what, folks sometimes don't understand is that the US is is the exception in the and a lot of great things about business. And the regulatory framework, for doing business in the US is incredibly friendly, for businesses. Yeah. The US is easy to create a business. You go to a website, do a a a few things, push a couple buttons, and there, you got a you got a company.

 Luis Garcia [00:01:50]:
And, we take that for granted in the US, and, we think that that's how the the rest of the world works, but it's not like that. Most of the the world, it takes months and, 6 months, a year to create a company just to do that step. And and, so it's very easy to create companies and to fail in the US, which is so important for entrepreneurship and so, for creativity and and, innovation. And, also, the regulatory framework around employees is is also very different. The US is very business friendly Mhmm. Which some people may argue is not a good thing for the employees. And and, but it also goes to that point of of allowing businesses to be created and failed quickly. So if you I'll give you an example.

 Luis Garcia [00:02:51]:
In many countries, by law, you have to provide paid vacation to the employees. Sometimes it can be a month, you know, by law. So if you're a start up, if you have to give somebody a month of pay vacation, I mean, you might not be able to afford that. And, so it's difficult to start a business. It's expensive, and then it's expensive to fail. And, so all that to give you that that, Latin American countries are very much like that. They're very pro employee, very difficult on businesses. So the way you approach it is with a lot of patience.

 Luis Garcia [00:03:31]:
And, and speaking the language does is not enough. It's it's a a a a great thing because you can communicate with people easier. So I I speak obviously speak Spanish, and I also speak Portuguese. So it allows me to communicate with people and connect with them. And, but that's just part of the equation. And, doing businesses overseas is really hard.

Michael Berk [00:03:54]:
Does it differ in academia?

 Luis Garcia [00:03:59]:
Well, it's it's academia here in the US works a lot like Latin America. You know? They got this thing called tenure, and and, folks in academia get a lot of vacation and had a lot of big benefits. So and, so I will say no. Probably, it's it's similar. If you work in academic, probably it's similar and that you take the summer off, you know, that kind of stuff. And, so, but in in business, it is very different. And, also, the other part is that, and I'll I'll try to, choose my words. The American worker tends to be a lot more focused and self driven and accountable than workers from other countries.

 Luis Garcia [00:04:48]:
And and, again, the the US being deception. So you think that everybody's like that, and, and it's not. The work is in other parts of the world require a lot more follow-up and, to finish tasks. And, and that, again, adds to the complexity. So I approach it, again, with a lot of patience. Got it. Yes.

Michael Berk [00:05:14]:
Ben, you've worked with hundreds of companies. Some of them, no doubt, not in the US. What are your thoughts on the topic?

Ben Wilson [00:05:21]:
I couldn't agree on Luis' summarization more. Like, if you're particularly if you're an American who is not who's never dealt with what I've seen is, like, Americans that are born here, 2nd, 3rd, 4th generation Americans, regardless of what country of origin your family comes from, you're American. Even though Americans love to say like, oh, I'm Brazilian. It's like, no. Your great grandparents were born here. You're you're American. You speak like you're from the Central Valley. They don't get exposed to cultures.

Ben Wilson [00:05:59]:
You know, people that come to this country in this business environment and the culture here, this capitalistic business focused, like, industry focused country, I've noticed that they don't really understand how the rest of the world actually functions and the fact that work is not life, elsewhere. So when they end up having to interact with people and companies in different parts of the world that don't share that same culture, they find it, like, really frustrating. I've heard comments from people like, oh, these people that I work with, they're so lazy. I'm like, no. They just have other priorities, like family and food and enjoying life. You know, you're, you know, you're an American. You grew up here, and you work in this in this tech company or something, and that's your focus is, like, I wanna build cool stuff or or get famous or something. Like, they don't care.

Ben Wilson [00:06:53]:
You know? It's a job to them. And the people that I've seen that are successful that interacting with people from other countries are doing the, I'm gonna walk in your shoes and understand this from your perspective and and actually ask questions. Well, like, hey. Get to know the person a little bit, and you get to understand, like, where their headspace is. And if you can get to a common ground, that way, you can learn to work together really well and learn stuff from each other.

 Luis Garcia [00:07:21]:
Yeah.

Michael Berk [00:07:22]:
Yeah. It reminds me I one of my first, overseas engagement was in Australia for Databricks, and it was it was quite the experience. I was there for, like, 3 weeks. They put me up in a pretty nice hotel. I lived out of a hotel, which was an interesting experience. But what I was so surprised about is the, like, total volume of hours that people worked. It was, like, 5 or something, 6 maybe. And they get stuff done generally, but it's just a very different culture from traveling to SF to the SF office and seeing people, like, there from 7 AM to 7 PM just writing code nonstop.

Michael Berk [00:07:58]:
It's yeah. It was pretty mind blowing. And, like, the value of team camaraderie, having a 1 or 2 hour lunch, like, spending time together, it's it's just a very different, set of priorities and Yes. Whatever makes you happy.

 Luis Garcia [00:08:11]:
I'm on vacation. You know? Exactly. Yes.

Michael Berk [00:08:15]:
Which honestly right now wouldn't be too bad. I I wouldn't mind being in Australia. But, yeah. Okay. Interesting. So, going back to your life in academia, you held a variety of positions. Can you walk us through why you took so many different styles of jobs?

 Luis Garcia [00:08:35]:
Yeah. I think it has to do more with with, you know, how I approach work than than, than anything else. Established organizations need people to innovate inside, and, and sometimes it doesn't mean that you come up with great ideas. You just come up with sometimes it means that you execute somebody else's great ideas. And, especially with universities are are entities that have a very clear business. Business framework is they get revenue from tuition. And in the case of University where where I work, revenue only came from tuition. And that's so that business model is very simple.

 Luis Garcia [00:09:21]:
So it's doing if student population is going up, everybody's happy. If student population is coming down, we're in trouble. And and and to that extent, everything is very focused into as as they should and, you know, getting make sure that students are are are being educated and trained and and being successful. And, but it's very hard to then innovate inside that. And, and many times, the ideas are there. There's just no one to implement them. No one to really explore them. So I have made a career of being the guy that say, well, okay.

 Luis Garcia [00:09:56]:
I'll do it. And, and, raised my hand and said, okay. I'll I'll try it. And, and what I tried to do because I spent there 19 years and, is that come up with an effort that will take about 5 years to do. And, because, like, after 5 years, I don't I I can't imagine myself doing the same thing anyway. And, but it also serves a purpose for the organization. You start something new and and it is successful, the best thing you can do is step out of it and, and let somebody else run it and and and do and and take it to the next level. That creates career opportunity for others, and, and that allows me to go do something else.

 Luis Garcia [00:10:39]:
And, so that's why I did it. So when you look at my my tenure, there is about 5 year projects. And, so we'll we'll set up a goal in 5 years. Usually, it had to do with student population of of a particular set of programs or new programs that we're gonna launch or a big effort. And, 3 years in, you know if if it's gonna happen or not because it's not linear. It's usually a little bit, you know, and slower at first. And then if you get the traction that you need to have by the 2nd or 3rd year, you know they're gonna get there or not. So by the 2nd or 3rd year, I knew if it was gonna happen.

 Luis Garcia [00:11:13]:
If it wasn't gonna happen, then we'll fold it. We'll fund the new project. And, if it if it was going to happen, and then and then we'll continue, run it. And then the 5th year was basically a transition year, finding my replacement, seeing where this fits in the overall organization, transitioning that. And then I will be out of a job, in 5 years end and then looking for a new one. So that's that's how I did it usually. Can be really busy. You know?

Michael Berk [00:11:41]:
Okay. Lots of really interesting points. One that I wanted to ask more about is how do you think about being replaced? Do you train the person below you? Because you said it's important after the 5 years to let other people have the same opportunity. Mhmm. Do you facilitate that, or do you just get out of the way and let someone else come in?

 Luis Garcia [00:11:59]:
No. Usually, you have to always be thinking about who can replace you in the role that you're you're you're doing. And I think that's true for every organization. And and, and it's important for 2 reasons. Number 1, if you make yourself, irreplaceable, then you are irreplaceable. And then that not only means that somebody cannot replace it, it also means that you cannot do anything else. And, I'm I don't like that, And I prefer to change. And, so I do facilitate it.

 Luis Garcia [00:12:27]:
I usually, if you whatever you do, you're growing and there's there's people under you, and then you start off serving who can the people. Sometimes it's not only 1, sometimes it's a couple because as as a general manager, you do different roles. And if it grows enough, it's it really is 2 roles or 3 roles depending. And, so I'm always in the lookout to be able to replace myself. I think it's irresponsible if you don't have a replacement of yourself and because you can be, you know, run by a truck outside, and and then if the the whole effort falls apart because of you, I see it as a bad on me. And, and, so, yeah, I very much facilitate it in, to find my replacement. But it's not my replacement of my person. It's the placement of the different roles that I do.

 Luis Garcia [00:13:17]:
And, and some sometimes it can be 2 or 3 people.

Michael Berk [00:13:21]:
Yeah. And that 1 month vacation is pretty tough if you are essential and irreplaceable.

 Luis Garcia [00:13:26]:
Oh, then we can never do that. No.

Michael Berk [00:13:28]:
Yeah. Yeah. Exactly. Okay. Cool. Do you have any skills or learnings that you got from the 20 years in academia? Like, what do you think you you developed in that time frame that you wouldn't have gotten in the private sector?

 Luis Garcia [00:13:43]:
Well, I work I work for a private university, and, so it felt very much like a corporate job to me And, because I wasn't really in the, in in the faculty, I didn't teach. And, but I did work a lot with them. And, so I was very much a a general manager inside of an, a university. People don't think of universities as as corporations, but they really are. And, and they have a lot of employees. They have different departments. They have sales. They have finance.

 Luis Garcia [00:14:14]:
They have HR. They have operations. They you know? It's very they they open, they close, they merge, they buy others. I mean, is is, we don't like talking about that, but it really is a corporation. So for me, it really felt like like like corporation. What is different is that it is a consumer business with a long sales cycle. And, what I've been mean by that is the students and, depending if they're the traditional student that comes into from high school into into, into university, takes about 2 to 3 years to make that decision. And, but you're speaking directly to a consumer.

 Luis Garcia [00:15:02]:
So so usually consumer business have short sales cycles. This is a consumer business with a long sales cycle. And, so it's in that sense, it's it's very unique. I think it prepared me to to be in the b to b world. And, because it gets me, it's very relationship driven and, you you you know, no one goes to a website, sign up for a university. You know? There's always lots of conversations involved, and, there's a lot of bonding that you have to do on connection with those customers and with their parents. So it's a complicated sale. And, so it really prepared me for the way to way world, which is very much like that.

 Luis Garcia [00:15:40]:
It's very relationship driven. The sales cycles are long. And, so I think that that's a very something that you can translate very well.

Ben Wilson [00:15:49]:
That's how I've always thought about, particularly the private education and public education systems in collegiate level is actually that, the b to b model because it's more reputation based from what I have seen

 Luis Garcia [00:16:05]:
Yeah.

Ben Wilson [00:16:05]:
And the the marketing effort and, like, the you have to focus on the image of your business and what quality you're portraying to your potential clients, which are the the students. Because you can see, like, what happens with the US News and World Report, like, college rankings that comes out. You know, like, I can only imagine what the provost of the university or the chancellor is waiting for this report coming out, which is the similar to, like, the Forrester report for Big Tech. You know, like, where do we rank on this? And Yeah. What what's our image? When you actually think about it, like, it's not that important. It's it has no bearing directly on the services that we provide, but it's people's perception of those services about when they're gonna make a big decision like that. It's yeah. I think you nailed it.

Ben Wilson [00:16:56]:
It's very much like b to b. Like, that word-of-mouth and that that reputation status means everything.

 Luis Garcia [00:17:05]:
It means everything. And, in our case, what we did is that we promoted the success of our graduates. And, and that also translates very well for for the b to b world because in, in the b to b world, you're usually serving serving another company and help that company be more successful. So their success story becomes your success story. And, and, and then the university world, the way, we did at our university, it was all about promoting the success of our graduates. So that our graduates were doing great, then those were the stories that we were, using to shape our brand, like you're saying, Ben, for the incoming students to see. And, and so they can reflect themselves. If somebody went to this university, end up working at, Google or Universal Pictures or because we're very entertainment driven.

 Luis Garcia [00:17:58]:
And, with and now they are, directing a movie. And, for those that were a filmmaker, aspiring filmmakers or won a Grammy for those who were kind of for recording engineering or worked in the, you know, triple a games that came out, then they thought, okay. So that person came to this university, did all that, and then end up in that place. It's not a guarantee, obviously, but but you can draw that direct connection. And, and the b two b world where I live right now, that's very much it. I mean, you you empower a customer with your platform. If they do better, then that's your success story. That their success story becomes your success story.

Michael Berk [00:18:36]:
Saucy question. How much attribution should a university get for the success of its students? Business success.

 Luis Garcia [00:18:46]:
You know, I think a lot. And, you know, the time that you spend in in university is is a very transformational time. And, even if the skill set isn't directly tied to what you end up doing and the connections that you make, the ability to think and and solve problems, which is basically what undergrad is for, is it shapes your the rest of your career. And even the successful the the long careers because what what we observe over time is that, professionals end up having 2 or 3 different careers. And, the one that they do right out of schools usually is the most tied to the skill they learn. And, but then the success on the after the 5th year, the 10th year, 15th year, 20 years, and, what they trade back is certain behaviors that they learn in college. And, so I do think and and some some people will disagree with this. And some folks like to say that they learn nothing in college, and and all the success is absolutely only theirs.

 Luis Garcia [00:19:53]:
And, I don't deny that it may be some of those cases for for the most part. And, you're learning how to think and how to behave, and that carry us carry you very far.

Michael Berk [00:20:05]:
And, also, even outside of academia, just like how to live as an adult. Like, typically, that's the first time people are living away from home, how to, like, take care of yourself, how to manage stress, how to

 Luis Garcia [00:20:15]:
That's fine.

Michael Berk [00:20:15]:
Yeah. So many things, how to build relationships. And those are just as important as we were talking about earlier. Your job is not the whole world. Okay. Cool. So I wanted to shift gears a little bit. Mhmm.

Michael Berk [00:20:28]:
Collectiva or Collectiva?

 Luis Garcia [00:20:30]:
Collectiva. But I I I I love to actually talk about Pete, if you don't mind. Yeah. Sure. And, although, Collectiva is my own practice, and I I work with great companies and in, with Collectiva, in very exciting technologies. And, I I joined Pete about a year ago, and it's a it's a AI AI company that creates, educational materials automatically. And, and, and that's, where my focus is the most these days.

Michael Berk [00:21:03]:
Got it. Okay. So let's do, like, 2 seconds on Collectiva

 Luis Garcia [00:21:07]:
Okay.

Michael Berk [00:21:07]:
Just just for the the gaming context.

 Luis Garcia [00:21:10]:
So Yes.

Michael Berk [00:21:12]:
So how do you guys approach funding, and how do you guys approach finding a good idea?

 Luis Garcia [00:21:18]:
So it's kind of the other way around. I think that what what I have found with Collectiva is that there's there are entrepreneurs out there trying to create a technology and, but they're missing everything else. And, and, because when you're creating a a a a product and you need funding, you're really creating 2 products. You're creating the, the technology product that's gonna serve a customer, and you're creating a financial product that's gonna give a return to somebody. And and, so when I get approached by by, by entrepreneurs, it's because they're not really paying attention to that other thing, which means it's like, are you sometimes you do you have a brand that that you can do you have a marketing strategy in place? Do you have product product strategy in place? You know, if an investor comes comes and look into this because they are excited about your product, are they seeing only a product or are they seeing a company? And, so that's where my focus has been in there, in helping those entrepreneurs. And, I I chose technology and gaming because it's an exciting place where there's a lot of investment, and, and, and and those entrepreneurs tend not to have a lot a lot of business experience. And, and I have a lot of affinity there because, Full Sail was is a leader in in in gaming technologies and education, so I have a lot of affiliation with that. And, and that's why we went in into that.

 Luis Garcia [00:22:46]:
And, so, my focus has been in trying to give them room to do everything else and, besides, building a fantastic product.

Michael Berk [00:22:57]:
Got it. Okay. Cool. That makes a lot of sense.

 Luis Garcia [00:23:00]:
Yes.

Michael Berk [00:23:02]:
Alright. Well, thank you for that brief interlude.

 Luis Garcia [00:23:04]:
Mhmm.

Michael Berk [00:23:04]:
Back to Pete. So, as you were alluding to, it offers, the ability to generate technical materials at scale. How did you get into the AI space for starters?

 Luis Garcia [00:23:17]:
Well, I mean, I think we all being in AI, whether we like it or not, for a very long time. And and, every time I grow an organization, at some point, you you run into the exact same problem. And, the first the first person you hire learn from you, the third person hire learn from the second, the 4th person. But that doesn't scale when you are in 50, 60 going to a 100. And, things become more complex and, and and you need to do you need to onboard and and train and address learning gaps and getting people, ready for their job faster, more efficiently. So the solution is, okay. Well, I gotta create a training program for this. And, and, if you have done that before, you realize it's a lot more difficult than you think.

 Luis Garcia [00:24:09]:
But most people haven't done it before, so what they do is say, I'm gonna go to my experts. So I'm gonna go to Ben who had been doing that for a very long time, and you're the expert, so therefore, you're the trainer. And and then, Michael, you've been doing that for you're the expert or you're the trainer. Therefore, you're the trainer. Or sometimes, are the actual experts realizing that they're spending too much time onboarding new people that they say, I need to do something at least for my even for myself. And then they they sit down and try to translate their expertise into training materials. And that's when they realized this is a lot harder than I thought. And, so I probably they perhaps they thought, you know, you know, I'm I'm a programmer too, so I'm gonna say, this without wanting to offend anybody.

 Luis Garcia [00:24:52]:
But we think we're smarter than anybody else. And, and that's it. Oh, that'll take me a couple hours. And and and they they realize it's not even a couple days or a couple weeks is really when you try to do something, some, good training materials, it takes a couple months. Because the process of translating expertise into content materials for training is not simple. And, it's called instructional design. You hire a person whose job is to interview a subject matter expert and trying to translate that knowledge into training materials. And and that's very expensive.

 Luis Garcia [00:25:27]:
It's very, very complicated to do. In in the world that I'm describing, usually, you don't do a good job. The expert creates some PowerPoints or some Google Slides, put them in a in a drive somewhere, and people cannot take them or some of them don't take it, and you just stumble and live with that problem as long as you can. And, so my cofounder, who is a CTO and and exited from a FICC tech company, had this problem too in in his last exit because they acquired 4 companies, and they're trying to bring all these people along. And they say, there gotta be a better way to solve this problem. And we were friends from here in Orlando, the tech ecosystem. And he said, do you think we can use that, generative AI for that? And I'm like, yes. I was thinking the size of the thing.

 Luis Garcia [00:26:12]:
And, you can literally automate the process of instructional design. And, if not a 100%, at least 90%. And, so build a prototype, and we got clients immediately because they could take the procedures while already they're doing, then put it into our software and get courses out. And, and, we also then acquire a small learning management system, which is a content system for distribution of learning. You know? And, so it's centralized, so it's not living in different drives that people and you can actually track who's taking what. But the part that really excite me is that in the education equation, you have 3 components. You have the content. You got the distribution, and those 2 you know, the distribution is the one that has been addressed for the last 20 something years, and learning management system have been around.

 Luis Garcia [00:27:06]:
And, it's not really technically, it's nothing really complicated there. But the content development is the part that we address, but the part that seldom address is the part of evaluation or learning. In in education, we call that assessment. And how do you know that the person learn? How much did they actually learn? And, in the traditional education world, it's been, like, well, I'll do a this test. You know? I I you know, I have a question, different options, and and, somebody's sometimes they have open ended questions that's more fancy. It's more complicated to evaluate. Features don't like that because it takes a lot of of their time. In the digital world, that didn't really help.

 Luis Garcia [00:27:48]:
Either. Digital world was even easier to automate the test. And, because now the machine can actually know which one you know, the computer can actually do the grading. And, so we've been really stalking assessment for a very long time trying to find out a good method to measure the level of learning. And, most sophisticated methods include, project based, doing projects, and, having a conversation with the learner, you know, trying to go deeper into what they know, what they don't know, identifying gaps, trying to fill up those gaps. You can't really do that at scale. And, you can't do it in person. You can't really do it in digital education when the ratio is structured to to students much larger.

 Luis Garcia [00:28:33]:
But regenerative value can't. And, so we've been building also tools for evaluation of learning when we can have conversational assessment at scale. Because you build a knowledge base to create the course, and you define the learning outcomes that you want somebody to learn afterwards as part of the instructional design process. So why not put a conversational AI on top of that knowledge base and then have a conversation with the learner, a 1 on 1 personalized conversation with the learner to see how much they learn? And, so now we're including that also into our solution. Wow.

Ben Wilson [00:29:09]:
Yeah. Awful lot to unpack there. Yes. I just wanted to add anecdotally. I believe in your company, a 100%. I think this idea you basically productize something that we, in like, maintainers of MLflow as we're we've been working on making our docs not terrible and adding stuff like tutorials and guides. The sheer amount of effort that goes into doing that manually, where sometimes a feature is built, and all you have is the source code. You know, you might have been the person that wrote it, or you can definitely read it because you're, you know, a fellow maintainer.

Ben Wilson [00:29:49]:
But then how do you craft, you know, a tutorial for somebody who's never even heard of this feature? Didn't doesn't really know how to use it or what it's there for. It takes a lot of cognitive load to put yourself in the shoes of that person and craft something. It's it's an art and a skill as well, you know, as a technical skill. It just takes a lot of time. You think about scoping of something like that. Like, oh, let's talk about this, like, introduction to a major part of of this product. How do we teach people about this? Like a beginner. And then how do we teach an advanced person about, you know, specific new features.

Ben Wilson [00:30:31]:
You're now talking about a week of time to talk about a moderately sized feature and another week of revising that because you have to get peer review. But your peer review is not a fellow developer. It's people that aren't familiar with it and get their feedback from this. And that iterative process takes a just a very long time. So we use, you know, we all use chat GPT. We have I have a a series of prompts that I've written that are pages long that I insert into my session and say and it gives very clear instructions about who the audience is, what the tech is, what code I'm gonna be pasting in, what sort of style I want, formatting, and what tone to use. And we can bang out, you know, new feature documentation. You know, we don't copy paste directly into the docs, but having that that assistive tool with that process means that we can get a full tutorial or a guide done in a day that used to take a month.

Ben Wilson [00:31:43]:
And as another anecdote to to what you're saying, that evaluation thing, we actually built a prototype to do something like that about teaching concepts. Just did it for fun. It's like, I wonder how well chat gpt could do something like this. Like, if I get a good prompt and then give it a couple of good good and bad examples of something, I was shocked at how well it could evaluate my own knowledge of content that I that I gave it. And when you start talking about marrying a system like that up to, you know, vector search indices where you can provide contextual information and then get, you know, massive document dump that goes into the the input to generative AI. And it can look up the right answers in a vast quantity, vast body of text. You all of a sudden can create like, I was I was amazed at at how much fun I was having with it. And,

 Luis Garcia [00:32:38]:
That's so cool.

Ben Wilson [00:32:40]:
Yeah. Did a a scrape of Wikipedia for, like, some obscure topic that I didn't really know much about. I was like, evaluate my knowledge. It started generating questions and stuff, and I was typing it. It's telling me, like, you have no idea what you're talking about. Here's the right answer. I'm like, oh, this is awesome. Wish I had this when I was in college.

Ben Wilson [00:32:58]:
You know?

Michael Berk [00:32:59]:
Yeah. Absolutely. Are you familiar with Conmigo? I I am not. So, the Khan Academy created

 Luis Garcia [00:33:11]:
Oh, yeah. Yeah. Khan Academy. Yes. Yes. The so they they very love what they're doing in that direction. They wanna actually be in the in the support and assessment side of it. And, I think they're starting more with the support side, which is kinda just answer questions in the from the student without and they also wanted to have guide rails where it will give away what, the answer without giving away how to do the answer because they want they're scared that the the students are gonna use it to do their homework, which I have mixed feelings about.

 Luis Garcia [00:33:45]:
But but I can understand why they do it like that.

Ben Wilson [00:33:49]:
One thing that really made me nerd out about your product though is the ability to, basically, text to speech in, in your own voice.

 Luis Garcia [00:34:05]:
Yes.

Ben Wilson [00:34:05]:
And how well do is that received by both trainees and curriculum developers?

 Luis Garcia [00:34:16]:
So is

Ben Wilson [00:34:17]:
Is it uncanny valley level of people being like, that's kinda weird here in, like, my colleague's voice even though I didn't I know they didn't record this, but it sounds exactly like them. Or does or does everybody just love it?

 Luis Garcia [00:34:29]:
Yeah. They actually love it. And, when the instructor or creator of expert have already created courses that included their voice, they have gone through the frustration on writing a script, getting behind a microphone, reading out, and then playing the course in front of people or or getting and then realizing that was in the right tone, that was in the right phrase, and then then going back and rerecord and rerecord and rerecord and, rewriting and rerecording. And, so we have seen we have had customers that have licensed our product, yes, because of that. It's it's just gonna give me so much time back That is that feature alone is worth is worth is worth having your your product. So that's for people that already tried it. For people that have never really developed courses and they really love the, the idea of using artificial intelligence for training. So people think of training as something really personal still.

 Luis Garcia [00:35:41]:
And, there's there's something that is is a little uncomfortable about it. So having their voice be part of it is something that brings them a lot of comfort in that because it humanizes, of course, a lot. It's just not a bunch of slash with with, with bullets in them. And, it's it's it's more than that. And for the learner, it gives you another, another cognitive dimension beyond readings only. It gives you the reading and the voice, as well, which is is is known to help in, the process. Some people will just listen to the voice or, are not reading. Some people just wanna read, but some people wanna do both.

 Luis Garcia [00:36:22]:
So it's it's it helps the the learning process for sure.

Ben Wilson [00:36:28]:
Potentially silly question for you. How many users have attempted to generate training material with sir David Attenborough's voice reading?

 Luis Garcia [00:36:39]:
Well, we we, so we we have very close stuff our our demo side. So our demo side, they cannot supply their own voice. We we supply the voice, so we don't have, to do that and also to be. And and in order to get the, a course with a voice in it, we have to we are in the middle. And, so to to make sure that, you know, somebody's gonna not do something undesirable with it. Or, I mean, more important to us, we wanna generate leads. So so that's why we put a a gate. They'll in the other side, you know, it's 1 per customer.

 Luis Garcia [00:37:21]:
Usually, the customer uses one voice, and that's what they go for. And, cool thing is, like, we're experimenting now is with with different languages in the same voice.

Ben Wilson [00:37:32]:
Nice.

 Luis Garcia [00:37:33]:
And, customer that may have employees that speak different languages, the expert the expert and the learning outcomes are the same. It's just one course, but then we can deliver a version of it in different languages and basically have somebody that doesn't speak Spanish speak Spanish.

Ben Wilson [00:37:52]:
How well does it get? Now we're going into nerd tech territory. If you have somebody who's a native Spanish speaker from Mexico City, record their voice, and then you have, can you actually have your tool say, I want a Colombian accent? And will it actually do it correctly?

 Luis Garcia [00:38:16]:
Yes. We have not done that. I act I have my I have my reservations with that because I do think that at that level, at that point, you're really being on a. And, and, so customers in Latin America like, we have customers for Argentina right now, and they do have a strong accent, and and they, we ask them. We supply them with with with scripts, and they have to read I I think it's important for the for the personality of the structure to come through even if the voice is being generated synthetically. And, so I actually wouldn't recommend they do that. And, I think it will be a technical challenge to do as well, but I don't think that it's a it's a good thing to do. Now that doesn't mean that they should have tried to avoid, expressions that only understood in particular regions.

 Luis Garcia [00:39:06]:
You should always try to avoid that a little bit if you want the content to be consumed in the whole region. But I don't think you should strip down your accent, to do that.

Ben Wilson [00:39:19]:
Interesting.

Michael Berk [00:39:20]:
Yes. That does make sense. Yeah. If you're looking for consumable content, sort of the the traditional Spanish or the traditional English. I mean, at the same time, though, typically, in in language settings, there's British English and American English.

 Luis Garcia [00:39:35]:
That's right. So yeah. You you could I don't I I I think that people should be who they are and, and and the idea of this is enhancing enhancing what you do be able to, multiply your effect, not to make you be somebody you're not. However, it it does I do I I I I am smiling at the irony that you just said that I can make you speak a different language that you actually don't speak. And, and I don't know how genuine that is, but, but I do think suppressing somebody's accent and try to speak in a different accent, that is changing your personality.

Michael Berk [00:40:14]:
Right. Yeah. That makes sense. So I had another question about, sort of the rag application of basically putting documents into a vector search. Mhmm. Chunking them, making sure they're easily retrievable, and then using that to create content. That's a a well defined use case as of the past 6 months or whenever it came out probably a lot earlier. In terms of evaluating human learning, that is such a fundamentally human thing.

Michael Berk [00:40:45]:
And just initially, I'm wary to offload that to artificial intelligence. For instance, is retrieval, like, knowing a topic and being able to regurgitate a fact, is that knowledge? Or do you need to be able to work with those facts to build and create something new? So with that, how do you guys define knowledge and learning? Is there, like, a default that's it's probably retrieval. Right?

 Luis Garcia [00:41:09]:
That's a good question. And, it really depends. The way we think about it is that we think about it in learning outcomes. And, and in education, you think about their levels of of of of knowledge, requires if it's fundamental knowledge, all the way to the pyramid if you are if you are in in the adding model or whatever. And that is that as you move up, levels of of cognition, you move becomes information that requires a lot more critical thinking. So and, so when you define a course, you learn define what was called learning outcomes, which is what do I want the learner to know? And that and that very well could be just repeating something back. And, and there are roles that are purely memory based. In that.

 Luis Garcia [00:41:57]:
If, for instance, one of our customers, we have several venues. For some reason, venues really like our product because they have a lot of, temporary people that come the the game of the show the day of the of the show, the day of the game. Those are not employees. Those are temporary employees that they came in that they go from 50 employees to 2 50 on a day and then back to 50. Right? And, so those additional 200 employees receive very little training. Creating digital courses for them is cost prohibited because it's very small margins. But with our tool, they can do it. So now they are training them in things that they didn't know before and, like, a little bit about the history of the of the of the venue, some customer service, some facts.

 Luis Garcia [00:42:46]:
And that very much is memorization. So the learning outcomes for those courses are be able to repeat back. And, what you're talking about is, okay, what happens when I have to take 2 or 3 pieces of information, synthesize it, upgrade something new, which is higher level thinking? Then you have to define learning outcomes that will get there and then define the activities that would give you those learning outcomes. You have to kind of teach the whole machine, the whole process of learning in order to get there. We haven't really done done that yet, and, so it will be very interesting. For sure, memorization and, and some level of of interpretation is definitely can be done. We've done, test for, like, call center. And so so call center, you have to make sure that you meet certain benchmarks in in the conversation.

 Luis Garcia [00:43:41]:
And and AI can very much evaluate those and give you and give you a good grade or a bad grade or even a a suggestion based on that. But if you were to have a course on how to come up with a new relatively your relativity theories in the in the in the in quantum mechanics, I don't know I don't know yet. Usually, not our customers. They're not trying to do that. They're really trying to do very basic stuff. Yeah.

Michael Berk [00:44:09]:
Probably a different demographic.

 Luis Garcia [00:44:11]:
Yes. I

Ben Wilson [00:44:12]:
mean, I think this like, looking at your product and the market that you're trying to address seems, like, very broad when this you know, so I think the stuff like this is gonna eventually really take off with respect to things that people don't really think about too much, which is even for extremely high technical positions in roles. We just hired a new manager for our team. So, like, 16 years experience in some of the biggest tech companies in the world. And this lady has some serious software development chops, and is a brilliant human being. But coming into a new organization that's already pretty mature and has a whole bunch of history about things that for people that have been around for a while in that organization, you don't even realize how much tribal knowledge has been built up. And then you start you start realizing how potentially broken some of the systems are for how, like, inefficient it is to do something when a new person who has context at other places comes in and they start asking questions. Like, hey. How does how do we get this thing to to happen? And you're like, man, this isn't intuitive, but I have a feeling that there's gonna be, you know, 1500 questions that are gonna be asked over the next 9 months about critical business processes, about how to do this role just to onboard somebody.

Ben Wilson [00:45:49]:
And having a tool that can sit there and go through all of our documentation, like our internal documentation that is private within our department. Like, what are all of our processes for doing stuff, like responding to an emergency, to how do we follow-up from a stripping a regression, to what are all the processes for doing stuff like planning out work for a quarter, or how do we manage stuff like making sure that our our engineers are taking off enough time throughout the year. There's all these procedures that we have and processes that are just they're everywhere. They're in all these different systems, and you have to you have to really know how those systems are designed to even find where that information is or whether that information even exists in written form. So having a rag solution with all of this advanced capability and being able to just say, you know, because I've been looking at your your FAQ and and your use cases. It's just super cool. You can have a list of chatbot and say, hey. I have a a general question about this thing.

Ben Wilson [00:46:58]:
And they can get the exact right answer instantaneously. You know, within seconds, you're gonna have, here's a summarization of this, and then here's a link to this document if you wanna go and read the whole thing and have that actually prompt that person with, like, what what other questions do you have about this, and I can go find out more information. I I think that's gonna be just part of core business operations. And it's such a critical part of efficient operations at a company is to empower people to be successful earlier in their time.

 Luis Garcia [00:47:34]:
Yeah. I I agree with you. In fact, one of the use cases that we do not have yet and, and I love to have is with technical people because no matter, who you are, how much experience you you have, every time you onboard a new technical person to your team, they have to learn your tech stack. They have to learn your processes. They have to I mean, there's no way other way around it. And, and, and, usually, they just stumble until they finally get it and they're integrated. And, and, I do not know many technical teams that will go to HR and tell them, helping me create an onboarding for my team. They they will not do that because they think that they won't be able to do something as good as they can, but they don't sit down and do it either.

 Luis Garcia [00:48:24]:
And, and so I love to to have a use case specifically for tech stack, for, for technology teams because I think it will be a a a very a very a very big hit in, in technical teams.

Ben Wilson [00:48:42]:
Yeah. Because you'd have supporters not just from, like, the human resources department. Like, I think they would be supportive of that. They're like, hey. This helps retention. This helps people feel like they're they're cared for, basically, at the company when they're coming on. But to anybody in in engineering leadership who wants to get an employee enabled and empowered to contribute and to feel like they're not lost at sea wondering, like, I don't even know who to ask this question. I asked these 3 people on the team.

Ben Wilson [00:49:12]:
None of them know this because they've never had to work on this thing. But having a source system, they can say, hey, where does this one part of our tech stack run CI every night? And then, like, have that system be able to look that up and understand deployment architecture of, like, hey, For this particular test failure that you're seeing, this actually runs here. Here's the link to gain access to the system.

 Luis Garcia [00:49:42]:
And you you you're not teaching them technology and, or how to program or any of that. We're not taking any of It's just just teaching you how this organization works. And

Ben Wilson [00:49:53]:
Exactly.

 Luis Garcia [00:49:53]:
And, so, anyways, I I'm I'm so glad that you get it so well and that I just support you.

Michael Berk [00:50:01]:
Yeah. Just to add a little bit, it's also a really nice application of Rag because, at Databricks, we use glean, which is basically ingest a bunch of documents, put it into a vector store, put an LOM on top of it. And a common issue with these is, yes, they apply citations and things like that, but how do you know it's correct? How do you know it's up to date? And then there's a lot of back end that you have to, implement so that you have all of the sources. So Google Docs, email, Slack, Confluence, you name it. And with your application, it's one static dump of content. It creates courses, and then from there, you can optimize within that static set of documents. So it's, it's sort of easier on the back end, and then you because of that, you can focus on the actual human knowledge, human learning, the growth aspect, which is really exciting to me.

 Luis Garcia [00:50:50]:
Thank you. Thank you.

Ben Wilson [00:50:52]:
And it reduces the the information overload as well. Because, like, smart search engines, I'm not the biggest fan of, because usually you have data dumps internally in your company. Like, you have oh, we have a 150,000 Confluence pages that have been contributed to you over the last 10 years. Nobody's pruning that stuff. Like, sometimes groups that maintain the source of truth, they're gonna go in there and and clean that up and maintain it. But a lot of times, like, how do you effectively flag stuff as stale, out of date, needs to be revised, and where do you get the people to revise all of that stuff? Training courses give you the ability to restrict that amount of information and refine the input data in a much better way. It it prevents you from that that explosion of like, I don't know which of these things that returned are actually valid right now. Yeah.

Michael Berk [00:51:55]:
Awesome.

 Luis Garcia [00:51:55]:
I would

Ben Wilson [00:51:55]:
use it today if if we had

Michael Berk [00:51:59]:
it. So I have one final question.

 Luis Garcia [00:52:01]:
Yes, sir.

Michael Berk [00:52:02]:
So we've talked about evaluation.

 Luis Garcia [00:52:04]:
Do you guys speed up

Michael Berk [00:52:05]:
the learning process beyond just having an interactive assistant to work with? Do you have any, like, tips or tricks that you actually implement on the back end so people learn better?

 Luis Garcia [00:52:16]:
Man, that's exciting. And, I don't think we we have been around long enough to be able to get there. And, but I think it as as we go we go a lot longer and have a longer engagement, we definitely can get there. A part of that excites me is be able for the system to tell you the learning gaps instead of you telling the system the learning gaps. And, and that's all. Let's say, you know, we have a a the the the organization creates a series of courses that everybody has to take. Right? But then be able to go, in the background and say, okay. Well, everyone all these people took these courses, but Ben is missing on this and Michael is missing on that.

 Luis Garcia [00:52:59]:
Let's create micro courses on the fly for them so that they can fill those gaps. And I think that's completely doable and, is in the road map, but, we're probably a year away from that. And I think for sure, I will if we had that today, I would tell you, yeah. There's no way we're not affecting the learning speed. And, for a lot of our customers, we are replacing a we're replacing nothing. We're replacing a manual of processes. And, the the best they can do is say these people I give it to this person, and I give it to that person, and, hopefully, they'll rate it. And, and so the the difference is massive from that into having a course that people take, you know, ingest and, you know, and and and get evaluated on.

 Luis Garcia [00:53:50]:
And, but I think that's just the the very basic. I mean, we can take this very far.

Michael Berk [00:53:58]:
Yeah.

Ben Wilson [00:53:59]:
Yeah. I'm thinking back. Like, now this discussion is making me go back decades now into when I was studying for, the NRC board. So whenever you're going to get qualified to operate a nuclear reactor in

 Luis Garcia [00:54:16]:
the

Ben Wilson [00:54:16]:
United States, You gotta study a little bit, just just a a little. But you get evaluated on multiple different fronts. And it's a huge human investment, not only on the learner, but on the teams of trainers that have to poke and prod and figure out where are your gaps in order to prepare you for your final assessment. And that process is like, the first time you do it, it's 6 months. Well, you have 6 month classroom training. That's very intense. It's like 14 hour days collegiate level. Like, some of its grad level courses where you're learning stuff like nuclear physics and, you know, heat transfer and fluid flow and metallurgy

 Luis Garcia [00:55:02]:
and, like,

Ben Wilson [00:55:02]:
how the academic aspect of how these things are built and why they work the way they do. But then you have to go into a training that they call prototype. And the prototype trainer is 1 month of classroom instruction followed by individual self guided learning, where you have topics, and you get this massive book that's given to you, like a 3 ring binder that you carry around for 6 months everywhere you go. And that book is filled with on both sides. It's if I remember, it was, like, 400 pages long, both sides, fairly small font. And it's just a bunch of bullet points, you know, sectioned off into sections, and sort of groups of topics. And each line is a sentence about, like, one meta question. Explain how the ship service turbine generator works, or they're explain how the control system works for voltage regulation on turbine generator.

Ben Wilson [00:56:09]:
And that question has a box next to it. Well, 2 boxes and a barcode. Signature box 1 is for the learner to confirm that they have self taught themselves, and you're given this massive library full of all those, like, classified information. Your job is to independently research that topic and learn it as quickly as you can. And then the second box is for an instructor to sit you down in a in a meeting room that has 3 walls of the room are whiteboards, and you get this big pile of whiteboard markers. And they ask you questions. They drill you on that that topic. You can bring reference materials, schematics, and diagrams and stuff in there with you, but you can't bring the actual answer books with you.

Ben Wilson [00:56:58]:
And you get evaluated on either for really quick stuff, it's 15 minutes. For the most complex stuff, it's 4 hours, in that room. And you do this 6 days a week for 6 months. And then interspersed with that is physical on the job training, where you're you're sitting with a staff instructor operating a real nuclear reactor, and they're there to slap your hand in case you're gonna, you know, do something bad that's gonna damage things or hurt people. But that whole process is super stressful. Like, the dropout rate was 40% or something in that in that prototype trainer. The classroom, that was, like, 90% dropout rate. But in the the prototype, you know, we had 40% of people that just dropped out of that school because of the intensity of, like, hey, I have all these things that I need to learn really quick, and I I don't even know where this information is, or I can't self adjudicate.

Ben Wilson [00:58:00]:
Because if you sign up for that hour evaluation for a moderately complex topic, you don't find out until minute 59 whether you failed. And if you fail the assessment, you gotta go back to your cubicle, go get the books, and learn it, and you can't go and resubmit for an evaluation that same day because they don't want short term memory to influence being able to pass that. It has to be a long term memory. So you have 72 hour wait time if you fail on eval to come back and reassess. So there's all these rules around that. But I can see tools like this if they're enabled to have, like, that that customized training and to be able to distill concepts down to here's what you really need to know, and then here's some supplementary information to really dive into these questions, preparing somebody in the most efficient way possible to be successful so they can safely do this job. Yeah. I think that that the opportunities are are pretty broad for tools like this.

 Luis Garcia [00:59:05]:
Yeah. I mean, we are up for really, transforming that learning. And, not only us, but other companies that do and say we've been talking about transforming learning for a very long time in higher education. I think it's finally gonna happen.

Michael Berk [00:59:23]:
Then I had one more question. Sorry. Do you guys offer certifications and, like, build certification courses?

 Luis Garcia [00:59:30]:
We we we don't. And, because we're not in the content business with the platform for people to create our content, so what happens is that organizations will create the the courses, and then we'll establish that the learner taking that course certifies them in what they need to know. And Got it. There's there's not really authority on certification. Yeah. And, usually, it's the it's the owner of information.

Michael Berk [00:59:54]:
Yeah. But theoretically, if there's enough trust in the tool, you could create courses that are robust enough to be certification courses.

 Luis Garcia [01:00:02]:
Yes. We could. And and it has been asked before that do we wanna start in the business of of having a catalog of courses? And, and I've been in that business, and I don't. It's a very difficult business to be in. And, and that's how that would be a whole another podcast.

Ben Wilson [01:00:19]:
Yeah. You have to hire a whole team of experts

Michael Berk [01:00:22]:
Yeah. To, like,

Ben Wilson [01:00:22]:
get that stuff up to date. That's why I think it's an important thing on your FAQ that I read. That is I think it's a really smart decision that you made in that a lot of companies feel pretty protective of their intellectual property for internal documentation, internal processes. That's that's what makes their business successful in in certain ways. And your tool is very adamant, and it's very prominently displayed on your website. We don't own your stuff. We don't take take copies of it. We don't store any of your data.

Ben Wilson [01:00:57]:
What you what you generate is owned by you, and we're just giving you the tools to build the that content, and then you manage that.

 Luis Garcia [01:01:06]:
And I

Ben Wilson [01:01:06]:
think that that's a recipe for you to get support in governmental type stuff. Because other competing products that might be doing, like, oh, we'll build courses for you. Yep. That'll never fly in, like, a secure environment. We

 Luis Garcia [01:01:22]:
we as part of our onboarding, we cocreate because this still, you have to train the customer on how to use the tool. It's you know? And, and sometimes they don't know where where to start. They can't really think an instructional as an instructional designer and say, what is the course I should do? You know? And, and, so we cocreate with them, but we don't own it. And, and, and our goal is to cocreate until they're ready and they're out. You know? We don't wanna be in the content business. I've been in the content business. It's really hard business to be in. And, must try to be in the platform business.

 Luis Garcia [01:01:57]:
But if you need to help them, then you help them.

Michael Berk [01:02:01]:
Cool.

 Luis Garcia [01:02:02]:
Mhmm.

Michael Berk [01:02:03]:
Alright. So I think we're at about time. I will quickly summarize. So first, when doing business internationally, empathy is essential. Second, when you're pacing projects, Luis specifically looks for little 5 year chunks. That'll give you about a 2 to 3 year time frame to see if it's catching on. And then, from there, you can go into other things and have other people take your position. And then learning is built on memorization of context.

Michael Berk [01:02:28]:
And then from there, you can go perform higher level thinking with the chunks of context that you learned. And then finally, do not be in the content business. So, Luis, if people wanna learn more about you or Pete, where should they go?

 Luis Garcia [01:02:42]:
Well, we have a the easiest domain possible is pete.compete.com, and, you can read all about it in there and, and and troubles align if you want a demo, request a demo. We're also in LinkedIn and at Pete Learning, and, and, you can find me there, or you can find me in LinkedIn at luisiegarcia. And, find me in LinkedIn and connect with me. I'd love to show you our platform. If you think your company can use our help, love to give it to you. Awesome.

Michael Berk [01:03:14]:
Alright. Well, until next time. It's been Michael Burke and my cohost. Ben Wilson. And have a good day, everyone.

Ben Wilson [01:03:20]:
We'll catch you next time.