Matt Van Itallie is the Founder & CEO at Sema. This episode covers a wide range of topics, from the impact of AI and machine learning on software development and educational systems, to the importance of code reviews and career advice in the tech industry. Matt Van Italy shares his diverse experiences in law, consulting, public schools, and the tech sector, emphasizing the value of using data to drive improvements.
The conversation also touches on the use of GenAI tools in development and the need for organizations to embrace new technology to stay competitive. They also explore issues such as defense spending, career transitions, and the significance of investing in education and human capital.

Socials

Michael Berk [00:00:09]:
Welcome back to another episode of adventures in machine learning. I'm one of your hosts, Michael Burke, and I do data engineering and machine learning at Databricks. And I'm joined by my co host,

Ben Wilson [00:00:19]:
Ben Wilson. I look at logs at Databricks.

Michael Berk [00:00:24]:
And today we are speaking with Matt Van Italy. Matt studied at Swarthmore and Harvard Law. And after gaining his JD, he entered the professional world as a consultant. After about 4 years at McKinsey, he moved to help manage public schools in the Baltimore and DC area. Then after that, he entered tech specifically focusing on impact focused consulting and development. And today, he works at CEMA as a founder and CEO. This organization focuses on uncovering legal and regulatory risk in software. So, Matt, starting with a super, super technical question, why did you decide to settle in Baltimore?

Matt Van Itallie  [00:01:01]:
I was moving up and down the East Coast. First off, I'm so glad to be here. Super excited. Second, I'm gonna pull up an Excel spreadsheet to answer that question, maybe some pivot tables because it's it's pretty it's pretty technical. I was moving up and down the East Coast and, looking for a place to settle down when we had kids. And, Baltimore has so much, to so many good things about it. Great quality of life. Great, great access, to other cities and grandparents and all of that, and it's been a a wonderful place to call home.

Michael Berk [00:01:37]:
Nice. And was it a good jumping off point to work in the public school district because you're so close to DC?

Matt Van Itallie  [00:01:43]:
Yeah. So I after I was a management consultant, I did about 5 years serving, school districts, in and around, in the United States, low performing school districts. I did it in Baltimore. I was chief of analytics, for the school district. I was in New York. I was in d, in DC. I did some time in Cleveland. So when I was, it was really the last stop.

Matt Van Itallie  [00:02:08]:
Working in the Baltimore City Schools was my last stop on a, direct education service journey, and so decided to put down roots here.

Michael Berk [00:02:16]:
Awesome. Yeah. So for a little bit of context on my end, I've been volunteering with a nonprofit called Learn to be. They provide free online tutoring to a similar demographic, it sounds like. And I worked with a lot of their raw data processing, basically, who should be accepted as an applicant, who should not. Are the students learning? That's a really difficult challenge. So what does data look like in these systems, and what are the challenges surrounding that?

Matt Van Itallie  [00:02:45]:
Of course, the, you know, the fundamental challenge and the grand challenge, probably the hardest one of the hardest policy questions on earth is how do we take poor kids and pull them out of poverty, and how can we help use the educational system to do that? I don't, I've lived for a while now and had the opportunity to work on pretty hard problems, and I think that is that is one of the hardest. Multigenerational poverty, is such a challenge for most first and foremost for the families, you know, who are affected. I am, I'm the son of a math teacher. You heard beforehand, I coded early and then my my father was a coder too. So I was I have a I have a moral commitment to data. And so I do that's my bias. But I do come in and believe that understanding data and applying the right context to it can make almost any problem better. Huge caveat for the context.

Matt Van Itallie  [00:03:39]:
Data without context. Are you writing more lines of code? God's sakes. Right? Like, you gotta you gotta put the context in right. In the context of the educational system, I absolutely am in the favor of high quality assessments, that understand how much students learn, running them regularly to make sure they're getting the right kind of education. We've all been in classrooms or settings where we've been taught something we already know or something we have no way of understanding, and that stinks. And just because kids are the same age doesn't mean they have the same educational need. So the only way to know that is to ask them, do you know this or not? And then map educational, work to it. So on one hand, rigorous quantitative testing, I I am definitely in favor of.

Matt Van Itallie  [00:04:23]:
But on the other hand, understanding schools and systems of schools, you have to bring in the qualitative expertise. But in the absolute best way I've seen it is professional teachers, who become observers to do classroom and, classroom visits. And what they're looking for, in those inspections and visits is this is really simple. What percentage of the time are the students, observed students what percent of the students, excuse me, are engaged in an instructional activity? Are they writing? Are they reading? Are they taking notes? Are they asking questions? And, of course, that becomes a form of of quantitative data as well, but it's it's only observable truly by, the qualitative use of of professionals. And the combination of the 2, tells a really rich story about, how much students are learning and whether the the classrooms they're in, are set up for their success.

Michael Berk [00:05:18]:
Got it. And is is student evaluation the primary use of data, or is it more at a meta level at the school level?

Matt Van Itallie  [00:05:27]:
Oh, on a great philosophical question. So, different people disagree, but you asked me. I'm on the show, so I'm gonna tell you what I think. I think there are lower stakes and higher stakes uses of data, and that there is not a substantive black and white difference between when you're using it for lower stakes and when you're using it for higher stakes. It's all on a spectrum. And the question I like to ask is how do we put that data to use to, to improve outcomes? As an example, if you did have these, these, instructional experts come into a classroom in September and observe that 0 out of 30 kids, 30 is too many, but it's real, 30 out of 0 kids are instruct are getting instruction, there is a coaching conversation with that teacher. How do we move it from 0 to north more than 0?

Ben Wilson [00:06:24]:
If you

Matt Van Itallie  [00:06:24]:
come back in October, you have another conversation. If that conversation is in May, unfortunately, it usually takes that long and it's still 0. That teacher's it's it's not a fit. When it becomes coaching and when it becomes evaluation is, there's just a ton of there's ton of gray involved at the at the adult level. At the student level, maybe I do have a slightly different framework. It's just all about time on task of growing. What do you know now, and how can we help you learn more? And so to me, that's not that's not judging in any negative sense. That's understanding what you know and freaking celebrating how awesome it is to learn and celebrating that by giving kids, in this case, but it's true for adults too, the instruction at a level that they need.

Matt Van Itallie  [00:07:09]:
Judgment has nothing to do with it. And and, it is about helping them become the fullest person, the most knowledgeable person they can

Michael Berk [00:07:18]:
be. Right. And does ML come into play?

Matt Van Itallie  [00:07:21]:
So it's been a while since I've, worked in the the ed tech space. Let me think on this for a little bit. So AI, broadly speaking, can be used I was a consultant, so you're gonna get a taxonomy. Again, that's what you get for letting me on. AI can be used, as part of a product. It can be used as part of an internal tool. It can be provided by a vendor, and it can be used for coding, to help coding. That's not a completely exhaustive list, but I think those are the major the major categories.

Matt Van Itallie  [00:07:57]:
So just in, let's say, in the educational in a school district's case, the the school district is not a vendor, so it's not applicable. The school district could be using it, as could be purchasing it as a vendor. So they're not providing it as a product function, but they they're using it as a vendor if they get ML or AI from, from the from their suppliers. They could be doing internal data science on, on the results, without using a tool, and if school districts had school districts don't have a lot of coders. They use commercial off the shelf software, but the vendors could be using it. I am a very big fan. Does that just taxonomy make sense?

Michael Berk [00:08:37]:
I think so. Yeah.

Matt Van Itallie  [00:08:38]:
Okay. So for the software providers providing in, educational, EdTech services to, to school districts, the, you know, the word I know best, They are providing it, you know, in that vendor role. I would expect them to be using to be at least experimenting, with machine learning to see what insights they can get to produce, more useful and more actionable, information for teachers, and more information more actionable insights about, students. I certainly would expect, I certainly would recommend that every developer, on earth, barring a tiny number of certain circumstances we can talk about, has access to at least 1, preferably 2, high quality Gen AI, coding tools because it is such, an advancement in, developers productivity and and and work, work satisfaction. So I really hope, you know, the ed tech company is doing that. Then within the school district, certainly would expect an, some, some machine learning, data science on the results. I do like matching you know, you wanna match the tool to the situation. There's an awful lot you can tell from pivot tabling, to bring that up again.

Matt Van Itallie  [00:09:56]:
If 10% of students were learning in October and 11% were learning in November and 12%, you don't need you don't need anything fancy to know that the the rate of change, is not great. But there certainly are circumstances, especially with large districts, that it becomes a big data problem where you'd wanna apply ML to it.

Michael Berk [00:10:17]:
Got it.

Ben Wilson [00:10:18]:
Got a a question that might be super frustrating, but it's what I've been thinking of the whole time that you've been talking. Please. Big tech gets a lot of money. The defense industry gets a lot of money. And like yourself, I'm a a child of a public school teacher, in math as well. What do you think would happen if the sort of funding that goes to the DOD were applied to public school education and the focus of technological advancements and the application of those tools were applied to the public education system?

Matt Van Itallie  [00:11:02]:
I, I actually am a believer in, in robust defense spending, full disclosure. And I think the ROI of investing in kids, in a serious way would have literally generational, level impact, on our country. And let's just talk about our country, but it's true true anywhere. The difference in life outcomes for the students and their children and their grandchildren, being pulled out of, of intergenerational poverty, 1st and foremost, is so impactful to them, but has community level impacts, has, national level impacts that I would be incredibly supportive of a serious and, method of, of investing deeply and pulling kids pulling kids out of poverty. And, tech is part of it. I know you were talking about giving money to tech, but so much of it is about human capital, and supporting the teachers and having the right teachers and instructional supports, for them to, to deliver that kind of that kind of impact. With reading about, was it Jonathan Kosel? I'm gonna embarrass myself. Just yesterday about a a scholar who's been writing about kids in poverty and schooling for 50 years.

Matt Van Itallie  [00:12:28]:
I'm so sorry. I don't remember his name exactly. And 50 years of really marginal, if at best, improvements in students' abilities to to read, to read and, do math, etcetera, and life skills. And what would it mean if 50 years ago, we really started spending the money in the right way that would really not just more money, but money spent in the right way with accountability. Not most importantly, their lives would be better, but, communities would be so much better too. I'm guessing, Ben, you think the opposite, and the answer is to cut education budgets and and build more submarines. Is that is that your sense?

Ben Wilson [00:13:07]:
I mean, as as a former US Navy sailor who was on submarines

Matt Van Itallie  [00:13:11]:
Oh, in that case.

Ben Wilson [00:13:14]:
I'm not pro get rid of all DOD funding. I think that's insanity. However, I think that societies that invest more in human care and the betterment of society through education and social programs that enable that. So even if your parents are incredibly not wealthy, you still have the same shot as everybody, as the the kids of the most wealthy. So that it's an equalizing for you know, forcing function on society to say, it really is up to you, and society is here to to bolster you up. It's

Matt Van Itallie  [00:14:00]:
stuff that

Ben Wilson [00:14:01]:
I think about at randomly sometimes where I'm like, where would we be? When I'm working on, like, some new feature that we're we're talking about building in our software product, I think, like, if we had invested differently a 100 years ago in this country and there were instead of having this, you know, single digit percent of people who are coming up with technological innovations over time, and instead, it was something like 30% of society was able to have the education that's required to come up with these ideas. Would I be working on something different right now? Just due to the nature of what the the tech to zeitgeist is focusing on, would we have been at a point of having readily available, you know, GPU instances running in in some cloud 40 years ago, and will we now be working on something completely different? It's something that, I don't know, just randomly comes to me sometimes. I'm like, oh, would life be like if we had made different decisions as a society?

Matt Van Itallie  [00:15:09]:
And obviously, we have such a long ways to go, but relative to 250 years ago, one 10th, 1 5th of America when, 5% of Americans had full opportunity, rich rich white, men. And we've slowly but absolutely steadily have brought in women and people of color, and people with lower incomes, to have chances. Are we and the women example is really powerful to me because it had to in my mind, it just happened so fast and then, you know, in our lifetimes with things like Title Mine and breaking down barriers of women in the workforce, the idea that a country could be a com or a community. I'm a boy scout and thinking about leaving your campfire better, leaving your community better. It's it's instilled in me. The idea that you could with a anyone with a straight face could say, you know what we're gonna do? We're gonna take half of our human capital, and we're gonna say you're not worth you're not worth it. And, of course, we did more than that with, with people of color. But, it you do wonder how what how we would be better, and all you can do now is contribute in all the different ways that we can in bringing giving more people their real shot, which I just think is such a beautiful way to put it then.

Michael Berk [00:16:26]:
Do you think money is spent well in the public sphere?

Matt Van Itallie  [00:16:29]:
It's a hard question. It's a really hard question. As a general rule, I would like more, more research and more accountability to the connection between money spent and outcomes. I think we've done, enough experimentation to know what does and doesn't work. We know high quality teaching works. And I'd like more money spent on things that we know work. A little bit of money spent on testing, and less money spent on things that we're pretty confident don't work.

Michael Berk [00:17:16]:
Very diplomatic. Alright. That makes sense, though. It it's, the it's the only argument that I've heard against giving more funding. But reallocating funding seems I mean, without getting too political, it seems like at least an interesting thing to consider, because, again, investing in the country and the future of the country, is for sure going to have an impact. And military is almost like an insurance policy. You'll only need it when you really need it. And in that case, you will frickin need it.

Michael Berk [00:17:49]:
But, if there's peacetime between all countries, we would reallocate a ton of money to very different things. And, Ben, out of curiosity, you said what would you be working on if, we had reallocated? What would you be working on, do you think?

Ben Wilson [00:18:06]:
I wanted to offer a rebuttal to what Please. What your comment just was about about DOD because I do know about that. And I did see things that civilians that I knew at the time and have known since where people are like, Ben, what did you think about the Littoral Combatantship, and what a dumpster fire? Like, yeah. Some decisions were made that maybe weren't the best, in hindsight. But the real reason why I think people find such a problem with DOD spending, and they see this price tag associated with something like, oh, this aircraft carrier costs 3 and a half $1,000,000,000. Like, have you ever been on one? Do you know how many moving parts are on that thing? And the fact that it's it's not an economy of scale. There's equipment on one of those ships, like the arresting catapults for an aircraft carrier, for instance. They're a 120 feet long, 40 feet tall, 30 foot wide, and incredibly complex.

Ben Wilson [00:19:16]:
Took 40 something years to like, just design those things and and build prototypes and test them to make sure that, you know, they're capable of stopping a several ton aircraft traveling at a 140 knots, on the span of an aircraft carrier. So complex stuff, they only make a couple of them. So they're very expensive in all the the parts and stuff. But when you talk about programs where there's some massive investment because somebody thought it was a good idea to do something or are trying to predict for the future, their their outlook, they say, hey. We're gonna be we're no longer deep ocean navy. We don't need that. That's not a threat in the world right now. We need coastal support because that's the type of conflicts we're in right now.

Ben Wilson [00:20:03]:
We need to launch missiles onto you know, from 10 miles out to shore targets. So that wasn't the it wasn't the best way of going about it, like, how they built them and designed them. And what I saw when I was in and have paid attention to since is I don't think the process it's not what the DOD does, and it's mostly a deterrent force, at least the US military is, particularly in the Navy. Don't mess with them because, woah, they have a lot of, you know, ships and and aircraft. It's more the reason that bad decisions get made is more political, I think. The process itself is broken, and everybody's terrified to fix it. So when you're in this hierarchy and the same thing I've seen in big companies too that I've worked at where you're middle management, you're responsible for this this one thing and this project or that spans multiple years or decades sometimes. You're terrified of being the whistleblower even if you see it early on in the project, early on in a project like that a big company now is is tackling, like, in tech.

Ben Wilson [00:21:22]:
Early on is your 1st 2 weeks. Like, you're doing designs. You're especially with, like, software and stuff. Early on in a DOD procurement process, that's 2 years into that. And if you're like, hey. This is a bad idea. You're terrified of getting fired or your performance ratings. You'll never advance up.

Ben Wilson [00:21:44]:
If everybody's doing that, and that's the culture of fear, nobody's ever gonna speak up. Or if they need to get funding from congress, which they do for these things, you now have to play the political game of the 4 year cycle of elected officials in Washington to say, we made a mistake. We actually don't need that money. We're gonna go a different direction. Congress is gonna flip out, and they do frequently.

Michael Berk [00:22:14]:
So,

Ben Wilson [00:22:17]:
yeah, it's there are things that could be fixed, but I think the culture needs to be fixed about how all of that stuff works about what are we spending that money on, why are we spending it. That's my take on the public funding, and I think that extends way beyond the DOD. That's just public funding in general. Like, because taxes are controlled and this the expenditure of taxes are controlled by an elected body of representatives in this country. That's why it's broke.

Michael Berk [00:22:48]:
Yeah. So, Matt, you worked in that space for a while. Is there a reason that you left?

Matt Van Itallie  [00:22:56]:
Yeah. It's, it's extremely, extremely important work, and I have such incredible respect and admiration for the folks who do it day after day, year after year, decade after decade. After, you know, 5 or so years in the space, I came to the conclusion that it wasn't, it wasn't the best fit for my personality. And, organizations people need to match, you know, it's the career advice I give, need to set them the best thing people can do for the world is make sure they're working in a job where they are a good fit for that. And, the way I comport myself, is better suited for the private sector, especially, high growth or fast change organizations. And, I was ultimately able to be to lead a a healthier life, and hopefully long term, make the most kind of contribution despite not directly serving.

Michael Berk [00:24:02]:
Interesting. Okay. So you went into the tech space. Was this a pulling force that you wanted to write software, or why did you decide to go into to this area specifically?

Matt Van Itallie  [00:24:14]:
Yeah. So I went from working immediately in school districts to working for an EdTech company. And so, I think the thought process was, I care deeply about education, and, I am ready to return to the private sector, which sounds cliched, but it was absolutely true. I was ready to return. And while I was there, my interest focus shifted from the content of EdTech to, the content of engineering, running engineering teams, helping engineering teams do their best work, and the systems of, systems of high performing software companies. I love thinking about the systems of school districts, systems of schools within school districts, systems of classes within schools. And so being able to think about systems of school of, software companies check the box on being able to do system thinking and was a fit to my, my metabolism is a better fit for working in the private sector.

Michael Berk [00:25:18]:
Got it. So it seems like you've gone from law school all the way sort of in a very circum whatever the word is, a very nondirect route into this tech space. Do you think that nondirect route was beneficial and give you more of a generalist skill set so you're more dynamic, or would you have rather specialized early? What are your thoughts?

Matt Van Itallie  [00:25:42]:
Yeah. It's a great question. Any good career advice, is the a statement is true and the opposite is true. And both of those, I I actually think one of the best pieces of, I ever read in law school was something called competing canons of construction. Construction is how you understand the law. Competing means at odds, and canon are rules. And basically, Carl Llewellyn and then a a realist scholar in the 19 twenties found a whole bunch of legal things that sound true and, like, yeah, sounds good, and found the opposite that also sound good and said, listen. When we say we're picking a reason because it sounds good, a canon, there's an equally compelling, counter count canon that also sounds good.

Matt Van Itallie  [00:26:32]:
So let's let's be realist and decide what we're actually doing and not use the analogy. Let's actually get to it. I think that's totally true actually in so many things, but including career advice. So the career advice alternative is, and let's call it the Sheryl Sandberg view. Get to the top of your field as fast as possible because once you are recognized as a leader in your field, then you can move from place to place already having established yourself as, established yourself as a leader. The advantage of that is you get to work on problems of scale, repeatedly. You immediately have trust in all of the places you go. It's, I think, a more comfortable life, for sure, if you get as far as fast as you can to the top and then go at different directions.

Matt Van Itallie  [00:27:18]:
The counterexample, I'm gonna use an analogy from, Captains Courageous. Did either of you by chance read it growing up Ben, maybe?

Ben Wilson [00:27:26]:
Yes.

Matt Van Itallie  [00:27:26]:
No? Okay. So Rich Bratt, I think what? Late 1800, going on a a voyage, with his parents back to the sea, of course. All things go back to the sea. Falls off the boat and, is picked up by, a a sailing crew. And, you know, they, they lift him out of the sea, and he goes into his back pocket to, to pull out money to pay them to say thank you. And he's lost his wallet. It fell out. And so instead of going into his pocket, he has to raise extend his hand and shake their hand to say thank you.

Matt Van Itallie  [00:28:05]:
And, to take the path, like, that I have taken, which is reinventing myself at least 3, maybe 4 times, in a 25 year period, I've had to not rely on, some of those other things. A stat a wallet full of status, although I've been incredibly lucky to have some extraordinary training and had to learn things anew and learn cultures anew. The advantage of that the disadvantages is hard, and there are some difficult moments along the way. The advantage is, I feel so freaking lucky to have seen a a really diverse set of experiences that inform what I do today. And I certainly to the extent at all, I'm a I'm a decent CEO, and founder, it's because of what I've learned along the way. I thought long and hard about teaching about how to use data to understand what teachers are up to that absolutely informed the right and wrong ways to think about what developers are up to. Very similar. It's incredibly similar.

Matt Van Itallie  [00:29:04]:
It's it's super easy to get it totally freaking wrong, of how to use data to understand, teachers, teachers or, or, software, developers. Also, I started this business, CEMA, because so again, I was adjacent to tech, but was a consumer of it consumer of it. I went to a this, you know, EdTech company, and I sat in an executive team meeting with extraordinary professionals. And I know this story sounds coined, and I have told it many times, but I swear to goodness it's true. The chief revenue officer pushed a button on Salesforce at the executive team meeting and said here is hey, executives. Here is the status of sales and here's the status of our sales team. And everybody understood and everybody nodded along. And then the chief marketing officer pushed a button on HubSpot.

Matt Van Itallie  [00:30:01]:
Alright. And here's the status of the marketing. Here's the status of our marketing efforts. Okay. So we have code that serves as a chief ex officer tool to help understand what's going on. And you know where this story is going. The chief technology officer, who is great, I love you, Chris, stood up and said, here are 15 pages of PowerPoint and some spreadsheets to explain why we need to do a refactoring. And I didn't know what a refactoring was, and I was like, did you like, did you like, my head is exploding.

Matt Van Itallie  [00:30:31]:
Salesforce is code to understand sales and salespeople at the executive level. Is there not code to understand code and coders at the executive level? What are we doing? And that insight, right, is the basis of our first product Asima. And no way. You know, you're sort of in the water. You don't know that you're a fish or whatever. If you're a fish, you don't know what water is. I certainly come to this with a different perspective having had all those other experiences before I got to and and frankly fell in love with enterprise tech. I I doubt certainly, if I was doing it for status, I'm sure, that would have been one thing, but it's never my choices have never been status.

Matt Van Itallie  [00:31:06]:
They've been impact and challenge and growth and a chance to do some good, and I wouldn't trade it. That's a very long answer. But I I I wouldn't trade it. But anyone who's thinking about that path, my god, is it full of stress. And I'd listen really hard to Cheryl's advice and seriously consider just getting to the top and having a career like that.

Ben Wilson [00:31:27]:
I find it just incredibly logical and also inspiring to see the way you thought about the careers that you're working in. But with respect to you basically took the model of analysis of a classroom or a school district and said, there's a parallel here that a team of developers, like software engineers, would probably not even see, and it's not something that is ubiquitous everywhere that I've been when working with software teams. I've I've seen the most dysfunctional groups of people working together where it's competitive, cutthroat. People are trying to burn one another. You look at, like, PR reviews and stuff they're submitted. It's hostile. People calling each other idiots and stuff. And you look at the like, what that team produces, And the corollary to that is an underfunded, teachers don't care, kids are children of very bad upbringings.

Ben Wilson [00:32:33]:
You know, a completely dysfunctional school where nobody's really doing anything because they've lost motivation. There's no system in place to support them, and there's no visibility. Their little echo chamber that they're in, they don't even realize how dysfunctional it is, I think. And I saw that with software teams too. Because, you know, working as consultant, you you end up talking to hundreds of different teams. And some of them you walk in the room. You're like, man, everybody here hates each other. I can tell.

Ben Wilson [00:33:04]:
Like, that is the vibe I'm getting from walking in this room. And then you start talking to them, asking them questions, and people start getting threatened and feeling like, hey. Who is this guy who's coming in here, you know, telling us that our house is is not in order? But it, you know, it is just like that. And when you go into a really well functioning team, you see the manage manager is basically like a vice principal. A director is like a a principal. VP is like a superintendent. And then your teachers are your team leads who are there, you know, working with everybody in different capacities. It's all customized.

Ben Wilson [00:33:49]:
So your very senior ICs, they don't need to be taught how to code. Their code's good. They're like they know what they're doing. They might need to be taught how to do a product proposal or do a design better or just how to communicate their ideas better. So you have customized learning plan for that person. But then when you have a new engineer that's coming into your team, you are teaching them the fundamentals. You're going back to, you know, hey. Welcome to 1st grade.

Ben Wilson [00:34:19]:
I mean, not 1st grade. Like, freshman year college.

Matt Van Itallie  [00:34:22]:
Courses. Yeah. Yeah.

Ben Wilson [00:34:23]:
And you're, yeah, you're you're walking them through this thing that's not taught in university because it's not applicable to that, but it's still the same process. So those people are teachers whether they realize it, recognize it or not. So it's just it's fascinating to me to see, like, this is the product and the person that that formulated this by applying that same sort of pattern to it because it makes total sense to me.

Matt Van Itallie  [00:34:48]:
Very well said.

Michael Berk [00:34:50]:
Yeah. So what are the tools that you look to enable this sort of teaching environment within a software team?

Matt Van Itallie  [00:34:59]:
1st and foremost, code reviews. Actually, it's not that's not true. 1st and foremost is taking very seriously, the privilege and responsibility to be able to be on a team. And I have found in every kind of organization I have ever been a part of in my entire life, the most important thing you can do to help make a team better is make sure that people who don't fit and aren't contributing have some chance to improve, but then have to leave. And all listeners just think through think through classes you've had where someone didn't pull their weight, think through teams where someone didn't pull their weight. It brings down everybody. It just makes you feel like a chump, and it is hard to be heading in the same direction. Listeners also, you must guarantee that by listening to me, you will follow this.

Matt Van Itallie  [00:35:50]:
Never ever call your business a family. Because your business is not a family. Your business is a team. You're a team working on performance of achieving the outcomes for the organization and its users. Teams cut team members. Families don't cut family members, and your job is to produce, help serve your users and your customers and the organization. And it's all correlated to taking seriously the the hardest thing as leaders, but the most important thing. And you can lead from anywhere in from your seat.

Matt Van Itallie  [00:36:24]:
It's not the title. It's determining people who shouldn't be on that team need to go. So that's number 1, broadly every organization. For coders, number 2 is code reviews. How unbelievable is it that, coding is a craft? It's not a competition. How incredible is it that this this amazing career path of coding is not only the chance to build things, but a chance for learning from colleagues all the time to get that kind of substantive feedback 2, 5, 20 times a week. Are you kidding me? On top of how, on top of the comp, on top of getting to solve problems that you actually get to grow as humans. And so getting a really positive, doesn't mean fake positive.

Matt Van Itallie  [00:37:14]:
I mean, constructive code review process, is, is just think is incredibly important. And then after that, I'm I'm pretty much an experimentalist. I'm not a professional coder, and so it is for the team to be continuously checking on their performance, seriously thinking about blockers under seriously I'm thinking about what could unblock, whatever the heck those things are, and to be thinking about technologies that can improve, improve their work, and improve their quality of life. We're at the beginning of the adoption of coders using AI to code. You think, you know, we're at the, I I don't know, Ben, if this is overhyped. We're at maximum adoption of open source. It's used. People love it.

Matt Van Itallie  [00:38:03]:
It is absolutely understood. The using AI in, using AI is a is beginning days of open source. It's gonna transform, in my opinion, a very good way for developers. So it's just one technology. 1, obviously, I'm thinking a lot about, which is more broadly, leadership and especially nontechnical leadership need to be thinking hard about, like, just incessant focus on blockers and getting rid of them and a commitment to experimentation or at least looking at potential, you know, changes, because technology always changes fast, and it's only gonna change faster.

Ben Wilson [00:38:41]:
Yeah. I I couldn't agree more with the the state, you know, sentiment of, like, people are comfortable using OSS. Like, it that's it is used everywhere now. And a lot of great contributions come to those projects and keep them in states of quality that rival proprietary software, in my opinion.

Matt Van Itallie  [00:39:04]:
Rival and exceed in some cases. React. I mean, rival and exceed.

Ben Wilson [00:39:08]:
And with the Gen AI stuff, I've interacted with some other, you know, teams at other companies in my in my current role, and it's one of the things that always comes up. You know, just out of a curiosity, it's like, how many people at your team are using, like, Copilot or or GPD 4 to, to and how are you using it? Because I know how we use it, every day. Every single engineer uses it. And it's not, hey. Write my code for me. For what we do and what we're building, there's no way that that PR would merge. It's just the broadness of the context that you need in order to build something a certain way. But it's more like, I I need to make a tactical decision here, and I need to decide in my head.

Ben Wilson [00:40:01]:
I know there's 4 ways of doing this. Please give me the 4 prototypes that can explain this so I can put it into a document so I don't have to spend the next hour handwriting 4 different, you know, explanatory prototype code snippets. Or, hey. I'm not quite sure about what you did on this PR. Like, I see what you're doing here, and and I can follow along. I just wanna double check if this is the the most optimal way to do this with this other contextual bit of information that I have in my head that we wanna support in the future. Writing up prompts, writing down your thoughts into the into the GPT 4, and asking that question to give the context, which then allows you to be like, oh, that's the term I was thinking of, and I forgot what it was called. Now I'm gonna go search in the source documentation, you know, source code for this library that I'm using and get the the real answer here real quick so I can give a link to the person.

Ben Wilson [00:41:07]:
So, like, hey. Let's think about this instead. It saves so much time, and that's really what anybody who's not using it and who's writing software, who's not using it to help them draft up designs, write docs, and assist during code review, for just generating code examples or recommendations for fixes or as a linter. If you don't have a good linter for some esoteric, you know, pattern that you're using, it's amazing at that stuff. So I've got 2 windows. 1 is reserved on one of my monitors. One half of it is just for my code interface where I have a 4 page long prompt that I've written to help me with specifically Python open source development. Then I've got another one that's that's configured for docs.

Ben Wilson [00:42:00]:
I've got another one that's configured for Scala. And they all do different things, and they're invaluable.

Matt Van Itallie  [00:42:08]:
Well, Ben, thank you for coming to my podcast on why developers should use AI. I completely agree. Let me say a couple of different things. I'm sure you guys have seen I hope you've seen Inception. Yeah. So the the the world builders who have the ability to conjure something, by thinking about it and constructing it. I I think the prototyping nature of Gen AI absolutely reminds me of that in code, but also not in code in in in documents. Has hallucinations.

Matt Van Itallie  [00:42:37]:
You gotta treat it seriously. But for the can I conjure something and see what it looks like at a high level of detail, is so expansive? Gives you such opportunity to experiment without being weighed down by the by the details of it. So I I I think that's incredible. Second, I do think, Ben, I heard you say, and if if if I can for a teaching moment, some people's some people's concerns are, well, I'm gonna have Gen AI write it for me. And there is that very strong sense. I've heard from some some people, the code is writing it. It's doing the work for me. I know a lot of developers who are afraid of what it means as from an identity conversation.

Matt Van Itallie  [00:43:15]:
Am I really a developer if I'm using GenAI? And I like to say to them, are you a real developer if you use open source? And if the answer is absolutely yes. Why would I rebuild something that already existed? My work is not just keyboard jockey. My job is to think about what the right solution is. Is to start thinking about it from the beginning and bring the right solution to bear. And so I know and we're engineers on our team. We were working on an AI product for 6 months, and engineers came to me and said, is it really okay that I'm using AI to do this? And that's why I say, is it okay for you to use open source? Yes. It is bananas not to use open source. If your company has allowed it and I'll cut to that point third, it would be bananas not to use AI, for all of the reasons that Ben Ben so eloquently said.

Matt Van Itallie  [00:44:03]:
The third point for folks who are not who are in management positions, this isn't too self promotional, but we have white papers that you should give to your leadership teams and your lawyers about why everybody should be using, should everyone every engineer should have at least one high grade, enterprise grade, AI tool, at their fingertips. CEMA does not sell these. So I'm I'm hyping legal documents. I'm hyping legal documents. The ROI in almost any possible situation for coders is so substantial combining their job satisfaction, their productivity boost, the ability to mitigate the risk. The risks are real, but they are absolutely mitigatable. And the cost of an enterprise grade product, the ROI is out off the charts. The finance team needs to understand it and then the legal team and compliance teams need to know how they can overcome those risks.

Matt Van Itallie  [00:44:59]:
But, you know, the it is absolutely in almost every possible situation the right answer as long as it's implemented the right way with the right safeguards. Side note, if the organization is not letting the or the developers do it, developers are doing it anyway. So it is in almost every situation. So let's not pretend that the choice is adherence to no Gen AI code in the code base versus Gen AI code. It is bring your own LLM to work, which is such a problem from a data leakage issue. It's such a problem from a management issue. That is the real choice. 92% of developers are using GenAI today.

Matt Van Itallie  [00:45:36]:
They use it at home. They use it everywhere. And it's not, Copilot and Checkat gpt enterprise are not at 92% adoption. Companies need to need to be honest with themselves. Look at the risks. If you really, really think it's not appropriate for all of your code, okay. Fine. But then put in some tracking to make sure people aren't using it and explain people why they can't.

Matt Van Itallie  [00:45:58]:
Much better have a system in place to manage those risks and give everyone access to the right tool with the right enterprise grade security protections, etcetera.

Michael Berk [00:46:10]:
Yes. Yeah. So to give a little window into field engineering, we are not allowed officially to use chat gpt. I use it many times a day, and I just pay out of pocket. And the rationale for, like, not being able to expense ChatChBT specifically and, like, having licenses for us is there's a worry that there will be customer data leakage. At least that's my understanding, and I could be wrong. That said, we officially support Copilot and give enterprise licenses there. But the amount of times that I've thought about just spinning up a Databricks, endpoint and hosting a closed source LLM just for my team to use and use customer code, I've I've almost done it many times.

Michael Berk [00:46:57]:
The the core issue is that typically those closed source LLMs are a little bit worse than chat gpt4. And so the the like, the decrease in quality isn't worth it for me. I would rather just abstract customer specific information and then put that into chat gpt. But that's that's at least how I've seen field engineering do it. I'm not the most up to date at all on field engineering policy. But as of, like, 6 months ago, that's what it was. And,

Matt Van Itallie  [00:47:24]:
so really would and I I think you Michael raised a really good point. We don't we don't recommend just one tool. Give engineers more than one because they are specialized enough. And, the ROI on multiple ones is still great. It's still great. The hard part is what your engineers are doing at work. Put them to use at the maximum potential. And not everyone Michael is a great example is a great, you know, counterexample.

Matt Van Itallie  [00:47:50]:
Not every everyone at our companies that are listening to this is gonna be as conscientious as Michael on sufficiently sanitizing the data before it is shared. And so because it is so powerful, not for, like, selfish reasons, Michael's not doing this to shirk. He's doing this to deliver the best possible outcome for his, for his customers. Make it easy for him to do that safely. The Michaels of the world, which is everybody. Make it easier for them to do it the right way. So I wouldn't stop, at one tool necessarily if your engineers can make the case for a second tool as well. As long as it's the right kind of enterprise grade and it checks the the right kind of boxes, give them the ability to to use multiple to the right.

Matt Van Itallie  [00:48:32]:
You wouldn't give them one application, but they only use one application. Don't let them don't require people to use 1 GenAI, tool because they're they're sufficiently diverse at this point.

Michael Berk [00:48:43]:
Ben, what's your GenAI stack?

Ben Wilson [00:48:48]:
Some things I can't mention because we haven't released them yet, but, robust. I'll leave it at that. Lots of internal stuff, that Databricks is working on, but also, you know, Copilot, GPT 4. I've got plugins within my IDE that can call, both of those services. I've got, you know, chat interfaces. We built one for MLflow, like an open source server that can interface with GPT 4. I use that in the terminal just to feel cool, I guess. But, yeah, I have multiple ways of interacting with them open right now, actually.

Ben Wilson [00:49:32]:
One of the things that's related to, though, that I saw GPT 4 do just a couple of days ago. We were responding to a security alert that, an open source security reach researcher, found on one of our repos. And they write up this private report, and, of course, we we go to investigate it. And while right like, while validating it and getting a reproduction, of course, I'm gonna, you know, take an example of my fix and the the previous code and submit it, the GPT 4, and say, hey. I'm trying to solve this this, this weird, you know, hacking vector, the where somebody can can basically access the the server's operating system, LFI attack, through this really clever, you know, submission to a REST API. And the response that I got from GPT 4 was your solution fixes this problem. However, it exposes an additional one that wasn't there before, and here's how to fix it. And I'm not a security research.

Ben Wilson [00:50:46]:
I'm a back end, you know, software engineer that knows a very embarrassingly little amount about front end dev, and certainly no expert on, you know, web traffic and stuff and certainly not security. And I just looked at that. I was like, this thing just saved me from creating a vulnerability because it just has this broad general contextual reference set in its training data, that very few humans would be able to understand all of that con context. Like, do they exist? Of course. But they're not writing code anymore, generally.

Matt Van Itallie  [00:51:28]:
And they're not sitting next to you doing your code review.

Ben Wilson [00:51:31]:
Exactly. So it's it's the ultimate pair programming partner. It boggles my mind that people don't wanna use it or companies are saying, we we can't let you use that. So you want you're now taking your entire engineering development team and holding them back from productivity. Well, every one of your competitors who's using and embracing this technology, they're turning each of their engineers' time and doubling it without having to spend any money and build a bigger team or build more, you know, more teams.

Matt Van Itallie  [00:52:06]:
Yeah. And let's be super clear. They are doing it. No one's doing it for nefarious reasons. They're doing it because the risks this is new. It could be dangerous. You know, Seema, we've absolutely honored to serve some of the, world's best organizations, and they've had us study this for, you know, 9 months now thinking about how to mitigate risks. And, of course, tell whoever's listening to listen to go talk to your counsel, but the risks are overcomeable.

Matt Van Itallie  [00:52:36]:
The risks are overcomeable. You can prevent data leakage. You can get intellectual property protection for your code. You can deal with maintainability issues. You can deal with with code quality issues. These are all solvable. Are there some situations where the risk of data leakage is so gigantic that you really need to think twice? Maybe the folks who are using COBOL to upgrade our nuclear arsenal system, maybe you wanna keep that away from the public Internet. And maybe, you know, the equivalent of the software that's, you know, the Coke secret Coke formula.

Matt Van Itallie  [00:53:16]:
Maybe you have certain repos that are off limits, but they are so far few and far between relative to, relative to the huge advantage in advancing safely your intellectual property and advancing your developers. I just can't stress it enough. We did a, you know, our focus is is our focus is AI in the SDLC, but we've, you know, been putting together ROI, you know, return on investment cases for other uses. On the short list, you know, the the highest one we've seen, is and the SDLC. We estimate at least 40% return on investment using some pretty conservative I'm I'm sorry. Forty x, not 40%, return on investment. Document review is, you know, is so well suited for machine learning, but even that's, an AI is maybe 15 to 20 x. Please have your compliance teams talk to us.

Matt Van Itallie  [00:54:15]:
Look at our documentation. We're you know, and read what we've said and the with the research we've done and and come to your conclusions. But the guaranteed benefit of your developers using it compared to risks that absolutely can be mitigated, What I hope my hope is we can sit in March of 2025 and say it is as bananas not to use Jenny I when you code as it is bananas not to use open source. We're not there yet. Maybe develop most developers think it's it's a really good idea. But as an organizations, we're very as a whole, organizations are are are lagging behind, and I want organizations to get to be as bananas as not using open source. Write it by hand. Figure it out yourself.

Matt Van Itallie  [00:54:59]:
We don't we don't make it we build our own frameworks here. Like, it's just it is that transformational. And if you really, really wanna know what I think, it is bigger than open source. It is like Internet level disruption. And so if you could with a straight because it's not just coders, It's every function. It changes different parts of the workflow. Honestly, we tell this to all of our clients. You have a year to figure out putting in at scale where it makes sense, AI solutions across your teams and across functions.

Matt Van Itallie  [00:55:33]:
Because otherwise, as Ben said, your competitors are doing it and you're already falling behind, but you will be toast. You'll be toast in a year. So lean in lean into the compliance side. Figure out what you can and can't do. You can't do all. I'm not saying every single thing needs to be needs to be rearchitected. Let's start with all the higher ROI use cases. Document review, coding, customer support should be enabled, prototyping.

Matt Van Itallie  [00:56:03]:
Every step of the prototyping process should involve GenAI. Write the business case, look at the market research, write 5 different counterexamples of the business case, then prototype it, then anything prototyping related, the whole thing is 10 times faster, to is 10 times faster. And so those start with the higher ROI ones, and then, and then go from there. But it is the faster you get there, the less likely it is you're gonna be you're gonna be left behind.

Michael Berk [00:56:34]:
Where is CEMA hanging its hat and sort of focusing on from an intellectual property perspective, and where is Sima avoiding?

Matt Van Itallie  [00:56:48]:
Give me an example of avoiding. You mean, like, the scope of our products?

Ben Wilson [00:56:52]:
Yes. Got it. So,

Matt Van Itallie  [00:56:57]:
our AI related our our primary AI related product is, it's called the AI Code Monitor, and it is detecting the amount of generative AI used in code basis independent of what l l m is being used. The use case for that is helping understand where developers are using it and are not, including a developer level view so they can get access. They can see how they're doing compared to what they could be doing and understanding their their own stats because that's the best way for for craftspeople to get better is to have data at their fingertips. But then also keep track of, you know, at a global level, the the compliance perspective on, are we using gen AI the right way? 30 seconds of terminology in open source. Once you pick the right library, you should try incredibly hard not to modify it because it's been tested, because it's more secure, because if you start modifying it, you won't be able to keep using the version, the version. You should keep open source pure, unmodified. The the absolute opposite is true for GenAI. Once you get it, you need to review it for correctness, for security, for understandability.

Matt Van Itallie  [00:58:07]:
You need to make it yours and put it in the context. That's blended Gen AI. Blended Gen AI in almost every situation, is better than pure. Now, of course, you can pull in individual lines, whatever that might be exactly right. But the mental model of open source oh, this code that came from somewhere else, not from me and I'm just gonna, insert it but not touch it because it's right. Everyone needs to unlearn that for Gen AI and bring it in. Definitely definitely bring it in but then modify it to make sure it works, to make sure it's understandable, etcetera. And so helping engineers and helping companies, so at the comp at the individual level, developers should be seeing how much is pure versus blended.

Matt Van Itallie  [00:58:47]:
At the company level, if 50% of a customer facing code base is pure Gen AI, it's, probably 25% is pure Gen AI, it's probably time to worry. Because you worry about maintainability. Worry about quality. It's possible. It's possible that it can be okay. But as an organization, folks need to know and and start looking. So that is our focus for the AI Code Monitor. There's plenty of gigantic problems in AI that we are not solving.

Matt Van Itallie  [00:59:18]:
This one is a pretty fun one and and one that's keeping us busy.

Michael Berk [00:59:21]:
Cool. Well said. I I understand the product perfectly. Awesome.

Ben Wilson [00:59:25]:
Pretty slick. I mean, any listeners out there who are unfamiliar with it, go to their website, semasoftware.com. You'll see some very interesting dashboards that get generated for all these different products for just getting a snapshot view of for that product, what are you trying to look at? Like, your development team or this, I've got it up on another screen right now just looking at it. The code based composition report. It's pretty fascinating stuff where you can kinda see where are we overextended here potentially and how are we using this. From a CTO perspective, I think this is absolutely critical if you're concerned with stuff like that. Like, are we overusing this, or where are we using this? And I'm sure you can tie this potentially to why did we get this huge bill this past month? Mhmm. Who's using this so much? Because, otherwise, if you're a leader in tech and you need to understand that and you don't have that mental map of all of these different projects that are using services like that, you're just gonna have to start email blasting out all the tech leads saying, like, hey.

Ben Wilson [01:00:37]:
Who who owns this? Who's doing this? What's this for?

Matt Van Itallie  [01:00:43]:
You're very kind. All kudos to our extraordinary product and engineering teams and implementation teams of who've put their heart and soul into this. I'm really proud of them. And it it's a it's such a privilege to work on something that matters, as you as you both know very well.

Michael Berk [01:01:01]:
Yeah. Alright. One final question before we wrap. Your tagline is you save a $1,000,000,000,000. How?

Matt Van Itallie  [01:01:10]:
Well, I may have to so at CEMA, good things happen because of the team. Bad things happen because of the CEO. So if this is an error on our website, I will take responsibility for it. Our AI product is our second product. Our first product, one of the dashboards Ben's alluding to, is a like a credit score on code basis. Like a it's sort of a CTO dashboard, but at a point in time. And when we use that, in, in what we call moments of evaluation, and the particular one, especially where we got started, was technical due diligence. When a potential adviser, investor, or an acquirer, is planning to invest or buy another software company.

Matt Van Itallie  [01:01:51]:
We also help with sell side diligence. So helping companies getting ready, taking the SAT in advance to see what you're gonna score on the real thing. We have, through that product, analyzed companies worth over a $1,000,000,000,000 worth of code. So taking the market valuations of all of those companies, it exceeds a $1,000,000,000,000. If it says saved, I'm gonna go look it up and fix it immediately because that is we know the truth, Fatima.

Ben Wilson [01:02:17]:
You're exactly what it says.

Michael Berk [01:02:18]:
I was talking

Matt Van Itallie  [01:02:19]:
to you.

Michael Berk [01:02:20]:
You just explained. I misspoke. So thank you, Matt, for calling me on that.

Matt Van Itallie  [01:02:23]:
Not at all. If it was wrong, we, nobody is perfect, but we fix errors as soon as possible. That would be a big one. But, that is a it is an such a freaking honor. And it's not just it's not just, our clients, and our advisory partners, but cost but, organizations who've put their whole life's work into building a a business and then, like, literally getting goosebumps. And then let us be a part of the process to decide what happens to their company and their lives and their, you know, their their family's financial future. It is an unbelievable honor to be part of helping people achieve their dreams by getting investment and and selling their businesses.

Ben Wilson [01:03:03]:
Speaking as someone who has looked into random code bases more times than I would choose to admit. And not this like, now I just do it with open source, and usually it's pleasant. Look at an open source repo, and you're like, yeah. They know what they're doing here. It's very interesting design that they chose, and I can see why they did that. And you can strip all the comments out of most open source packages as a professional software engineer, and you can read it. You understand what the heck is going on for the most part. However, on the enterprise software side where it's a small team of people, you know, maybe 30, 40 people are committing to a repo, over a period of 10 years.

Ben Wilson [01:03:52]:
I've walked in too many times to something like that, and I'm just like, who designed this? Like, how do you even find, like, where an entry point is to this code base, and how how do you interface with this API? It makes no sense. Like, I have to write 4 YAML configs, save them into a directory, then call this module as a script to read from those, then that's called from an API, which then has a REST wrapper going through proto definition. Like, who thought this was a good idea? So I think that's fascinating, and it it makes perfect sense to me why VCs would contact your company and say, can you do a report on this? Because we wanna know if this is a lemon or not. Is this all hype, or is this actually, like, legit and the tech that this company is buying is worth it?

Matt Van Itallie  [01:04:52]:
Well, now, gentle listeners, remember the beginning of this conversation when we're talking about understanding classrooms and understanding happy making systems of schools better. I believe in assessments, quantitative assessments, but I also believe in the qualitative reviews. Now they have that analogy, mind blowing for talking about code. You'll see what's coming. The absolute best way to understand a code base is also a combination of the quantitative and the qualitative. Absolutely. R scan is quantitative. If you have, 10, high risk CVEs and your if you have a 1,000 current developers and you have 10 high risk CVEs, You have a very rigorous, security program in place.

Matt Van Itallie  [01:05:39]:
You have very high quality and this is pre triaging. So like the 10 would be in the back end or something like that. If you only have 10, you have an unbelievable program in place. You have the right tooling. Engineers have, time and the roadmap to do it. If you have 10,000, you almost certainly don't have a tool. You laugh, but they're there, man. Companies that size.

Ben Wilson [01:06:00]:
We we

Matt Van Itallie  [01:06:00]:
see a lot. They don't they probably don't have the tooling, but they, by definition, have not given developers enough time to do it. And what's so fun about treating code as math is then you have to you have to do it by segment. You can't just treat all companies on is a 100 a little or a lot. Well, it depends on how big the code base is. But if you start with the quantitative data, if a company has 0% unit testing, a thousand developer company with 0% unit testing and a thousand, even a 1000 high risk CVEs, and developer turnover is 40%. You know a lot about that company before you open their code base.

Michael Berk [01:06:37]:
Yep.

Matt Van Itallie  [01:06:38]:
And that's the the fact some of the things that the sit not in effect. Those are literally things that the CMOS scan does, but then you have to turn it over to the qualitative experts, to someone who can look at the architecture. Architecture is not reducible to test scores. Right? You know, something like that. You you you need both. But man oh man, can you go faster, and folks like Ben know what questions to ask if you start if you start with the quantitative data that puts it in context. And so we are we are an aid to, the amazing technologists who try to quickly ramp up on new code bases. But, boy, are we an aid.

Matt Van Itallie  [01:07:14]:
Is it different? Is it different? Also shout out because it's free, so I could do this. If you go to our website, you can go to a code based health calculator. So of all this work we've done it, you know, this $1,000,000,000,000. We've done some white papers, $1,000,000,000,000 worth of companies. We found 12 metrics that we think matter the most about the health of a code base. Those are literally in our real product. We slightly simplified it so that people could do it themselves. If you go I just wanted to make sure it was online.

Matt Van Itallie  [01:07:43]:
It's in beta, but it it's accurate. Code based health calculator. You put in basic information about how big your company is because the context matters or how it's gonna get evaluated. And it's about 12 questions that you should know off the top of your head. You'll get the score just like, just like if you were running the scan. Listeners, I'm happy to to talk with you the results if you just give me a screenshot. We don't save it. It's private to you.

Matt Van Itallie  [01:08:07]:
But if you want someone to look at you and see the majority of your code on the 12 metrics that we think are the most important for code based health, It's it's a pretty fun tool. Shout out to, shout out to the team for building it.

Michael Berk [01:08:21]:
Awesome.

Matt Van Itallie  [01:08:21]:
The most important one I know we're late. Most important one and I get to work in, my my favorite overused expression. But, man, it's the my favorite. A huge portion. I started this business to help bridge the gap between tech and non tech. I coded a long time ago, but I wasn't a coder when I started this business. So I was our one of our audiences, before I really started to understand tech deeply. In our code based health scan so code based health is like nonfunctional requirements.

Matt Van Itallie  [01:08:51]:
Of all of the possible metrics that matter, there's one that matters the most for whether or not a company will pass diligence or pass this. And that's not just qualitatively our score reflects the most important factor, most important variable, and it is data. Would any of you like to either of you like to guess what it is?

Michael Berk [01:09:10]:
I have like, nothing quick is coming to mind. Number of 4 loops.

Matt Van Itallie  [01:09:17]:
I like the cleverness, but it's it's a it's simpler one than that. Yeah. I don't know. The retention rate of the major contributors to the code base.

Ben Wilson [01:09:28]:
Yeah.

Matt Van Itallie  [01:09:29]:
Because and this is the phrase I use over and over. If you use it, send me a nickel. I hope I trademarked it sometime. Having a codebase without the coders is like having a half written novel without the novelist. Code doesn't end. There's no amount of documentation that is a substitute for the folks who, who actually understand it and understand how the pieces work. And so in that, you know, in that business where we do diligence, if too many of the major subject matter experts have left, that is the that is the the deal killer. And shout out to, Dave, Mango, who's also observed high developer turnover is an anti pattern, that there's also probably something, is it fishy in Denmark? Rotten rotten in Denmark, that people might be choosing to leave too.

Matt Van Itallie  [01:10:20]:
So that's the single that's the single biggest factor in, if you had to pick 1 about whether or not a code base is healthy is are many of the of the deepest expertise developers still around.

Michael Berk [01:10:33]:
That makes sense on a lot of levels because you're maintaining expertise. It's a signal on the quality of the team and the team culture. Yeah. That that tracks.

Matt Van Itallie  [01:10:44]:
We did not know that quantitatively when we started, and it was fun fun to have data back that up.

Michael Berk [01:10:51]:
So Yeah. It must be so cool to play around in that space, test out hypotheses. Like, my 4 loop example was a bit of a joke, but maybe the number of Python files, like if you have a lot of files, that could mean you're over organizing. If you have the right amount of files, it's perfect. And then you have one mono file. That's bad. So there are many things that I would be interested in testing out. That's super cool.

Matt Van Itallie  [01:11:14]:
Exactly. It's a fun it's a really fun dataset.

Michael Berk [01:11:17]:
Yeah. Cool. Well, I will summarize some of the high level insights that I got at least from this this episode. 1st and foremost, fear within organization leads to very bad decisions. Also, to increase gen AI adoption within an organization, specifically from a management perspective, you should look to share legal white papers and think about the ROI. The the returns often outweigh the investments. And, also, equating Gen AI to open source, I think, is a great way to put it into perspective. Would you use your hand if you have a hammer? Probably not.

Michael Berk [01:11:52]:
And then also when evaluating quote code, it's important to have both a qualitative and a quantitative lens, and quantitative can set the stage for its qualitative analysis. And then finally, some career advice. Get to the top of your field, if you can, because the analogy that I've heard used is if you climb to the top of a tree and you wanna jump to a different tree, well, you'll only fall as far as and then this is where the analogy falls apart, and I'm forgetting it. But the it still sort of sticks. If you're a squirrel and you wanna jump to the top of another tree, you should probably get to the top of your tree first instead of climbing that other tree. That said, changing fields and doing that tree hopping jump is is challenging. So, know that it could be stressful. And then finally, don't call your family business.

Michael Berk [01:12:37]:
So, Matt, if you wanna learn more about you

Matt Van Itallie  [01:12:39]:
call your business a family. That's why you're

Ben Wilson [01:12:43]:
Certainly don't call your family a business.

Matt Van Itallie  [01:12:45]:
Also both. Either one. They're both different. Yeah.

Michael Berk [01:12:49]:
That is a good point. Yes. Don't do either. Anyway, so Matt

Matt Van Itallie  [01:12:54]:
do either. They are different.

Michael Berk [01:12:56]:
Yeah. They are. So if people wanna learn more about you or CEMA, where should they go?

Matt Van Itallie  [01:13:01]:
Cemasoftware.com, semasoftware. Clicking those buttons of contact us definitely gets to me. We'd love to hear from your listeners and what an incredibly fun conversations, Ben, Michael. What a freaking treat. Thank you very much for having me.

Ben Wilson [01:13:16]:
Hey. It was our pleasure.

Michael Berk [01:13:17]:
Yeah. Thank you so much for joining. Alright. Until next time, it's been Michael Burke and my co host, Ben Wilson. And have a good day, everyone.

Ben Wilson [01:13:25]:
We'll catch you next time.