In this week's episode, Michael and Ben sit down with Artem Koren, Chief Product Officer at Sembly AI, to explore the future of AI integration in the workplace. We'll delve into Sembly AI's mission to accelerate team efficiency through powerful AI tools—imagine an Iron Man suit for your daily tasks. From proactive AI assisting with time-consuming tasks to ethical considerations in data privacy, this episode covers the cutting-edge developments and challenges in AI implementation.

They also discuss the evolving landscape of workplace automation, the intricacies of data collection, and the balance between privacy and productivity. They also highlight Sembly's latest advancements like Semblian 2.0, a breakthrough in digital twin technology that promises to redefine meeting productivity. Join them for an in-depth conversation on AI's transformative potential, the ethical responsibilities it entails, and the practical impacts on the project. 

Michael Berk [00:00:05]:
Welcome back to another episode of Adventures in Machine Learning. I'm one of your hosts, Michael Burke, and I do data engineering and machine learning at Databricks. I'm joined by my co host,

Ben Wilson [00:00:13]:
Ben Wilson. I spend my mornings figuring out why React won't build with node package manager.

Michael Berk [00:00:21]:
At Databricks?

Ben Wilson [00:00:22]:
At Databricks. Yeah. Cool.

Michael Berk [00:00:26]:
Today, we are speaking with Artem. He is currently the chief product officer at AssemblyAI, a tool that records, transcribes, and generates smart meeting summaries. And, it's really fast and cool. You guys should check it out. But more importantly, Artem was a guest in episode 110. So if you're curious to hear more about, the fundamentals of startups and the basics of the Assembly product, give it a listen. So Artem, how has the past year been? What have you been up to?

Artem Koren [00:00:50]:
Hi, guys. Great to be back. It's been a very busy year. It's, it's hard to think through all the things that happened in the year because of in, in our world every week is, is a lifetime. And so it's been exciting. There's been a lot of new things, happening in technology, in the market, in the AI industry as a whole. And so, we've been, both keeping up and setting the pace, at the same time, if, if that's possible.

Michael Berk [00:01:30]:
Yeah. I can imagine this industry is just becoming more and more saturated. What are the players that you're seeing or emerging? How do you guys sit? Are you comfortable with that market position?

Artem Koren [00:01:42]:
So when we talked, a year ago, this, idea of, meeting assistance and note taking automation, this was a pretty new idea still, and it was in the process of companies becoming comfortable with the idea. And, at the time, you know, something I would say often is that what we're seeing today are the baby versions of these companies, including our own, in terms of the value that they're offering to their customers. Because, at the time, large language models like GPT were still fairly new. And, companies were trying to capture that low hanging fruit as fast as possible. So there were some very obvious use cases. There were also low barriers use cases in terms of market entry and in terms of technical implementation. And so, a lot of products similar to ours, there's different flavors of kind of what a meeting assistant was, developed very similar features. And so we actually had a a, an abundance of new tools and products from all kinds of different areas.

Artem Koren [00:03:02]:
They were all kind of trying to do similar things. And so that was last year. And so we we went from a place where we were one of, like, very, very few companies, if any, that, we're working on, understanding the information in the meeting and and presenting something valuable after that, to being one of, like, dozens that was trying to do something like this. Because of the background that we had and and because of the many years working on it, we were one of the leaders in the pack. And, and that was great. And so in that area, we did we did very well. Some some companies took a little bit more of a, let's say, a populist angle. We took more of a deeper, kind of larger organization angle.

Artem Koren [00:03:51]:
But but generally, you know, we were we were a leader in that space and and still are. Because when you're when you're talking about doing this at scale, internationally, compliantly, with consent, with privacy, with security, you start to shave away a lot of these other smaller competitors. And you only are left with a few products that that can do what we do. But a lot has changed, in the past year. And we're very much evolving, as a product. And so if we were, kind of a baby in this space a year ago, we're now, like, getting up to toddler, and we're starting to show signs of very distinct differentiation from some of the other companies in the market. So that's kind of where we are today.

Michael Berk [00:04:47]:
How do you think about integrating yourself with plat like video calling platforms that already have a large customer base where they could just sort of create or even, like, buy a service like you? Are you looking to sell to those services? Are you looking to sell direct to consumer? What's the model when these large organizations already have a lot of your user base?

Artem Koren [00:05:11]:
So, we we ride on top of conference platforms, and one of our hallmarks is that we have a proprietary tech that emulates a meeting at Indeed, very effectively. And so we are, we're we're less of an add on into, like, a conference platform, and we're more like a virtual participant. There's so many ways of of kind of attacking the the meeting content problem, because that data is so valuable, and it can be used in so many different ways. And so you have companies that are trying to build something internal to try to do some things very narrow with that with that data set. You have companies that are trying to, like, API themselves into meetings and then do something with that. For for the very large companies, so like, okay, Microsoft Teams is making a lot of inroads in this direction. It's difficult for them to to bring very sharp, very deep value features into that platform. They can do basic things like they can do meeting recording, transcribe, and maybe ask basic questions of the meeting.

Artem Koren [00:06:25]:
But that's, but it's very difficult to step outside of that and start to take into considerations the the historical flow of meetings, the team structure, individual goals of people in those meetings. Now you're starting to get into the business side of things. And so for for Microsoft Teams, that have product that appeals to, like, over 200,000,000 users a month, you know, that's a difficult that's a difficult reach. So, you know, there we we've we've talked with companies about acquisitions, we didn't yet find an acquisition. We we haven't run into, let's say, an acquisition partner that we felt was a good fit for us. That might happen in the future. I don't discount anything. But right now, we're really focused just building a really fantastic product that that's doing something that's no one else is doing and bringing monstrous value to the teams that use it.

Ben Wilson [00:07:22]:
I'm really curious about the disruptor market when you're in, like, an established leader in an area that's using technology that everybody is really excited about or they think it's really simple to just, oh, I can just use Gen AI to solve this problem, and it it should be simple. Like, I can I can write write some scripts that kind of do this? How often do you see one of these small startup disruptors that are trying to edge into the space coming up with a novel idea that you're like, we should really think about doing that. And what would what's the process that you you follow for evaluating ideas that other people are are establishing?

Artem Koren [00:08:09]:
We see ideas that we like all the time. I don't shy away from, our goal is not to be the one that comes up with the idea. It just often happens that way. But if we see a great idea from a competitor, or an upstart, we will think about, you know, how would this make sense in our world, and if we love it, you know, we'll we'll roadmap it. For for the really kind of small, like niche startups that try to get in. I actually am, I'm a big fan of them doing that. Because the more players you have in and around this kind of technology, the more normalized it becomes. And so just by participating in the ecosystem and talking to people and getting some early customers, they're helping to spur the word of this new layer of products that has come into being which is the the AI powered layer of products.

Artem Koren [00:09:18]:
So that's, that's a good thing. Typically, it's very difficult for early stage companies to serve organizations. So what they end up doing is they end up serving individuals. So there's a great example. An old friend of mine, his name is Josh Moore. He was one of the Uber first fifty. He was the New York general manager for Uber, like, in the first couple of years that Uber started. So, he recently, you can say kind of came out of retirement in a way and decided to write his own app called some wave, I believe.

Artem Koren [00:09:59]:
And the app is eerily similar to in concept to what Assembly does. And he knows he knew about Assembly when we just kicked off. So I'll take some credit for that idea as well. And so the app is entirely iOS, and it records your phone calls and gives you a nice transcript of the phone call. That's it. That's all of us. Super simple. And it's been, he's in 4 months, he's been able to reach like, you know, a 1,000,000 AR.

Artem Koren [00:10:27]:
And, and he's he was he's not a coder, and he solo coded this for iOS. I think that's fantastic. It's not our target user base. Like, we don't do like personal phone calls. We don't do kind of, like, consumer stuff. Like, if you're a freelance professional, we we our product is great for you, but, but not really for individual calls and not on the mobile phone. Like, it's just an iOS interface. There's no other interface.

Artem Koren [00:10:54]:
So, they'll the the early stage companies, they'll tackle a part of the market that is not our target. And and and so we're not concerned about them kind of eating into our space. But I think it's great that they're adding to the information. And then when we do see a feature or or use case or an experience from a competitor, that we like, we definitely think about it. We also think we also often see new things from competitors that we don't agree with. And we love to see those too, because that means that tells us they're going in the direction we don't believe in. And it's kind of like, let's see, you know, with let's see who's dragging the stronger to, to bring back a term

Ben Wilson [00:11:39]:
here. Yeah. I mean, just the the zeitgeist that you get from a tool like that on the iOS store. I mean, I can imagine all those people that are using that that are also they're just like, man, I've got this app on my phone that does this thing that I love. Why don't I have this at work? I'm gonna go talk to some people and then people like, actually, we found this this service that does that for us. Yeah. And it just here's the monthly subscription cost. Let's just get that.

Ben Wilson [00:12:07]:
So, yeah, it seems like it's super beneficial.

Michael Berk [00:12:09]:
Yeah. It's a really interesting meta concept of people being resistant to sharing their data. And this is like a theoretically invasive concept where there's a thing listening to you. Right? And do you think that fully leaning into this publicizing all of your data is required? And where do you draw the line?

Artem Koren [00:12:35]:
All your data? Probably not. But I think the further along we go, the more value we will find in sharing more and more data, especially from your work activity. So first of all, we draw kind of a dotted line there. And I would say there's data around what you do in your personal life. And then there's data what you that you that's related to your work life. I understand in some cases, there may be sort of a blurred boundary. And that's, you know, that's a special case that needs to be considered individually. But in in those situations where that boundary is not is natural or organic, I think, those are very 2 very different datasets.

Artem Koren [00:13:21]:
And so personally, I would be sensitive to anyone capturing data from my personal life. That's it's for what I do. It's like, how I, you know, how I wanna spend my weekend. But from a work perspective, I think, we'll probably like, my my sense is that we're gonna go in a direction where most of content that's work related, interactions, chats, video, any artifacts or anything like that that you produce will all be kind of compiled and subject to data collection because there's so much power in using that content to create, great outcomes for the for the business.

Michael Berk [00:14:06]:
Ben, where do you draw the line?

Ben Wilson [00:14:09]:
I mean, I'm I'm as far from a Luddite as you can get when it comes to stuff like this. So anything that I do at work or anything that I do on my my work computer, I consider fair game. Like, I know that there's abilities for data collection, both internally at the company, like, legal could get access to this stuff. I'm fine with that. So in the back of my mind, I know, like, if there's something that I don't want, you know, a whole bunch of people reading that I have no idea that they're reading it or who they are, I'm just not gonna not gonna enter it onto my keyboard, or just certainly not discuss it in a in a meeting that's being recorded or that's not being recorded. So with that mindset, I welcome, you know, that invasiveness. Because in my current role, having an assistant that can help out with stuff that's discussed in meetings and making sure that the notes are updated and that there's a transcript that other people can read or a summarization of important things that were happening. I'm looking forward to the day where I can discuss stuff in a meeting and then have some advanced AI agent that's that's understanding what my calendar is, what my Jira board looks like, what my quarter, you know, road map planning is, what my capacity is for the team, what current status of things, and to have it start generating, you know, probabilities associated with prioritization of things that need to get done at certain points.

Ben Wilson [00:15:55]:
I think the more integration and the and with that data in advanced tools that can do that, the better. It saves us from doing all of the annoying context switching that we have to do day to day in a in a workplace environment.

Michael Berk [00:16:11]:
Got it. Yep. That makes a ton of sense. Alright. So question for Artem on the more, like, technical side. What are the technologies that you guys are leveraging that allow innovation? Where are you guys placing your bets? What are the types of models if you can speak to that? What are the types of software stacks? Those types of things.

Artem Koren [00:16:37]:
Agents. So we are, in the past year, that's been our big bet. And, actually, so the previous question very naturally brings us to to to this one, which is, I I look I I look at, kind of the the modern AI stack where, you know, the LLM is at the at the bottom of the stack from where I'm looking. Right? I I understand there's a lot more in there. And and I call that the substrate, which is it's kind of the the raw material. And and then there's the method, which is separate from the substrate. And the method is how you use the substrate. And, you know, one way to use it is to, like, ask a factual question or just ask a question like you do with Jigpt, and you'll get back an answer.

Artem Koren [00:17:36]:
That's one method. Right? Like qna. But then, there are much more advanced methods of using the substrate, And that's where agents commit. I kinda have my own perspective on what I consider an agent. So so to me, an an agent is some is is a technology that can can continue to interact with 1 or more substrates, as well as other content sources and other agents to get to a result that it's looking for. And so that's what we've we've been working on for the better part of the last year. During this time, we drop an automations component into our product that's very robust. And and what we and we can now, for example, so Ben mentioned, like, it'd be cool if you can pull Jira on calendar.

Artem Koren [00:18:43]:
So our automations allow us to connect natively to dozens of endpoint applications, task management, CRM, knowledge management, many, many, including Jira, and, send send information to those applications. That's what we do today, and we do that well. Like, we can send transcript notes, tasks, etcetera, and we do that well. But, we're actually positioning for something for something much bigger. And, in just a few weeks, and we already have a web page about this, we're coming out with what's called Semblion 2.0. And Semblion 2.0 is is the first time the world will meet what our agentic technology can do. And so we we use the agentic methods. Some are simpler, some are more complex.

Artem Koren [00:19:41]:
So like, for example, you know, on a simpler side, like an agentic method could be it generates some result. It thinks about, you know, what's good and bad about this result. And then it improves the result as a result of its own assessment. And that assessment can be done by, like, similar LMS or different LMS or or a custom trained LMS. So that assessment piece can be varied a lot. But that's, you know, that's typically called reflection. Right? And so you can use reflection to get to to a level of quality of answer. But then we use a lot of other kinds of methods that are sort of use user experience specific, to get to other kinds of results.

Artem Koren [00:20:24]:
So what what Sembly and something we talked about with you guys on the on the on the previous podcast is this idea that you're kind of by coming up by showing up in your meetings where we have this ability to digital twin you. Because essentially, we know when you're speaking, and we know what you're saying. And, and we can we understand so so much people are very surprised how much we can figure out about you. But when you think about it, when you're talking to chat GPG, GPT, you can ask it like, you know, with 2 sentences, and it gives you like really relevant stuff. But with assembly, you're talking for hours and all different kinds of context. So we know a lot about you. So we're basically now have a product that understands you very well understands you deeply and understands your work deeply. And so one of the features of Semly and 2 point o is that after a meeting, it figures out who you are, who the participants are, what, like, what companies, like, what business, what's the goal.

Artem Koren [00:21:30]:
And so it understands kind of your vector coming out of a meeting, your individual vector, meaning like, what, what do you want to take out of that meeting? What are you responsible for out of that meeting, and it will give you high impact suggestions on the most valuable things to do as a result of the meeting. So for instance, like, let's say you're a PM, like a product manager. And you drop in your just came out of a meeting with your team. It may say, okay. Like, based on this discussion, it looks like there's a need for, like, a feature requirements document. And it and nobody could have even said the word feature requirements or feature requirements document the meeting. But because it's able to figure out deeply contextually what this meeting was about related back to prior meetings, it's able to give you this really powerful response. And the way we do that is we use agent, workflow behind that.

Michael Berk [00:22:30]:
Super cool. Ben, I think you requested this either on the prior episode or another episode. It's really ringing a bell, But that's super cool.

Ben Wilson [00:22:40]:
Yeah. I mean, I think it's a good first stage for a tool that I'm looking for, like, selfishly in my day to day is exactly what you just explained, where it gives us, like, that summary is almost an application that I can go into. And it's like, hey. It looks like we need a product requirement stock to be written based on this this conversation. Would you like me to generate one for you that you can then review? And I'll assist you with, like, a chat interface on you highlight some text, you write a sentence about what you want changed, it it modifies that so that it can conceptually get, you know, that very valuable work item that needs to get done that takes a lot of time and that I don't have to generate this stuff to provide context to people that need to do the next phase of something. And if those people are not in that meeting, providing that summary of what was discussed as well as a formalized document and the the set of tasks that are the next actions. So, like, okay. We know that person a is gonna be the one doing the engineering design document based on this product requirement.

Ben Wilson [00:24:01]:
I'm gonna create a Jira task for them. I'm gonna look at the complexity of the PRD, and I'm gonna estimate the number of story points that are required in Jira to do that engineering design based on their historic performance of completions of those. If it can go and fetch their previous designs that they've done to and then what the Jira was that was associated with that to say, okay. It was a half point, and they got this done. They've done this 30 times. They're probably pretty good at this. So maybe this is 0.25 points or 0.75 depending on the complexity. And then to have that in that summary to say, I can click a button that's like, yep.

Ben Wilson [00:24:42]:
Create Jira. Like that, we're talking about saving person years of time in any company that this like, a system like this is in. It's like a no brainer. I I would not see if it works really, really well, I don't see a company that wouldn't jump all over this. Anybody who's producing software is gonna definitely be on board with this.

Artem Koren [00:25:10]:
So, Ben, you described the second major feature in Assembly and 2 point o. So next to those insights, it says, so it gives the insight, like, based on, you know, like, what it's gonna say something like there's something in this discussion that suggests that PRD needs to be created. Then it says next steps, like, do this, this, and this. And then it says the deliverable project requirements document or something like that. And then there's a little button next to it that says work on this. And you click the button, it opens up a chat GPT ish interface.

Ben Wilson [00:25:51]:
Mhmm.

Artem Koren [00:25:53]:
And the insight already appears as in your chat. It's associated intelligently with content that you have access to from all your prior meetings. Not yet associated with all the integration stuff we have access to. So not yet conversational with Jira, but we already have that connect. So we'll make it happen. And then you could tell you can either ask it a question like, what's a project requirements document? Right? For example. Or you can say, yeah. This deliverable sounds great.

Artem Koren [00:26:22]:
Go ahead and make it in English or whatever, or just make it. And, if it thinks you're asking for, an artifact, it will generate the artifact. And it will generate it contextually hyper specific to all of those details that we have about you, not just from that one meeting, but from all the meetings that could possibly be related to this. And that's also part of an agentic workflow that happens. And it generates the document with all the details, that are available to it from all the conversations, you've had. So it's a full on document that you can download in Word and PDF. And, In chat, you can say, actually, can you add, like, a summary section, like, towards the end that talks about something? And it will add that section to that document in chat. And then you can download the new version.

Artem Koren [00:27:24]:
And then you download it, and then you mess with it however you want. And then you send it to your team or attach it to your next meeting invite. Do whatever you want. And, you can and it's really good at anything that's, like, information worker type document, like anything that would be good in a word or a PDF. So we don't yet and we don't have any near term plans to do things like spreadsheets. Not really images. Like, I'll think about it. So far, not really.

Artem Koren [00:27:54]:
But anything that's like a a spec or a a project plan or a contract agreement or a compliance document or or or I mean, like, you know, there's so many possibilities. It's really, really good at.

Ben Wilson [00:28:10]:
I'm wondering what the cold start is. So if I sign up today, right, assuming or whatever 2 point o releases, new users to the platform haven't had the agents on meetings that doesn't really understand me. How long before it transitions from cold start behavior to being uncanny valley of, like, I think this is an actual human sort of thing, but there's some things that are not it doesn't quite get to all the way being mature enough to be as trusted as, like, an actual human assistant that's very good at all of those tasks?

Artem Koren [00:28:52]:
I don't think it's a 0 or 1 type situation. The the more it hangs out with you, the smarter it gets. So there's another, there's another piece to assembly in called work streams. It's actually it looks like we're actually going to delay the release of that piece. But it's actually fascinating. So what work streams us is it it takes the entire flow of your meetings and distills them into, like, context to your related things. So, like, essentially, if you're working on, like, a few projects, it will figure out that those are the projects you're working on automatically. Or if you're working with like, talking to 10 customers, it'll figure out, like, these are your customers.

Artem Koren [00:29:38]:
And it will relay information across meetings to those work streams. And so that feature takes about 5 days to to come online once you start using Assembly. So I don't know if that gives you an idea of maybe, like, the minimal kind of time. But that's what we found is, like, you need about 5 days of regular data daily meeting stuff for this to start to be interesting. And then it continues to build over time.

Ben Wilson [00:30:11]:
That's actually really that's a lot shorter than I thought it was gonna be, which is pretty cool. Yeah. So how much do you think this is going to be, you know, proliferated throughout industry? Aside from, you know, enterprise focused, you know, basically job helper, which is basically what this is, like, the ultimate job helper. Do you see stuff like this being within the next, say, 2 to 3 years, more present in just general human life. Like, the principles that your your team has worked on about leveraging, you know, agentic workflows and the power that these things provided that you're doing it in a controlled, safe, sophisticated manner of, like, hey. I want you to do this thing, and here's how to do this thing, and here's some controls around it that that you create. Do you see stuff like this being, you know, more readily available just in general everyday life?

Artem Koren [00:31:15]:
I would I'll say it like this. I think the barrier will be, compliance.

Ben Wilson [00:31:23]:
And I

Artem Koren [00:31:23]:
think that will be ultimately the only barrier. Because once you touch this and you understand what it can do for you, you wouldn't wanna live without it. Because it's like what you said. It's like who loves writing from scratch, like specification documents or contract agreements. Like, who loves to do that from scratch? Like, nobody. You wanna meet you wanna make it good. You wanna use your expertise. You wanna, like, you know, make it like, infuse it with your wherewithal and adjust for nuances.

Artem Koren [00:31:58]:
But you don't wanna be sitting there, like, okay, like, page 1, like, nobody wants to do that, for a 10, 20, whatever page document. If you told me like, okay, I'm gonna give you an analyst who's gonna work with you. And whenever you have a document, like the analyst will be or an associate, let's say, an associate, right, a little bit more senior. The associate will be smart enough that they can produce, like, a decent draft, like, that's actually very complete. Like, it's not final. Like, you still you're you're you have more expertise. You're much more in tune with exactly how things need to be put together, what to look out for. So this associate is gonna give you, like, the 80%, 90% complete, artifact.

Artem Koren [00:32:39]:
And then you just go in and, like, you're kind of put in your little flavor on it, but then you're done. Who would say no to that? Like, you know, I spent time working yeah. I spent time working in management consulting. I was an associate. I was a VP. I did why I was a senior manager for a time. That's like, that work is, like, very like, that's the hard work. Like, that's the work you need to, like, grind.

Artem Koren [00:33:06]:
Mhmm. Why? Why grind if you don't have to?

Ben Wilson [00:33:11]:
I can imagine this revolutionizing the project management space for consulting quite a bit. Are you like, hey. We sat on this 3 hour call that we talked about all these requirements. And at the end, we agreed on a final list of things that we will do, and here's a bunch of stuff we won't do. And creating an actual contract document out of that that breaks out work streams and estimates, you know, person week associated with task. Yeah. That that would save you a bunch of time. Right, Michael?

Michael Berk [00:33:51]:
It would. It most definitely would. Oh, it's a no brainer in so many different facets. I had a question for you, Artem, though. Well, like, 70, but I guess this one I'll start with. You said compliance will be the primary slash only barrier. What are the risks with this tech?

Artem Koren [00:34:15]:
Many. Really many. It's a it's, it's not something that, should be taken lightly for companies that that for whom controlling internal information and communication is a critical piece of their business. Listen. If you're running, like, a lemonade store, 0 risks. You you should be using it yesterday. But if you are Ernst and Young or, you know, there's there's things you need to think about. If you're a weapons manufacturer or, you know, there's many situations where it's very, very sensitive.

Artem Koren [00:34:56]:
And so, the the proliferation of information, could be sensitive. Using these artifacts as is without a human review, let's say, in the legal space, that opens up some risks because these things are good, but they're not human yet. So they need a human to to go in and, you know, you know, I had a great example where, we're we're in a process of getting our HIPAA certification and we have to BAA our subprocessors. BAA is a document that talks about how your subprocessors can treat, like health related personal information and things like that. And, when, we, you know, I use Semblion to help me generate the BAA, and like, for whatever reason that decided to indemnify the sub processor. And that's that's that you can, like, you need to be human to be like, why are we indemnifying the sub processor in this situation? Like, who wins here? Right? So I think those are those are risks is proliferation. It's like legal legal risk. You need a you you still need like, you need good controls over, how your data is managed, and you need also, good controls to make sure that you don't just take whatever AI spits out and and make it into something that will hold someone liable or, you know, legally liable for something.

Ben Wilson [00:36:43]:
I found that even for, like, somewhat trivial use cases where, you know, got a bunch of like, Michael, we were talking before, before this meeting together about multi turn agents and the some of the tests that we've done with those. It when you apply it to something like, oh, I wanna write, like, a demo, or I wanna do something for documentation for this project. And I don't want to sit and write, you know, 4 pages of text about this thing. And with multi turn agents, you get it so that it's like, okay, I'm actually storing the source code in a vector the exact code that it should be referencing and also a copy of our existing documentation that can search for for items in there. And then we have a whole bunch of agents that are going and and doing different roles, including one agent that's capable of just executing, you know, generated code and validating that it's producing what it's supposed to be producing. And then a code reviewer. And having it generate documentation, it removes my own personal bias from the editorial process, which I find incredibly powerful. So I'm getting an an additional reviewer for free in myself because I'm not the one producing it.

Ben Wilson [00:38:12]:
When I read through something that I've written, I'm kinda glossing over it. I miss a lot of things. I think most people do when unless you you do the whole, like, I'm gonna write it on Monday, and then I'm gonna review it on Wednesday after I've dumped this from short term memory. And then I'm gonna look and be like, why did I write that? Like, that's not right, or that's really confusing. Okay. Let me edit it again. But having, you know, Gen AI generate something for me, it's like somebody else on the team wrote it. And I'm basically doing a review of that and pointing out things that are, like, I would not do or say this, or this is highly confusing.

Ben Wilson [00:38:48]:
And I can make edits to that so that my first actual draft that goes out for peer review is so much better than if I had written it myself. But yeah. If you just trust what the what these things generate, no matter how sophisticated, like, multi turn agents are, if that nuance that's in there, you're like, well, that's not really a public API, or we don't really want that to be used. Even though that does work, there's a different way to do it. You know, if you just trust it and ship it, yeah, you can get into a a position where you're like, alright. We just effectively shipped the regression, or we we did something that is gonna break the user experience. So, yeah, I'm a believer of using them, but I just wanted to point out that that sort of psychological effect is pretty cool in my opinion.

Michael Berk [00:39:44]:
Yeah. And another question for you, Artem. You guys are sitting on a, like, a gold mine of data as you alluded to. You're starting to leverage that gold mine to build context and then sort of a digital twin about a user, then hopefully leverage business context, Jira context, all this other context to then give tasks and actual deliverables. What's the, like, 5 year vision with this data? Like, there's a lot that can be done with it.

Artem Koren [00:40:15]:
Our our goal is pretty has remained steady since founding, which is, we want to help, to help teams get to their results faster, better, cheaper, get better results. You know, our our motto is, flawless execution. And I think that's our North Star, and that won't change. So, we're we're going to develop continue to develop things that let teams accomplish things in the most, like, powerful way enabled by AI. You know, I was talking to, a customer this week, and we were demoing assembly assembly and 2 o. And they were trying to come up with, like, an analogy of what it is. And I'm like, well, it's not like it's not like a like a, it's not like a separate teammate. But it's more like, anyone who's enabled with Semblion 2 kind of has the Ironman suit on.

Artem Koren [00:41:21]:
It's kind of like that. And I think, you know, this is a good way. And we call it also like the whole kind of spiel we call augmented worker intelligence, like not just like separate pure AI, but like, we take the worker, and we empower them with like superpowers based on AI. And that perspective is important. And so 5 years, I so 5 years, you know, we're now at a point where we can start to sort of, like, leapfrog you across the time consuming, parts of your day. And, you know, next year, we'll be able to do that even better and broader, because we'll start to potentially add all the connected apps into that process. But then you go out 3, 4 years, what I think will happen is a lot more proactivity. The better we know you, the better we understand what you're trying to do, the more we can help you and not wait for you to tap us to help you.

Artem Koren [00:42:24]:
So, it becomes like, you know, if it's like a Iron Man suit today, maybe like, you know, in several years, it's a lot more like a, you know, like a Pikachu, where where it's sort of like, it knows, like, what you're gonna what you're trying to drive to. And it's like, okay, like, you know, I already I already lined these artifacts up for you. And do you want me to, like, publish them like that? Right? Like, so it's gonna become more more proactive. It's gonna and and it it's all it's in a in an interesting in in a way, there's a lot of new things, but in a way, it's very aligned, which is the deeper it knows you, the deeper it understands the work you're doing, the more it's able to help you with more kinds of things. So more accurate results, more different kinds of results, but all towards getting you, helping you to accomplish what it is that you're accomplishing. I can I'll add something else to this. Another, kind of angle that we've been thinking about, it's not for near term, but it is something that might come into play over the next few years is. There's also a question of alignment, but not alignment from an like in the AI world alignment, but alignment in the business world alignment.

Artem Koren [00:43:44]:
So as we get more and more proactive and as as we as our technology becomes more and more influential in the results that you're working on so consider we're already giving you, like, most impactful suggestions, but we can actually align those suggestions to strategic directions. And so we can align the re like, the suggested things that we recommend, but we can also then align the work that this Pikachu does to the the strategy of the company. And so this is a way to kind of create a very responsive nervous system. And and I I hate the term because it's so overused, but agility in larger organizations where, you know, today, like, something bites like a large company, like, you know, a year later, it might respond. But with this kind of system, something bites you, like, the next week, all the impactful things are aligned to address the bite. And that, I think, is also a very powerful idea.

Michael Berk [00:44:48]:
Cool. That's super cool. I

Ben Wilson [00:44:51]:
do have one thing that I'm thinking about with, if it's possible or if it already does this in the 2 final release as a sort of pre send feature where, like, hey. I'm I'm writing a draft of an email that's gonna go to a VP It's talking about a proposal that we have or some issue that's happening. And if everybody at the company is using these tools, is using AssemblyAI, And there's nothing stopping the system that's working for you or your agents from understanding the history of what happened with similar interfaces with this this person that needs to approve this and then stopping you from making a proposal that could rub them the wrong way or focus on the wrong things. Could you get a a system or is it capable of saying, Ben, you you don't really wanna phrase it this way. Here's something that's like, let me rewrite this for you with some suggestions, and you'll have a higher probability of, you know, getting this message across more effectively to this person. They like information presented in this way, and they have less less issues with things that are presented in this way. Like, maybe this VP likes just the TLDR. They don't want any detailed information.

Ben Wilson [00:46:21]:
They want, you know, here's the actions that I want to get approval to do and one sentence explanation for why. And then another VP hit the company. He's like, I want to know all the nitty gritty details I expect, you know, full explanation of everything, or maybe sensitive ways of of actually interacting with them. Like, some people might might want to hear the bad news first or the problem statement first and then a solution right away. Other ones wanna hear, like, a discussion effectively. Is there

Michael Berk [00:46:57]:
like,

Ben Wilson [00:46:57]:
question 1 is, does that exist? And question 2 would be, is this something that you would shy away from if it doesn't exist being like, I don't know how intrusive something like that would be?

Artem Koren [00:47:12]:
Fascinating idea. Does not exist. The intrusiveness is important. There's a lot of access concerns when you can start to use information for from someone else's activity to empower your activity. So, for example, in Semblion 2 0, one of the big challenges was, creating a layer that's semi permeable where you can chat with the work stream that you're participating on and other people on your team or even other teams might be participating on, but you wouldn't have access to any content that didn't occur on a meeting that you have access to or attended. Right. And that that's that's a little tough to do. With something like this, it would you the it it would have to be in, like, a very nuanced way.

Artem Koren [00:48:18]:
Like, what information about another person are other people able to infer? Like, can they can they use the preferred communication style of this other person in in in whatever they're trying to do? So I think there's a there's an information sharing and and potentially a privacy, thing there that that would probably be, like, the nut to crack to, to enable something like this.

Ben Wilson [00:48:50]:
You're responding with the ethical answer here, which is nice to hear, about like, hey. This is probably not good to do something. Do you think somebody's gonna build it and then not tell customers that this is actually why this thing is so good? And what do you think the impact on the industry's application of generative AI technologies would be if that got found out. Like, hey. This like, everybody's using this thing because it's so awesome. And then it turns out that somebody leaks the fact that, like, oh, yeah. Like, these agents are talking to each other, and it knows everything about you. And it's actually, you know, almost this sociopathic interaction that's happening to get your way because it understands this other person.

Artem Koren [00:49:34]:
Aaron Scary ideas. Yeah. I would hope that there's some level of checks and balances and scrutiny in place to prevent product like that from getting into a whole organization. Because you would need to be, like, networked across the organization to do something like this. There are plenty of products, by the way. I don't know if you guys know. Yes. I'll, like, break it here.

Artem Koren [00:49:59]:
That have zero concern about, consent privacy law Yep. At all. And they just record everything what they want. And they're breaking laws every day Mhmm. Because California laws are different than Iowa laws, different than New York laws. And they they can blindly record, like, from a browser window, and nobody would would know. We we don't do that very intentionally. We don't wanna be a sneaky recorder, but there's products out there that do.

Artem Koren [00:50:29]:
So that's already kind of happening in the sense. I think if that were to happen, I mean, that becomes a legal matter, really. That there's probably some law some privacy laws being broken if if that occurs.

Ben Wilson [00:50:44]:
Do we protect through privacy laws our, like, worst self as humans. I think a lot of those privacy laws are generated to protect people's financial identity and stuff. But this is the first time in history where we have the ability to interact with tools and build systems around them that can actually do, you know, some really interesting things with with not so much the human mind, but the human emotions. Where you can be manipulated by another person that but in a way that you would never detect with something like this. Do you think all of the like, we're we're co opting regulations about privacy that are meant to protect people's bank accounts into an area that we don't really understand quite yet, like, how powerful it could be?

Artem Koren [00:51:43]:
100%. I mean, we started with the social networks. Right? Like, they're learning a lot about our behavior, and there was that big scandal in the 2016 election.

Ben Wilson [00:51:53]:
Mhmm.

Artem Koren [00:51:54]:
I forget what the company out of. I think they were out of Washington, DC, what the company was called, but they were, I think, the first

Ben Wilson [00:52:00]:
Chamber to

Artem Koren [00:52:01]:
kinda use. Yes. Right? So they they they're like the the forefathers of this type of, stuff. It's very effective. And I don't I I I I'm I don't think we have a solid legal framework for this today. And you're I think you're right. We're co opting, privacy laws that were written for somewhat different purposes. And I think it's very dangerous.

Artem Koren [00:52:30]:
I'll add that as well. I think we do need, new codecs because, for example, you know, there was the recent arrest of Pavel Durov in France, and they were using, like, some legal thing, to arrest him, but he's a founder of a 1,000,000,000 user company. And he was arrested for potentially being liable for something some user did on this platform. These are also laws that are not correct.

Ben Wilson [00:52:57]:
Right. It's kinda chilly. We got all this, like, fun conversation about, like, all these cool things, which I don't wanna detract from that because I think how you're approaching this and sort of air gapping that is the model that other companies should use if they wanna stay out of international courts. Because even if the laws don't exist today, give it a couple years. I think it's gonna be a knee jerk reaction to somebody doing something super nefarious. And I think most of the big tech companies don't wanna be dragged into that, but some start up is probably gonna do it. And the reaction the reaction to what they're gonna do, whether it's intentional or completely unintentional, they're just like, oh, this would be such a good idea if we could, you know, just make all of this data just readily available between these. Look how how successful it is.

Ben Wilson [00:53:54]:
Everybody loves this thing until somebody says, hang on. We have a subpoena. We need to see how you're doing what you're doing. And an expert comes in and just like, wait a minute. This is morally corrupt. And then Congress gets involved. So, yeah, I think it's something like that is bound to happen. Those are safeguards that you you put on.

Ben Wilson [00:54:24]:
Yeah. It was a PR I was working on earlier this week, actually. That was about, OpenAI's new API response for basically guardrails detection. And now it directly informs you through a different interface that, hey, you just asked a question that I'm not allowed to answer. And I can't give you any information about that. It used to be that was just the response that came back in the the REST API. And now it's it's explicitly in there with a different name, and there's no contextual information on there. In order to make it easier for IT departments to say, hey.

Ben Wilson [00:54:58]:
Who's using these APIs, and what are they asking? And why are they asking this? Let's go have a chat with them and see if they still need to be collecting a paycheck here. So it seems like things are moving in that way for that base substrate layer that you're talking about. But it's not that hard to defeat that stuff. There's no way that's foolproof. And it's more like what you're building on top of it that doesn't have that regulation or that control unless you intentionally build that in and design that.

Michael Berk [00:55:33]:
Yeah. A a quick question to that front. Just really quick. Are you guys fine tuning models or is all the this sort of data segmentation coming from context windows or vectors via, like, a rag database? Because if you're fine tuning, I feel like segmenting this could be kinda hard.

Artem Koren [00:55:56]:
We we did a lot of r and d with, different kinds of models, fine tune raw. And what we've found so far, this might change. But so far, we get the best results out of the most powerful foundational models without fine tuning. And what's really interesting, and I was surprised by this, but in some of our fine tuning experiments, we're starting to get worse results than we would do with the raw, like, powerful foundation model. So I think fine tuning is probably useful, but in certain specific kinds of use cases. And I think, you know, our world is a world of natural language and, like, normal, very broad topic conversation. It's and and and the kinds of results we bring to bear are also very dynamic, let's say. And I think that's you know, we haven't found a reason to fine tune a model to give us something better than what we have today.

Artem Koren [00:57:07]:
I can see, like, little things like, you know, task detection or something like that might be fine tunable. But even there, like, super cheap models do a great job now doing that stuff. So why why bother investing in the fine tune model?

Michael Berk [00:57:20]:
Okay. Cool. I know we're at time, so I'll quickly summarize. Lots of very meta topics. So it was sort of hard to pull out some specific tips or tricks. But some things that stood out to me is when innovating in a socially taboo field, the more companies in the space, the faster that fields gets normalized and accepted. So, typically, there's sort of an inflection point where it's not taboo anymore. On the assembly side of things, their base is sort of building a bit a digital twin via meeting recordings.

Michael Berk [00:57:53]:
And with that information, there's a lot that can be done with it. So in the 2 point o released, they're gonna be able to suggest subsequent activities, so effectively Jira tickets. And you can also chat with Assembly AgenTic framework to build artifacts. And this is just gonna get better and better over time. And then finally, keep an eye on the government for what they have to say about data privacy. So, Artem, if people wanna learn more about you, your work, or Assembly, where should they go?

Artem Koren [00:58:22]:
Check out our website. That's www.assembly.ai. That's sembly.ai. And you can also already check out, at the top, there's a a page about Semblion 2.0, which has all the new goodies, that are really fantastic. And, if you wanna find me, you can, find me on LinkedIn. Just Artem Koren on LinkedIn, CPO at AssemblyAI. And, you know, if you have any questions and wanna reach out, I'm happy to.

Michael Berk [00:58:53]:
Cool. Well, this was a lot of fun. Until next next time, it's been Michael Burke and my cohost, Ben Wilson. And a good day, everyone.

Ben Wilson [00:59:01]:
We'll catch you next time.