The messy truth of your AI strategies

The messy truth of your AI strategies - Stack Overflow

Stack Overflow Business Stack Internal: the knowledge intelligence layer that powers enterprise AI.Stack Data Licensing: decades of verified, technical knowledge to boost AI performance and trust.Stack Ads: engage developers where it matters — in their daily workflow.They discuss governance approaches like deploying models inside approved platforms and routing calls through monitored gateways, and how broken pipelines from complex feature-engineering motivated Kumo.ai’s approach of using a single foundation model with on-the-fly database queries.Kumo.ai allows you to train and run state-of-the-art AI models on your relational data, allowing you to make predictions about your users and transactions in seconds.Connect with Hema on LinkedIn or reach out to her at her email hema@kumo.ai.Congrats to user BalusC for winning a Populist badge on their answer to How to sanitize HTML code to prevent XSS attacks in Java or JSP?.TRANSCRIPT[Intro Music]Ryan Donovan: Wavemaker Fuses design and development with an architecture-first, agentic app generation system for design-led enterprise dev teams that want deterministic outcomes from AI. Predictable costs, open standards, pixel-to-pixel matched full-stack applications deployable at scale. Visit Wavemaker AI or email demo@wavemaker.ai.Ryan Donovan: Hello, and welcome to the Stack Overflow Podcast, a place to talk all things software and technology. I'm your host, Ryan Donovan, and today we're talking about all the messy stuff that comes when you try to implement AI in a company that's trying to make money. Pipeline sprawl, shadow AI, all the best stuff that keeps you from getting the most out of it. And my guest for this episode is Hema Raghavan, who's co-founder and head of engineering at Kumo.ai. Welcome to the show, Hema.Hema Raghavan: Thank you, Ryan. Thank you for having me here. Excited to talk about this! [I'm] deeply passionate about this topic.Ryan Donovan: We love to hear it. Before we get into that, we like to get to know our guests a little bit. Can you tell us how you got into software and engineering?Hema Raghavan: Absolutely. I started my career in software over two decades ago, maybe three decades ago. And I was in India, and like many students in India, computer science was a blooming field. I got into it, and then back in the day, my first job was at a bank, and I had to write code for bill processing for trade, and it was – I assumed that the job of the person who was using my code was really boring. And what I wanna explain is, they were scanning bills of trade, and then they were just human transcribing that into records. And I was like, 'gosh, this has to be automatable,' and that's what took me to explore. The field was called information extraction now, and it's all AI these days, but I was building language models in 2000 to solve problems like this before they were the 'hot kid on the block,' and before we had GPUs, and when we had very small memory. But the journey started in the early 2000s, and I've been in the field working on many problems ever since.Ryan Donovan: Okay. Like you said, a lot of the automation language models are all in the generative AI space. Everybody's super excited about getting into it, which can cause some problems, right? [I've] been hearing a lot about shadow AI lately. Can you talk a little bit about that and what [are] the issues around it?Hema Raghavan: Absolutely. So, I think companies, CXOs across the board have mandate to go AI-first, from their boards, from investors, and so on. And therefore, AI budgets have flourished in the last few years, it means that many vendors are onboarded, but it also means that across function, whether it's engineering, whether it's marketing, whether it's sales, they're all incentivized to use AI. But it's right now beyond the IT team's control. And if I go to any CSO or CIO summit, now they're really worried about private data or company-sensitive data egressing out into some of these services, because it could be very simple. It could be just somebody trying to clean up a sales deck. But you've sent all of that information as a prompt over to an LLM service provider who is not approved. It could be that you are giving access to your CRM tools, or some of the others to AI tools, and it's outside. The company does not have the governance to understand what's going in and outside that perimeter, and I think that's starting to worry the CIOs quite a bit.Ryan Donovan: Yeah. Your company data is going through a lot more people in your supply chain, right? With AI.Hema Raghavan: Yes.Ryan Donovan: And we have things like Open Claw, which are super exciting, but big risk for organizations.Hema Raghavan: Absolutely.Ryan Donovan: What can your CISOs and others do to get a sort of hold on their information security with the AI space?Hema Raghavan: Absolutely. So, I see a few different models being implemented. I think one thing is a lot of the AI providers, Kumo included, but even the LLM providers are available as first-party and some of the platforms. So, an example would be Snowflake has this deployment modality called Snowpark Container Services. You can actually deploy inside the Snowflake parameter. So, we in our company deploy our AI models to customers through that. So, it never leaves the database or the data warehouse that has been approved by their security teams. So, that's one pattern I see. I see another pattern in some of my customers, that they'll have in VPC deployments of models, or they may implement a gateway, so they'll actually look– all calls go through a single gateway, and then they're actually monitoring on the gateway what's going in and out. And of course, that's for someone who has the resources to implement a gateway of their own. But I'm seeing this emerge amongst my customers, and even the asks of me or my company as to where I would sit inside their VPC, if it's outside, how does it work, and so on. So, I'm stuck. So, that's the pattern of solutions.Ryan Donovan: Yeah, I know gateway providers are definitely looking at the AI aspect of it and even managing some of that gateway traffic using AI. Do you have thoughts on that? Is it better to have this be the human process? Or can AI or automated processes manage the sort of inflows and outflows of a gateway?Hema Raghavan: Oh, absolutely. So, I think AI itself can solve that. As long as you have data going out through some controlled endpoints, and then the AI should be able to detect PII or company-sensitive data pretty well. It, it seems like a fairly identifiable pattern. And so, I think that's definitely one way to go about it. But the other one is also, when I look at the warehouses, it's what data do you give access to AI in the first place? When I work with customers in the FinTech and healthcare space, they really like deployment modes for AI where they can actually control what sensitive data the AI has access to. So, in IT team, actually, there is some method to the madness. So, it says, 'okay, you have electronic health record data. Okay. It's these individuals and these AI systems that actually have access to it.' And we actually have telemetry to see what is going in and out.Ryan Donovan: It's interesting, talking about that and putting it within the database provider. A lot of organizations have multiple database providers, multiple uses for these databases, and then have all these pipelines, ETL pipelines, or whatever, to transform it to get it useful for the application. Is there a use case for AI that avoids pipelines altogether? Or is there a better pipeline that people can build?Hema Raghavan: Ryan, you touched a topic very close to my heart, because before I started the current startup, I was leading AI at LinkedIn. And we had models for everything. People You May Know was a model built out of my team. All of the notifications, the feed, the job recommendation. So, you're talking about several dozens of models and several hundreds of pipelines flowing into these models. And I wanna give you an example of, even three Gen AI of what pipeline sprawl can do for you. Okay? So, just think of an app, just like LinkedIn, and there's data from a user's click behavior flowing back into these models and the models of training, right? We had one example where one of the pipelines, a front-end tracking broke, and the model started behaving really weird. Fortunately, we had the governance to actually detect that the model scores seemed to be going off, and we opened a war room. But to actually trace back, because it's pipeline A flowing into B, into C into D, and when the first upstream pipeline is the one that's broken, that lineage just was a nightmare to debug. And imagine that now for dozens and dozens of models, and hundreds of pipelines, and a data science team, and a warehousing team that spans thousands of engineers, and people leave the company. People stop maintaining those pipelines. There's bit rot. It was a lot to maintain. So, that's actually what inspired us to create AI that's much more simple and actually eliminates pipelines. So, where are pipelines in AI often coming from, and especially for predictive AI? Think recommender systems, think lead scoring, think fraud risk prediction, and all of those kind of models, they're often coming from feature engineering. Because you have a data scientist who's saying, you know what? I need to aggregate the last 30 days of clicks. And you have one pipeline that collects that aggregation and ETLs it, and then three people rely on that ETL'ed pipeline. But that's where all the problem start, as well. And when we created Kumo, we wanted to create a really simple model architecture. So, we said, ' can we have one foundation model?' Can you imagine that a company, just for all those use cases that I described, you just have one foundation model that you need to maintain?Ryan Donovan: Sounds like madness.Hema Raghavan: Yes, but very elegant, right? You just