The PRQL: What’s Driving The Evolution of Data Operations? Featuring Kunal Agarwal of Unravel Data

Eric Dodds 00:05
Welcome to The Data Stack Show prequel. This is a short bonus episode where we preview the upcoming show. You’ll get to meet our guests and hear about the topics we’re going to cover. If they’re interesting to you, you can catch the full length show when it drops on Wednesday. We are here with Kunal from unraveled data Kunal, thanks for spending a few minutes with us today.

Kunal Agarwal 00:28
Eric, Kostas, thank you so much for having me here.

Eric Dodds 00:31
Give us your background, how did you get into data? And what are you doing today? Unravel?

Kunal Agarwal 00:36
Yeah, so can I like her well, founder and CEO of Unravel Data, that I started with my co founder ship, not Babu, who is a professor of computer science at Duke University. We both started this company to simplify data operations. We feel data engineers and data teams spend too much time firefighting issues rather than being productive in the data stack. And we wanted to automate it simplify some of the daily activities that these teams have to wrangle with every day and simplify their lives a little bit and make the data work that they do more exciting.

Kostas Pardalis 01:11
Yeah, and I’m very excited like with you today, not because I really won’t like to talk about how did operations have changed in the past, like 10 years, like, it’s extremely interesting that you’ve seen this whole, like, from the Hadoop days up to today, like all the changes that have happened, and I have a feeling that like, the complexity around like, data operations has like exploded, right, especially with like, having pretty much like every one or two years, like new use cases around data coming in, right? So even like, let’s say, observability. In data, what does it mean? What it meant, like, five years ago, and what means today, when we have also like AI, for example, right? In the mix. So I’d love to get more into like this journey, how things have changed, and what it means today, like to operate with, like to be an operator or like data. And of course, learn about like and forever, like, how it helps you that? What about you what are like some topics that you’re excited about?

Kunal Agarwal 02:15
Yeah, nurses, never a dull moment in the life of a data team member, especially those last 10 years. So we’ve gone from doing things with Hadoop, as a one Swiss Army, if you may, to having a multi system stack now, primarily running used to run on prem now primarily running on the cloud, that’s a mega change that’s happened. The other is you’ve gone from doing these batch workloads with ETL, to now doing real time or near real time workloads, you know, in production and marches as a science project. And then we’ve gotten from doing the BI business intelligence or just advanced analytic workloads, to now doing machine learning in AI in production. So if you’re a part of a data team, as a data engineer, or a data scientist or a data analyst, you’ve had to keep up with the demand of your business, and also had to keep retooling and rescaling yourself on how do you work on a MapReduce basis to to now a BigQuery. And Snowflake system is incredible. The rate of the pace of change and the rate at which things are getting involved in this ecosystem, right. So that’s a very exciting part for us. Because what what unravel is ultimately helping me do is to simplify how these data engineers, or data analysts are creating their applications, how they’re making sure that these applications are reliable, that they work on time, every time, they don’t break, that they get the right results. And then for the data leaders, and for people who are managing these projects, making sure that they are able to scale they’re very efficient matter. It’s not a linear scale in dollars versus, you know, productivity, can we bend the cost curve, as these environments are scaling up, so that they’re starting to get more bang for their investments. And as we now see the most exciting thing, you know, it’s actually here right now, not even the near future is AI, and how those workloads are changing businesses and turning industries upside down. So it’s a really powerful industry to be a part of, and really exciting time to be a part of this industry. But it’s not for the faint hearted it’s it’s for people who are up for a challenge to like change who like evolving as a light to try new things. And that’s what makes it exciting overalls for everybody in this industry. And I’m sure you’ve had the same experience due process.

Kostas Pardalis 04:48
Yeah, 100% counterweights like to get stupid and all that. Eric’s What do you think?

Eric Dodds 04:53
Let’s do it! All right, that’s a wrap for the prequel. The full length episode will drop. Wednesday morning subscribe now so you don’t miss it