Next-Gen Data Governance with Stefanía Bjarney Ólafsdóttir from Avo

Eric Dodds  00:06

Welcome to The Data Stack Show where we talk with data engineers, data teams, data scientists, and the teams and people consuming data products. I’m Eric Dodds.

Kostas Pardalis  00:16

And I’m Kostas Pardalis. Join us each week as we explore the world of data and meet the people shaping it.

Eric Dodds  00:27

Okay, Stef, from Avo is our guest today. And the question that I want to ask her is, which this won’t surprise our listeners who’ve been around for a while, but it’s about her background. She studied mathematics, did work in bioinformatics, and then went from bioinformatics into consumer mobile. And whenever I see sort of a progression like that, where there’s a significant sort of change in the industry or business that you work in, it’s just always fascinating to me how that happens. So that’s the main question I want, among many others, but that’s the main one. How about you Kostas?

Kostas Pardalis  01:09

Hi, I’m really interested to learn from her about data qualities, she has an amazing background and actually started super early, trying to solve these problems. And this journey also ended up building Avo. So I really want to learn more about how she perceives data quality, why it’s important, how it can be solved. And of course, learn more about how Avo addresses these issues and delivers value to their customers. So that’s from my side. I think we are going to enjoy this conversation with Stef a lot. Agree.

Eric Dodds  01:44

Well, let’s let’s go chat. Alright, we have a guest. I’m really excited about Stef from Avo. Thank you for joining the show, Stef.

Stefanía Bjarney Ólafsdóttir  01:57

Thank you for having me. Super excited to be here.

Eric Dodds  02:00

Yes, yes. Well, so many things to talk about. But as our listeners know, I really love hearing about people’s backgrounds and stories and what they did before they ended up doing what they’re doing today. And your background is fascinating. So you studied mathematics. I’d love to hear about, you know, hear about that journey, and you did bioinformatics, and then you jumped into consumer mobile. And so I’m just fascinated to hear. How you went from mathematics to bioinformatics makes more sense, but bioinformatics to consumer mobile? Seems like a crazy jump. And I just would love to hear that story.

Stefanía Bjarney Ólafsdóttir  02:39

Yeah. Well, thanks for highlighting it. I guess maybe one interesting aspect that might explain it a little bit. I didn’t only study mathematics, I actually did a double. And I studied philosophy as well, which makes me extremely curious about people and helping other people answer questions. And also just ask good questions. So you know, that might explain a little bit about how I moved into also being curious about just user experiences and people, how they experience games and things like that. But well and other aspects as well, that I just highlighted recently, in another interview, I moved around a lot as a kid, I had very young parents, and I actually had moved around 30 times before I finished high school. And that also, I think, makes me extremely curious about people. So I love asking questions. I’m extremely curious. I think people described me that way.

Eric Dodds  03:48

One question I just have to ask, so mathematics and philosophy, you would generally think about those as completely separate. But since you were studying both at the same time, were there any commonalities that stuck out to you in the two different disciplines?

Stefanía Bjarney Ólafsdóttir  04:07

Yeah, I love that you brought that up? Because yes, absolutely. They are surprisingly similar. I feel like the world is sort of split into two groups, the people who think it’s obvious that you would study those two together, and then the people who think is like, well, that’s really weird. And then and then when you dig a little bit into it, both of them are very, I guess, logic oriented. Mathematics is rigorously logic oriented and has language to describe everything and trains you up and really, you know, being logical and deducing logic and, you know, finding the truth. And philosophy does the same except it does it via written language and you are trained into taking texts of very complex words and vague statements and translate them into what are the things that are the actual assumptions behind this? And is it true? And what are the first principles behind it? And so I thoroughly enjoyed both. But obviously, also philosophy stretches a little bit into, you know, ethics and sort of thinking a little bit more about people and behavior and society and how we should build our society. And so I really enjoyed that part of philosophy as well.

Eric Dodds  05:28

Yeah, it is. It is interesting now that you say it that way. And we can move on because I don’t want to spend the entire time on this because I know we could.

Stefanía Bjarney Ólafsdóttir  05:37

I could.

Eric Dodds  05:38

I know, I could, too. But we have to talk about data because we named the show The Data Stack Show. It’s a legal requirement. But it is interesting the concept of sort of having tools and mechanisms that you leverage in order to understand something or find a solution to something. And that’s the same with philosophy and mathematics.

Stefanía Bjarney Ólafsdóttir  06:00

Absolutely. Absolutely. It’s just extremely oriented towards sitting for long times and thinking about very short statements and digging into them and like figuring out what the hell is true behind them. So yeah, I love that.

Eric Dodds  06:14

Well, tell us about what you did in consumer mobile? So you took that background into consumer mobile? And what specifically, how are you applying your skill sets there?

Stefanía Bjarney Ólafsdóttir  06:24

Yeah, exactly. I’ll go through them. So I went from university to … well, first, I travelled around Australia for half a year, which was amazing and then I came back home to Iceland, I’m from Iceland, and applied for a couple of the sort of industry, which would probably be called called startups, but to me were like just huge corporate organizations. And one of them was a DNA analysis company, I guess, called Decode Genetics, where I had the opportunity to learn from amazing statisticians, world class statisticians that have released, you know, fundamental discoveries and statistics and imputation, regarding how you calculate people’s DNA based on sample data, or based on their familial information and things like that. So, in that role, I did a lot of distributed computing, setting up, you know, all sorts of calculations and merging datasets again, but I also worked with the people over there that were like doctors and biologists, and they were trying to, you know their perspective was, you know, I’m curious about the mechanics of a physical trait of a human and then it was my job to bring in the information about like, how could maybe the DNA statistics and the data about their DNA have actually impacted that physical trait? So I was correlating physical traits with DNA mutations. And that was really fun. Yeah, I love that. And so I was one of the few people in Iceland at that point in time that had really worked with huge datasets, for real and not just you know, what you study in school, or like, you know, I don’t know, you, because you can train yourself in, you know, writing software and working with big data sets, but nothing really compares to actually, you know, having to go through the pain of fixing it, in reality.

Eric Dodds  08:23

Sure, there is such a big difference between stubbed out data for, you know, sort of learning and exercises and like real world data. That’s a totally different game.

Stefanía Bjarney Ólafsdóttir  08:34

It’s totally that; yeah, absolutely. And that jump was really educational for me. And I had sort of started studying bioinformatics after this job and actually had gotten offered a position doing PhD also in statistics/bioinformatics in Oxford, and also in Berkeley, which was really exciting. I was excited to move to California as well. And then this mobile game called QuizUp, blew up all of a sudden, and reached 1 million users in its first week, which was the fastest growing app in the App Store at the time. And the founders were Icelandic and I had some friends that were working in the company. And I was like, I can do a PhD whenever but when will I have the opportunity to work with these really super fun people and building up a data team and like, learning so many things about these people that are playing the game and I can build like a recommendation algorithm for what quizzes you should play if you like Harry Potter and things like that. So I got super excited about joining that team as a founding analyst. That was sort of the shift basically. So it was like the Icelandic community type of thing, but it’s a small society and just for example, the coach of the national soccer team, he is also a dentist. Yeah, everyone does everything in Iceland. So I love that. So that that was the switch. And that was sort of the backstory behind me joining this company QuizUp. And this was 2013. And so that was super early product analytics. Like, I don’t think the term even existed at that point. And just a couple of developers had thrown together analytics events using Mixpanel, if I remember correctly. Yeah, this was super early Mixpanel. This was like Mixpanel was still early. You know, we had a bunch of people that were just placing, like, powered by Mixpanel on their website to get some free events. I don’t think Amplitude even existed at this point in time. Segment, maybe like, just launched their famous like, Hacker News point posts where they released like, a wrap around all of your analytics.

Eric Dodds  10:54

Like those are like the Genesis time periods.

Stefanía Bjarney Ólafsdóttir  10:58

Exactly. So when you’re googling stuff, I was just like, what is retention you say, what, what, what does that mean? And you couldn’t really find definitions of it. And so when the board was asking our CFO who I reported to about retention after every launch that we made, and for every board meeting, I was like, I don’t know what they really want, like, you know, you have millions of definitions of retention that we could work off. And, you know, I don’t even know what the benchmark is, I don’t even know what they would consider to be good, what they would consider to be bad. So I just spent my time at that point, digging into data. But, you know, most of the data was terribly broken and inconsistent. These few events that they threw together into Mixpanel. So I sort of spent my time just hacking together scripts, and you know, you know, making replicas of the operational databases, which were like 16 shards of the user base, and Postgres hosted on AWS or something. And it took maybe probably typically two weeks to answer some really basic question.

Stefanía Bjarney Ólafsdóttir  12:11

Obviously, that was not acceptable. So we ended up fixing that problem by basically throwing people at the problem. I started scaling up the data team and hiring more people so more people could be answering questions like these. But eventually, and while we were doing that, we were always for every single feature release, we were begging the mobile developers to add, you know, specific events for some things and questions that, you know, the board always wanted to answer, or the CFO or the CEO always wanted to answer. And it was so interesting, the dynamics, because we were in the position to have to report on these things. But we had absolutely no control over what data existed. Absolutely no control over that. But in some way, it was still our fault if we couldn’t answer the questions, which is a bit of a tricky situation to be in.

Stefanía Bjarney Ólafsdóttir  13:03

So that was sort of the birth of how I evolved this role, which was really amazing, and a great opportunity to be head of data science for this company. Because it was, I guess, a twofold role, I got the opportunity to really build the data culture, which is sort of, you know, getting people curious to use data to answer questions and, and go beyond their, their gut feeling. I say beyond because I know that gut feeling is extremely important in making product decisions. And sort of, you have to have some insights, and sort of an understanding of the customers and the users. But you know, it’s really important to be able to back it up with data. And it also just makes the buy-in from anyone in the company, just stronger if you’re able to do that. So just getting people curious to do that. And basically doing some internal marketing, like, you know, hey, here’s the cool things we know now, because of data, what could you know, and you know, have any, having regular meetings with, with everyone in all of the different departments in the company, trying to figure out what their needs were, trying to just like lobby for people using data. And that was really fun. So building up the data culture. So that was one aspect– data culture. And then the other aspect is just everything around the technical infrastructure of data. So making sure we have the right tracking in place, making sure  we had good quality of the data that we did have, making sure we have the infrastructure in place to both do self serve analytics via tools like Amplitude and Mixpanel, but also extensive raw data analysis, and building the infrastructure so that the data scientists were able to, you know, we built our own version control for Jupyter Notebooks because no tools existed at that point to to do that, but everything is already built around that today. So nobody ever has to do that, again, like, we scraped at SDK webpages to get some data from there because they didn’t have their own API’s. And we built out like Redshift clusters, and pipelines to pipe the raw data from all of our all of our data sources into Redshift, which could have been done with, with Segment today, obviously, or on Particle or any of the CDPs, RudderStack, all of the amazing tools that now exist. But none of that existed then. But yeah, so it was a hugely wonderful learning experience.

Eric Dodds  15:39

You were pioneering so many things.

Stefanía Bjarney Ólafsdóttir  15:41

It was fun. Certainly. I learned a million things.

Eric Dodds  15:46

So one more question. I have been very selfish with the time here. And I know Kostas has so many questions to ask and we want to hear about Avo, but one more question. And I’ll ask you to do the impossible task of distilling this down into just a couple quick lessons, but hearing about your experience, pioneering, tooling that almost a decade ago didn’t exist. And now operating in a world where product analytics is table stakes for any software company, you know, or sort of high performance software startup. And there’s so much infrastructure. I mean, that has been a tectonic shift, to not only sort of live through, but also have actively taken part in so I’m just interested to know, what are some of the, you know, the top, maybe two things that stick out to you having seen and really been part of sort of pioneering that entire shift in the data space?

Stefanía Bjarney Ólafsdóttir  16:56

Oh, that’s a really good question. So obviously, I have very strong opinions here. I guess loosely held, because I love being questioned on them and having the opportunity to discuss them. Because I know that there are just so many people that have amazing experiences and just like if anyone who is listening has anything amazing to add to what I’m about to say, I would really love to hear that. So reach out to me via Twitter, or like my email or whatever. But I think there are probably two things that are counterintuitive to anyone who is working in product analytics, and is sort of fresh in product analytics, and was very counterintuitive to me, also, when I started, and that is this, on one hand, I would say that less is more. That is sort of like a counterintuitive thing. And that’s also a very debated point of view, I would say in data. But from my perspective, it’s more important to choose fundamental parts of the customer experience and understand those points very well. Rather than having like vague understandings of like, you know, I guess like a wider set of data. This is one of the things that I think matters a lot. And there are multiple reasons behind it. And I can, I can definitely go deeper into them. And I would love to actually, but less is more; it’s a counterintuitive one.

Stefanía Bjarney Ólafsdóttir  18:34

And I think then the other one that I that I really fight for and and I’m very passionate about it myself is and I have had the opportunity to see through in QuizUp. And then as a consultant after QuizUp, working with a few companies around the world, and like just advising them about what they should do next. And then obviously, with our customers at Avo, and that is I have a deep passion for bringing data producers closer to data consumers. And when I say data producers in this context, of course, you know, I’m mostly thinking about customer experiences. And so that is often coined as product analytics. And so the data producers of those are the developers, the product developers that write the code within the applications, the web apps, the mobile apps, or sometimes the back end, that actually get events into other systems into a RudderStack, Segment, Mixpanel, Amplitude, Heap, Pendo, Firebase Analytics, your BigQuery instance, wherever you want to put your user experience event data, the producers or the developers that write the code in the product that triggers an event when the user does something. And those are the data producers and the data consumers or the data analysts and the product managers that use tools like Amplitude and Mixpanel, or Firebase Analytics, or Looker, or Tableau or whatever, to look at that data. And what I saw happening in QuizUp, for example, is the data producers and the data consumers were entirely separated. And I was just sharing this in the story just before, which is, I reported to the CFO and the CEO, and the board of directors, and I made reports for them. And their expectations were that I would be able to answer behavioral questions. But the product team had no obligation to sort of let me know when they were about to release something or you know, ask me for their input or feedback on how they should instrument some data or make sure the data existed. And that meant that when I was asked for information, I had just no way of answering those questions. And obviously, that was, I guess, a major driving force behind this in sort of bringing data producers closer to data consumers, and was the driving factor behind working closely with the CTO of QuizUp. And building tools and processes that added way closer collaboration with the product managers, the data scientists, and the product developers for every single feature release, and making sure we were really asking ourselves, what was the goal of the feature, before we even started implementing it? So that we would also be able to, you know, design, what are the metrics that we will use to measure the success of that feature, so that we will then in the third section, be able to actually make sure we have the data to measure those metrics. And so you know, when we started that process, we had maybe one or two developers in the company that opened charts at some point in their software development lifecycle, opened up, you know Amplitude or Mixpanel and asked some questions. But after we sort of made this a fundamental part of the software development lifecycle, or the product development lifecycle, I would say, like 70%, of the developers of the company, they started, you know, they started asking the questions, and they would be posting on Slack, like, Whoa, check this out, we just saw, like, because they wouldn’t know when the product was released. And they would be like just waiting, waiting for Amplitude data to trickle in, or waiting for Mixpanel data to trickle in. And so they would just like to go and look up their charts, because they knew the definition of the metric, because we had defined it together. So they would just be able to point and click a few things, and then open up the chart and share it with the team and ask some follow up questions. And this again, like this change, bringing producers closer to consumers that really impacted the quality because all of a sudden, the producers were consumers. And they fundamentally understood how painful it was themselves, when the data didn’t work.

Stefanía Bjarney Ólafsdóttir  19:36

Yeah, and it actually ties a little bit into the other sort of the other, less is more angle as well, which is, you know, if you think about data, and from this perspective, that you you plan it as a part of your product release cycle, you realize that it’s never a good strategy to decide to track everything, because you know, just like when you’re releasing products, you just have to say no to things, it’s really important to focus, because you won’t be able to do everything well. If you try to do everything, you will be able to do a lot of things really badly. And it’s way better to measure a single thing really well. And then have the opportunity to ask some follow-up questions rather than having, you know, bad answers to a bunch of questions.

Eric Dodds  23:51

Love it. I could talk about both of those points all day, but I have to hand it over to Kostas because I’ve been monopolizing the conversation. Take it away.

Kostas Pardalis  24:06

It’s okay. I mean, you had an amazing conversation so far. So I really enjoyed listening to what you were saying to both of you. Maybe we should arrange another recording where each one of us can impersonate a philosopher or something. And yeah, let’s do that. I mean, let’s make a recording with Russell, Wittgenstein, like these kinds of people. It would be great.

Kostas Pardalis  24:31

I do have to say hearing a mathematician and philosopher say that your gut instinct is really important–I thought that was really cool. All right. I’m officially done now.

Kostas Pardalis  24:46

All right, I’m going to ask, I’d like actually to start by giving you a bit of more background around the company itself, Avo. I mean, we talked a lot about what happened early in your career and what made you do the things you are doing today. But it would be great to learn a little bit more about the company itself. So can you share some more information about the company?

Stefanía Bjarney Ólafsdóttir  25:09

Happy to. Yes, well, I’ve given sort of a backstory. But basically, you know, the tooling that we built at QuizUp, the tooling that me as head of data science, and the CTO, who was at that point, sort of his background was in mobile development. His complaints were from the developers who hated how frustrating it was too and time consuming and like inefficient it was to implement analytics events, and particularly how often it failed, and how often I would have to go back to them and be like, oh, excuse me, I’m sorry, you did this all wrong and now I can’t answer my questions. And that was so frustrating, and it was frustrating for everyone, obviously, nobody likes, I mean, they, they don’t want to spend a lot of time on this, but they also don’t want to disappoint. Like, nobody wants to do that. Nobody wants to deliver something that, you know, is then in the end, just useless. And obviously, the board and the CEO and the CFO, they wanted us to be able to answer questions. So we, you know, being two very passive people, me and the CTO, we worked really closely together in building tools that automated a bunch of this stuff. We built a central source of documentation for the schemas, basically, that had event descriptions, and the types of the properties and all that stuff. And then we built like code generators, so that it would just be super easy for Android and iOS developers to just, you know, implement analytics events without having to think too much about it. All of the processes that we also built around this design part, the data design part, we call it purpose meetings at QuizUp, was this thing about, you know, setting goals, and then deciding the metrics, and then designing the events based on that. This originally was so that, you know, data scientists would be involved in this process from start to finish. And we would propose the event structures based on the needs of the data consumers. In the end, though, as the data producers became closer to the data consumers, we just saw this really clear shift where, you know, you know, it was just about aligning on goals, and maybe deciding together on the metrics, but in the end, everyone just was so sure of designing the data. So the developers were designing the event structures, based on, you know, the code base, and that data, you know, like how the data sort of best practices that we had in place at QuizUp, the naming conventions, that we had, all those things. So, eventually, sort of what I would like to say is, you know, we transitioned from a centralized analyst, you know, I was the person that answered the questions, to an attempt to have self-serve analytics, where, you know, we wanted everyone to answer questions. But that was super throttled by lack of data literacy and lack of data quality. And so, as we built out these processes of ensuring data literacy, even before the feature was out, and then the tools to ensure the data quality before the feature was out, we basically also built, we empowered ourselves around analytics by building the tools and processes for self-serve analytics governance. And that is something that then, you know, I started taking for granted. But fast forward, a couple of years later, QuizUp had been acquired, and I had started a company with a couple of friends. And there was only about five months into that company where we actually shipped a product update that was based on incorrect data. And it was just like a punch in my gut. And I was like, Oh, my god, I’m back to back to like, five years ago, or whatever, and I will never be able to work in digital products, because it’s always gonna suck. And I had this realization moment where, oh, my god, you know, all the tools that we built up QuizUp, what are we going to do without them? And you know, we’re trying to take this product to market and it doesn’t make sense for us to spend time on building those tools again, I can’t believe it. And that’s when I started sort of syncing with some of my colleagues from the industry, people from Spotify, and Twitch and Airbnb. Turned out that we were super early at QuizUp in building these tools. We were building them in 2014, while Spotify built a compatible thing in 2016. And how, like, what I can’t believe that that’s super cool. And that was sort of a trigger for us to really build Avo and start sort of bringing the solutions to the general business, the general digital product. And so that’s the backstory, but the sort of what Avo does today is Avo is analytics governance as a service. We are here to pave down the analytic steps of Silicon Valley, which is like, that’s, there’s a lot of that going on. And we’re here to really optimize this release process for the analytics workflow. And we’re bringing data producers and data consumers together with Avo product managers and developers and data scientists to collaborate, to plan, and to validate and to implement their analytics for every single feature release. And it fits entirely with the sort of the Git product release workflow, we sort of were like the pioneers for building a brand to workflow for your tracking plan, which, you know, typically a lot of schema management tools have had versions, semantic versionings, for your tracking plans, or for your schemas. But that’s something that’s scary if you’re not a data engineer, basically. And one of our earliest learnings was, you know, we really need to support the data consumers well, and they’re not necessarily data engineers, they can be product managers that, you know, effectively only understand what their goal is with the feature, but don’t have a deep understanding and event structures and good event structures or semantic versioning. And so this is our mission, analytics governance as a service, bringing self-serve analytics governance, to empower really that shift to self-serve analytics, which is a fundamental piece for companies to succeed today, I would say.

Kostas Pardalis  31:42

Absolutely, totally agree with that. Can you also tell us a little bit about when the company was started, when you launched the first version of the product, like your experience with the early traction, your experience with fundraising in the space, I think that would be amazing for people to hear about.

Stefanía Bjarney Ólafsdóttir  32:01

Yes, absolutely happy to share a little bit about that. We are very customer centric as a group of people. So we care deeply about our customers. We, like I was talking about earlier, have strong opinions in what we’re doing. However, it’s really important for us to confirm that with by talking to our customers and looking at the data for how they succeed. And that’s how we have built the product from the beginning. We have had customers from day one, when we started building our prototypes, we also got into YC. So we are a Y Combinator company. And that was because Gustaf Alströmer, who used to or was, I think the, one of the early growth persons, I guess, in Silicon Valley, he was a growth lead or growth product manager, or product manager for growth, or all of these variations of the growth titles, at Airbnb, so he, you know, was a fundamental piece in the growth strategy for Airbnb. And he experienced this problem immensely. So when we did our Y Combinator interview, he was like, I’ve had that problem. And so we did Y Combinator, which was, of course, an amazing adventure.

Stefanía Bjarney Ólafsdóttir  33:19

And following that we raised a funding round in Silicon Valley. Before that, we had also taken in some funding from Icelandic venture capitalists, but also angels that sort of supported us, also, because we’re, you know, it was a team that had a lot of, I guess, I’m lucky to have been working with a great team in QuizUp, obviously. And so that was a reputation that we had built also in, in Iceland. And so even though this was a very pioneering idea at that time, and nobody was talking about this, when we started doing this, nobody was talking about it. And so but they sort of, you know, they, they funded the team, you know, they believed that we were a team that would be able to ship stuff, and build stuff. And so that’s exciting, because it turns out they were right, yeah. And then after Y Combinator we raised a round also from a company or a fund called Heavy Bit. They are an accelerator and a fund for developer tools. They were founded by James Lindenbaum, among other fantastic people who was the founder of Heroku. And James’s perspective on developer tooling has really helped a lot of fantastic developer tools go to market for example, CircleCI, Stripe, just endless. Go check them out: heavybit.com. Fantastic group of people that have been really helpful. And then GGV Capital, fantastic partners there as well. They’ve funded giants, as well, such as Slack and our partner there, Glen Solomon, and Oren Yunger. They’ve just been immensely helpful as well and the entire GGV capital team as well. And I think they’re also very passionate about sort of both developer tooling and the data space. And so that was sort of their drive. They also work pretty closely with the Heavy Bit team, often, so I guess there was sort of hand-in-hand type of thing. And we’re lucky to be working with these world-class backers of like, you know, developer tooling, and then the data space that have huge experiences in both sort of like, developer adoption, and enterprise sales. I have them in my inbox all the time. And I’m able to ask them for advice. So I’m really fortunate and lucky about that.

Kostas Pardalis  35:44

Yeah, that’s amazing. And I understand how you feel about feeling fortunate, but I’m pretty sure that all these people got involved also, because of the team and the passion that you have about solving the problems. So the reason that they are around is also because of you, I mean, your personality and the rest of the team. So congrats for that, that says a lot about the quality of the team, the company and the product.

Stefanía Bjarney Ólafsdóttir  36:07

Thank you for that. Those are kind words. Thank you.

Kostas Pardalis  36:10

Alright, so moving forward, let’s discuss a little bit more about the product. So our audience can learn what Avo is about and how it works. And let’s start by what’s the value that someone gets from Avo today? I know you have touched this previously in the questions, both Eric and me, asked, but yeah, let’s try to make it a bit more concrete about the value proposition that Avo has as a product.

Stefanía Bjarney Ólafsdóttir  36:37

Yeah, thank you. So ultimately, we see a lot of our customers talk about how much developer time they save. Obviously, they save a lot of data quality, or like just they rescue their data quality, and they start paying down their analytic steps, I, I would say. That’s the path for, for example, companies like Rappi, which is a huge Latin American company that is like DoorDash, and a Lyft/Uber/Instacart, all of those combined, basically just delivering whatever you need as a service. And obviously, that exploded, in sort of COVID and March, April. And that is effectively sort of around the time when they reached out to us. And they had been trying to audit their data. For months, they had been working on that the BI team had been trying to sort of catch up on even just understanding what was currently wrong with their analytics, before they wanted them even to try to try to fix it. At the same time, the developers were just tearing their hair out, because of the inefficiencies of implementing these analytics. So their journey was they installed Inspector, which is our sort of data observability product. You install the Inspector SDK to inspect and observe your current state of tracking ongoing. And that’s how you get a snapshot off your your analytics steps there, and are able to fully prioritize what is wrong with it today, so that you can, so you can prioritize what to fix, like prioritize the top things that you really need to work, like three, four, or five events that you really want to make sure, okay. And then you can just like gradually, step by step, pay down your analytics, adapt. So that’s their first step. And then they team by team adopt the Avo workflow, as we call it. So the Avo workflow involves all of these stakeholders that I’ve talked about the product manager comes into Avo opens up a branch to plan the analytics alongside a feature release, the data scientist and the engineer, they sort of give it a review, request some changes, potentially, the engineer from the perspective of like, Yes, I can implement this analytics event in this format. And the data scientist brings in sort of like the bird’s eye overview of the company analytics, and like, yeah, these analytic structures make sense in comparison with the rest of the company. Great. Let’s move on. And once that’s been approved, that’s the third step where the developer pulls generated code, we generate typesafe analytics functions, we call them Avo functions, where you can implement using, you know, Avo dot item added to cart or whatever, instead of calling up generic analytics SDK, where you have to pass in a string for the name of the event, and then a huge object blob with like a name of properties and a bunch of property values. Instead of doing all that you just call a typesafe Avo function that you generated using your Avo CLI based on the tracking plan, which is all defined in a user friendly UI by the pm and the data person.

Stefanía Bjarney Ólafsdóttir  40:01

That’s the third step. And then the fourth step is, you know, the developer, you know, treats the build, just like they would for any other build, they just go through their product before they release the build, before they sort of release the feature update. And then they use the in-app Avo debuggers, where you have like a bubble that shows you when you’re triggering the analytics event. So you can confirm the timing is consistent across all of the platforms, you can confirm the data structures are exactly as you intended for them to be, the PMs and the analysts also love the in-app debuggers. And they use them even, you know, outside of the regular feature release process, because they can really get a full understanding of like, what really is being tracked in the product. And so after that is sort of quickly checked off, the branch basically gets merged, the Avo branch, and that happens just at the same time as you branch a merger git branch, and your tracking plan, update, gets automatically published into whichever other source you need schemas in. So that could be your downstream, you know, your BigQuery table somewhere or your your Redshift table or your Proto file or your Amplitude govern or your Segment protocols or your Mixpanel lexicon, just wherever you might need to manage your schema, you can then automatically publish that, after that branch has been sort of finally reviewed and implemented and just ready for production.

Stefanía Bjarney Ólafsdóttir  41:29

And that’s when then just everything is released. And like this entire cycle, the workflow, it’s shortening the time that people spend on this, by, you know, incredible amounts. So for example, Patreon, they reported, basically, they are getting 90% of their time back that they used to spend on this for every single feature release, they used to spend one to four days on this back and forth, and like a QA process and syncing between iOS and web or whatever and data. And now they consistently use maybe like 30 to 60 minutes on it. And so that’s obviously a huge time saving.

Kostas Pardalis  42:11

Yeah, that’s amazing. Actually, something that I want to add in what you said, Stef, is that my feeling, based on my experience so far, is that the most successful B2B products deliver innovation in two directions at the same time. One is, of course, like the technology and the product itself and how it solves a technical problem, for example, this case. But the other thing, which is equally important, and not everyone is doing it is that it’s organizational innovation, as I would say, which means that it helps the company to reorganize around more efficient processes, or automating processes. And I think that when you combine these two together, you have at the end, like an amazing, both value proposition for the company and the product. But also you have building an amazing model for your company. I’d love to chat more about this. I mean, it’s something that I’m a bit obsessed with to be honest. And something I’m always trying to figure out. And there are some companies that they do that, like they did that like Slack did that. Looker did that. That’s exactly what LookML plus the visualization layer was doing and creating a very clear and efficient distinction between the people who do the modeling of the data and the people who are consuming the data like exactly what you’re saying about the data producers and consumers. Anyway, that’s a very big topic that we can keep discussing. More,

Stefanía Bjarney Ólafsdóttir  43:39

We will bring it up in our Wittgenstein conversation.

Kostas Pardalis  43:41

Absolutely, absolutely, we can do that. So alright, moving forwards. Let’s talk a little bit about the focus around the data that you have. I mean, like a company generates a lot of different types of data. I mean, from logs that might be consumed, like for security reasons, to clickstream data without like, actually describing or like the breadcrumbs, let’s say of the customer behavior. It looks like you’re focusing more on the second case, like you, you are mainly working with product analytics and the clickstream data that customers are generating. Do you want to expand a little bit more on that? Like, why do you think that’s more important, if it is more important? And what are your plans in general, like as you move forward with the product or bigger plan, for example, like to support something else?

Stefanía Bjarney Ólafsdóttir  44:29

Yeah, these are really good questions. So I think the first part here is the why. Like, why are we focusing on customer data? Why are we focusing on product analytics and, from my perspective, I guess, like, obviously, what impacted is I joined as an analyst applying for a role to report to the CFO. The CFO obviously thinks about finance and cares about finance, and that’s really important to them, but at the same time this was you know, 2013 when you know companies like Zynga, Spotify, Facebook, these companies had just taken over. They were the digital products. They were the companies that it sort of showed us that we can have digital experiences, solving things that we use to solve in person, right, games, listening to music, Airbnb, another example, where we used to call and book hotels, but all of a sudden, we had these really wonderful online experiences, through going through a product going through an experience, and just booking a place to stay. And this shift was really going straight. I mean, it started, like 2005, 2006, 2007, when, you know, we started seeing all of the apps happening, like just more of the things that we needed to do as people, as humans, just it became more and more mainstream for us to be able to do that online as products, either on web or mobile. Obviously, like software development has been around for a while, and we had the dot com boom, but this sort of shift really started happening at that point in time, I would say. We were going into digital products, digital experiences. So even while I was reporting to the CFO, their biggest problem wasn’t financial data, their biggest problem was really impacting like understanding and being able to report to the board of directors, the leading indicators for the revenue, because revenue, I think most people have sort of agreed to in the space right now already, revenue is a lagging indicator, it’s not a leading indicator for your success. While your customer experience can act as a really important leading indicator for your key performance indicator, which is typically revenue, or some sort of other way of measuring that you are bringing value to your customers and that your business will continue to be able to thrive and sustain itself. And so I mean, that’s obviously my background into this and why, you know, I, I thought that this was one of the most difficult things to report on was this experience analytics. So what we focus on at Avo is we basically focus on event-based data, it’s typically event data that describe user experiences, because those are the things that really sort of are the leading indicators for companies’ successes today, for digital products’ successes today. And I would say, today, if you’re a company, and you don’t have a strategy for your digital presence, or for your digital products, it’s likely that you won’t survive. And so I think this is just a fundamental piece of business strategies for right now already. And we’ll see like the laggards just falling behind the train in the coming years if they won’t catch up and keep up. So that’s really one of the really main drivers. And just also to take an example, you know, one of our customers, like Patreon, they’ve been with us for a long time, they were one of the sort of earliest larger organizations to take a bet on us. And we have a fantastic relationship with them. They’re fantastic. Maura, and Jason, and everyone over there at Patreon, it’s been fantastic to work with them. And they know so many things about this space. And one of the things that Maura has already shared with me, she talked so much about this, like, obviously, revenue matters to them. The performance of the user experience in the customer journey also matters a lot to the financial teams, and they already use that in their modeling, to be able to sort of have some leading indicators for their success. So and then if I tie this to like, how do we see this evolve for Avo? I mean, there’s no limit to what we can support customers in, you know, fully structuring their event based data. However, I think, the biggest challenge and tying it back to what you were saying, like change management and organizational shifts, and how people build their organizations and their sort of focus, I think that is really around like the customer experience. So that’s a big passion for me, and in helping people sort of do that better. And so I think while we haven’t fully solved that problem, we’re gonna focus on fully solving that problem. I would say.

Kostas Pardalis  49:21

That’s, that’s some great points Stef. And yeah, I mean, I think we are going through like a major cultural change inside companies when it comes down to how data and KPIs and all that stuff are perceived. And I think it’s going to be super interesting to see how these things will change in the next couple of years. One last question from my side, then I’ll hand it to Eric. You mentioned at the beginning that one of the lessons learned for you was that when it comes to data, less is more at the end. Can you expand a little bit more on that? Because I think it’s going to be super interesting.

Stefanía Bjarney Ólafsdóttir  49:58

Yes, so again I think like, what I mean by that, so I touched on it a little bit before with like, when I was talking about, you know how important it is to bring data producers and data consumers closer together, and sort of bring this whole process really into the product development lifecycle. What the mistakes that I’ve seen people do all over, like, again, and again and again, is when they decide to track “everything.” And, you know, this mandate can come from, like, it can come from the developer that is just building the product, and they have a mandate to add some analytics for it. And so they’re like, yeah, let’s just track everything. But it can also be something from like, a data person or a CFO-type of person that really just doesn’t understand or haven’t taken the time to think through, what are the fundamental steps in the journey that we really need to understand the friction point for, so they give this mandate to track everything. And this is just the worst track that I see people go through, again, because of the same reasons why we can’t just build everything, right. So building everything, building, you know, building good products, means focus, it means saying no to things, it’s difficult to say no to things. But we really need to do it. Because if we don’t, we will either never ship anything, because it just takes so long, and we’ll never have anything ready. Or we will just do a lot of things really badly. Right? They just won’t work and they won’t have a delightful experience. And you know, you know what it is today, and building products is like delightful experiences are the fundamental piece in you succeeding and bringing your product to market. So the same thing really applies to data. Like if you just try to track everything, you won’t have time to do it. And you just won’t, you won’t, there’s no such thing as tracking everything. And instead, what you’ll end up with is like, you’ll track the random things that you were able to squeeze in before you had to press the release button. And that’s really bad, because then you definitely, or it’s highly unlikely that you have the most important things tracked if you go that route. It’s really highly unlikely. And obviously, here I’m talking about sort of custom tracking, I’m talking about manually adding events. And that brings me a little bit into the subject of like the outer tracking. And I know RudderStack supports outer tracking. And we have some fantastic products that do a lot of outer tracking, like Pendo, and Heap and RudderStack and FullStory, a bunch of outer tracking products. The challenge behind this, like I often put, like I often, sometimes at least, I recommend to like PMs or people who just don’t have time yet to think about their data. I’m like, Yes, sure, go ahead, install this, like, it might give you some insights. And it’s like a safety net. I understand that you’d rather have something than having nothing. However, it really just postpones the problem of really thinking through what is the information that you need to answer. It postpones it, so that you will have to be digging through a lot of noisy data after the fact. And it’s really difficult for non-experts or even just like non-developers to understand what data they are looking at. Because typically, those data points are what they mean, they’re named after some IBs that the outer tracker found through the DOM. And it’s a lot of work to figure out what this means. And even if you do, it’s not even guaranteed that you have the tracking that you need. Because good analytics events, they typically have event properties and segmentations in place that you won’t have automatically. And that’s that thing, like designing good data structures. And so even if you know, go through the outer tracking process, and you have your like safety net, it’s just, it’s gonna be difficult to sift through it. It is like looking for a needle in a haystack, certainly not the way to go if you want to build yourself for analytics and sort of increase the company’s trust in data and sort of their trust in themselves to be able to go into a point and click the UI and just like, figure out what they want to enter.

Stefanía Bjarney Ólafsdóttir  54:12

And also, it’s not even a guarantee, it’s not even a safety net, it’s not even a guarantee that you will have the information that you need. So these are my two heartaches, like tracking everything, the tracking everything perspective. And because this is really difficult to do like tracking a lot and tracking everything is really difficult to do. My general recommendation is just start small. Like if you haven’t started adding tracking yet, choose three events and start there, like just absolutely start there. And I know you want to track everything but just stop there. Because it’s way better to ship something and ship a small slice and be able to start building out that sort of like, Oh, interesting. Oh, that was fun. That was fun how we were able to make a decision based on Rather than just turning like tracking into accidentally for two sprints you didn’t ship any tracking, because you just decided to track too much, and you didn’t really have time to finish it. So that’s a little bit on the sort of less is more perspective. I have way more thoughts on that as well. But I think that sort of captures a couple of the essences of that, I think.

Eric Dodds  55:25

Well, unfortunately, I think we’re close to time. But also, I think we definitely wouldn’t have enough time if I asked more questions, but I have many, many more. But we can save all of that for our philosophy roleplay episode. I learned so much Stef. And I just think our listeners will really appreciate the lessons that you have brought us on this episode, based on your vast experience. It’s just really helpful. And I think there are lessons that apply to companies or data professionals that are just starting out, and really large organizations and just always appreciate wisdom, that time-tested wisdom that’s sort of applicable to all of our listeners. So really, really appreciate you being on the show. Sounds like you’re doing amazing things with Avo. So excited for you to continue there. And we’ll schedule a time to have you back on the show in the next three or six months.

Stefanía Bjarney Ólafsdóttir  56:28

Thank you. Thank you for having me. It was super fun. I can’t wait for our roleplay  episode. Yes. Okay, thank you all.

Eric Dodds  56:39

Well, that was an absolutely fascinating conversation. I think one of the things that really stuck out to me was how balanced of a perspective that Stef has. And I think the statement that just really, really stuck with me was, she said this almost in passing, but she mentioned how important your sort of gut sense is in making decisions about a product. And just hearing about her background and how much work she has done in data and data tooling and just the sort of the massive amount of understanding and capability she’s developed. Yet she seems to have such a mature intuition about how to do things well, and not necessarily just get lost in the data or put that above absolutely everything else. So I just really appreciated the perspective, what was your big takeaway Kostas?

Kostas Pardalis  57:39

Well we for sure need to do this philosophers role playing episode. That’s going to be a lot of fun. What I I keep from our conversation with Stef, is how important culture is when we are dealing with data. And as we said during our conversation with her, I think we are at the beginning of a tectonic move or change what is happening inside companies in terms of how they perceive data and their value. With very good example that Stef gave about how growth is perceived even from the CFOs and how even the CFOs perception has changed. And non-standard KPIs like retention and customer satisfaction is becoming like a leading KPI even for them. And yeah, I’m really looking forward to chatting with her again in the next couple of months. Avo is growing really fast. And I’m pretty sure that like in a couple of months from now, we will have even more to learn from her and her team.

Eric Dodds  58:47

I agree. Well, thanks again for joining us on The Data Stack Show. Be sure to subscribe on your favorite podcast tool to get notified of new episodes every week. We have some really, really exciting shows coming up in the next few weeks that you want to make sure to listen to. Until next time, catch you later.