What if Your Code Just Ran in the Cloud for You? Featuring Erik Bernhardsson of Modal Labs

Eric Dodds 0:05
Welcome to The Data Stack Show. Each week we explore the world of data by talking to the people shaping its future. You’ll learn about new data technology and trends and how data teams and processes are run at top companies. The Data Stack Show is brought to you by RudderStack, the CDP for developers. You can learn more at RudderStack.com.

Welcome back to The Data Stack Show, I am incredibly excited about our guest. Today, we’re going to talk with Erik Bernhardsson, who was one of the early engineers, Spotify did a ton of things there, including music recommendations, it started at the insurance company when they were just a couple of people and scaled that he was there when they grew to 10,000 people. So just an incredible experience. And he’s building something new and fascinating called modal. Kostas, this is not going to surprise you or the listeners at all. But I want to hear about the lessons that Erik has learned going through this drastic phase of scale two times over, which I think is pretty rare. And especially to do it at a company like Spotify, which I think solves a problem that a lot of other companies tried to solve. We tend to think of Spotify as dominant today. But back in I guess 2008 There were a lot of other major players in the space, and they were just a small, scrappy startup. So I’m just really interested to hear that story. How about you?

Kostas Pardalis 1:36
I mean, obviously, like, I’d love to hear about that. But Erik also likes building, let’s say, the new version of the next generation of cloud infrastructure. So this baseline server lives that it’s like, much more seamless as an experience like from, from the developer. So I’d love to talk more about that stuff and see, like, what sort of realistic experience here is, first of all, why it is important. And what’s the line you have to do to build something for this service? Right. So yeah, that’s what I’m going to focus more into I’m pretty sure where I’ve been kept surprises today. So

Eric Dodds 2:21
Oh, yeah. There’s no question. There is no question. All right. Well, let’s jump in and talk with Erik.

Kostas Pardalis 2:27
Let’s do it.

Eric Dodds 2:28
Erik, welcome to The Data Stack Show. So many things to cover. So thank you for giving us some of your time.

Erik Bernhardsson 2:33
Thanks for hosting me. It’s fun.

Eric Dodds 2:36
All right, well, give us your background, kind of what you’ve done over your career. And then tell us a little bit about what you’re doing today.

Erik Bernhardsson 2:45
Yeah, I’ll try to condense it down. I originally— I’ll do it in chronological order. I grew up in Sweden. I do like programming competitions. When I was little, I was always coding and like messing around woods. This was back in the 90s. Running code and building crappy websites. Ended up being a bunch of people on school who were early at Spotify. So when I was done with school, and I started physic put it in, not that I don’t think it matters, but it but I ended up joining Spotify into just eight, spent a couple of years there and stopped all building the music recommendation system eventually ended up moving to New York. I built up a team did all kinds of other stuff, too, at Spotify, not just music racks. But that was sort of the prime thing. But also did a lot of like, random sort of data infrastructure and ended up open sourcing of technology, which was an orchestrator and nothing called annoy or suspected database, built up to about 30, left in 2015, and took a job as a CTO at a company that was then 10 People that was there for six years was better. That was called better as a mortgage company. The company ended up growing at 10,000 People then went through some challenging times. So it’s a little bit smaller now. But so oversaw the technology there. And then I left almost two years ago, and left in order to pursue a bunch of random ideas. I had wrong data and started thinking about okay, like, how can we build better tools? What’s missing within him structure today? What’s the tool that I was wondering? 20 years, I started working on something. And last two years, I built up a small team and raised some money and I don’t know if I should name but yeah, my model is a copy we’re working on right now.

Eric Dodds 4:34
Very cool. Well, we want to dig into Modal because it’s so exciting. But first, you have such a unique experience. So Spotify 2008, a handful of people on the super early team. I think what’s interesting, actually, I think back on that and, when you think about streaming, Spotify is the first thing that comes to mind, but back then there was our do and like a couple of other major streaming services that were really large, actually, at the time thinking back in those sort of early days of streaming, which is super interesting. But you have been through this scale arc, you no more than once, right? So like, Spotify in 2008 to 2015, hypergrowth, the company and grind to 1,000s of people better 10 people to 10,000 people, just serious exponential growth. And with those unique experiences, I’d love to look at that from two angles. I mean, both personally, I’m just so interested in this, but I think our listeners as well, let’s talk about Spotify first.

So, between the time you started and the time you left, the technology landscape went through an unbelievable change. There were things in 2015, I think you said when you left, that didn’t exist back in 2008. And so you have this massive change happening in the technology landscape as your company’s going through this period of hypergrowth. How did you manage that on the technology side? Because it seems like you would constantly be facing decisions around infrastructure, like, we can get more performance or leverages new technology, like, when do we change this? Why do you change it? When are you forced to change it? So I would just love to know, what sticks out to you see, think back over that experience, on the technical side of how you manage through such an interesting time of scale with such a drastically changing tech landscape.

Erik Bernhardsson 6:47
Yeah, I think the most obvious thing is slightly to see like, the cloud is different. When we built Spotify, it was all on PrEP, I think we’ve wrapped like capacity in datacenter. And like, put it on racks there. But the stakes are that it’s now, like almost no one else. That’s probably the biggest thing, but also, look at the data side, like I found, it was like I do because people were inspired by what Google is doing and MapReduce, and there really wasn’t anything that scale and that data that that’s what BI generated with was relatively large for its time, within, didn’t want to go to like the Oracle route. And so, so Hadoop was the way to go. I don’t know, I was kind of starch ages. I look back at that. And I do not miss it. It was pretty terrible in many ways. And I think, I think it was it being like, sudden new tour, like there’s like a certain amount of like, normalization of bad things that I didn’t realize until much later, like how crappy was. And so, in terms of like, the technology, I wouldn’t say there was a particular day I woke up and felt worried midsi landing in the future, but it was like, every day, it was like, a little bit new. Yeah. But so. So staying up to date and up to date on that was like, it’s a little bit tricky, but I don’t know, like, I’ve been doing it so long that I feel like, if you look at like, one git GitHub repo every day, like I can sort of, it’s just you have to dictums going on. But the climb was always a big thing. And I’m very grateful that the cloud exists. And so I think that’s made a huge difference in terms of how we operate with data.

Eric Dodds 8:36
Yeah, for sure. So you leave Spotify in 2015. Did that influence the way that you did things? Were there any sort of any big lessons that you check into better based on that experience?

Erik Bernhardsson 8:57
I think so. I spent 12-13 years or something like that, consumer companies that doing data stuff. One thing I learned is how incredibly important it is, as a consumer complaints. It’s like really firm understanding of what your users are doing on the side and conversion rates and the friction points, and the onboarding flow, and everything that happens in that and like, what’s your activation and retention and churn all those things. And so, I think just coming into my second company, thinking a lot about those things, like from scratch, I think was incredibly helpful. I mean, if there was many other things, like sounds done on the data side, like just kind of assets like, I mean, the other thing is also like maybe like a counter learning. It’s like, I also learned, like, the fact that you can’t really do data only better, like, we didn’t have a DataSift for like, two, three years. Like, you don’t have any users and you don’t have any data than it does. didn’t make any sense. So I think there was also kind of clearance blog. I initially joined Spotify in 2008 to do music recommendations, like he kind of realized kind of quickly, like, we had much bigger problems to solve. And we also didn’t have enough data to do. So that was I ended up like really not where I ended up working on a lot of other stuff for the first few years, and go back to it and make something similar happened better. Like we didn’t really have a date at the first few years since, like, very basic stuff, but then eventually did become like, incredibly important. And then I think then sort of a transformation. I’ve seen a lot of the Dwolla complex too, is how do you start to think in a data-driven way? Like, I think it’s more than just like a technology shift ends up being kind of a cultural shifts too.

Eric Dodds 10:40
Yeah. Super interesting. One specific question there, just out of pure curiosity: What was the point at which you felt like there was enough data to really make meaningful progress on the recommendation side?

Erik Bernhardsson 11:00
Yeah, right. It’s like, so I mean, specific without music. So like, just like the promise working on was like, so unique in a way, like, I don’t know, if there’s like a general lesson to learn. They’re like, we had a very big matrix, like, the rows were users and columns were tracks, and we just wanted enough entries in that matrix. Yeah, in order to feel like, we could complete the rest of the matrix. So so so that probably took a couple of years. I mean, like, early on, like, we only had like, basic stuff that worked. But it didn’t cover a large part of the catalog, like you don’t want to cover up. We should only make it good enough for you.

Eric Dodds 11:38
Yeah, it’s like least common denominator, right?

Erik Bernhardsson 11:40
Yeah. So getting enough data about it would extend to like the full tail of the catalog. I think that took many, many years. So for a long time, the recreation system only covered maybe to talk 100,000 tracks, and then eventually, over 2 million tracks, and eventually, the tracks like that took a long time.

Eric Dodds 11:58
Yeah, super interesting. And then how about on the people side? It sounds like you were sort of on a very small or maybe even like a one-person engineering team, or a couple people to I would guess managing or like, overseeing hundreds of hundreds of engineers was brought to you from that experience of like because the technology side is one thing. But the people side is another one, which is arguably a lot more difficult in many ways.

Erik Bernhardsson 12:30
Yeah, yeah, people are tricky. Yeah. Spotify was weird. Spotify was just kind of almost like atrophy. In retrospect, I think it was kind of a unique environment, that sense that sort of told me that, like, if you just hired a bunch of relatively strong people and just throw them into a room and tell them to go build an amazing service. You can actually sort of make that work if you have the right culture. So I think, to me, that was a very positive experience that I had with me from Spotify, and the amount of trust in the amount of self-organization that new early days, Spotify that we had, which also have negative sides, like no one told me my dad is or was the first three years like, I mean, no, like, I didn’t have any one on one or whatever, but like, and like, No one told me what to work on either, like, so I was just like, sitting wow, like, I started building stuff and like, eventually out some stuff that’s useful that I just, like, Hey, I just like data, like people got interested in since I started working on so much like, random stuff, but But I think, I think the sort of complete anarchy that existed at Spotify, which might seem scary, actually, very positive thing. In the early days, I don’t think it like scale, we all like maybe not 2030. Engineers. Yeah, if we can post more structure. So whenever I sort of fundamentally like a believer in people and trusting people, and so I took that with me, what, when I built that segment, better Redshift, skeleton, junior engineers, that sort of aspirationally, you can get pretty close to culture, where people just come in every morning and just ask yourself, like, what is the number one thing I should do today, that adds the most business value, right, like, helping people have that inner voice like that sort of guiding star of today, the sort of things that matter, and let’s figure out together how to get there, I have no need, like, I think there’s always going to need, we’re going to need managers and like performance reviews, and all this stuff, too. But I think it’s like as like a thought experiment, like, how close can you get to that fully self-organized, platonic ideal of an organization. And I think the truth is, you can get pretty close to that. It can instill that culture into people.

Eric Dodds 14:48
Can I ask a specific question about Spotify in the early days because when you use the term anarchy to describe company culture, that’s pretty shocking. But I think for a lot of people, but it’s so compelling that there was self-organization, what do you think enabled that? Because the thing that comes to mind is like, okay, no one told you what to build. But was the mission, like, even if it was very broad and even, like, somewhat difficult to translate down into, like, what code should I write today? Was the mission like, clearly and consistently communicated so that people could sort of self-organize and at least prioritize? Like, what were the unique characteristics that you think enabled that and because a lot of times you think about, like, aligning around a mission, even with a complete lack of structure, at least people are building towards the same direction.

Erik Bernhardsson 15:46
No one sat down and wrote a bunch of like, cultural tenants and put it up on the wall, but I feel like it almost like kind of ended up being that way. Because I think Spotify just had such a unique sort of product. In fact, we had product and like, a lot, like, and I think that was, looking back, and this is my first job, at a school so like, I don’t think I realized, like, how unique that was, in a sense, but like how cool. I had everyone who worked at Spotify had this love for their own product. I mean, like, frankly, like, I don’t know, any other product, the employees of that company would use their own product. Like, I don’t know, like maybe you’re, you work at IKEA, like maybe you’re sitting on IKEA sheriff’s all day, you’re like, okay, but like I would listen to Spotify 12 hours a day while I was working at Spotify. If you’re using your product 12 hours a day. I think there’s so much care.

Eric Dodds 16:47
That is fascinating. Yeah, that’s like a visceral personal experience, but it’s actually shared among a small group of people that can sort of direct work. Like, through a high level of care. That is so fascinating. That is such an interesting dynamic.

Erik Bernhardsson 17:08
Yeah, I met and I think that factor was like, tremendously help. And I’m not saying like, it’s necessary, like, I think, I think it’s definitely harder. If you’re working on autunno claims processing software for like by corporate’s liability actually insurance, then it’s probably a little bit harder. But I think there’s still ways to create a little bit of that culture of people feeling like they have empowerment and autonomy, all those things.

Eric Dodds 17:41
So, yeah, absolutely. So fascinating. So interesting. Okay, one more question from your background, that I just can’t help but ask. Coding competitions. Is there like one coding competition that sticks out in your memory from doing those in the 90s are like one challenge?

Erik Bernhardsson 17:58
Oh, come on. Like, we’re only 2000s idling? Don’t know, I think. Yeah, I don’t know. Like, I, there’s like a lot of that, but there was like a crazy one we’d go to like Hungary every year. And there’s like this 24-hour programming competition. That was like really different. Now, it was actually kind of fun. Because you have all this weird programming competitions. Like, this is like problems like, you have to control. I don’t know what the Lego train or something like that like that. It was a lot of fun. Plus, there are like 24 hours. It’s kind of fun to manage your own energy.

Eric Dodds 18:31
Sure. That’s like a physical challenge, too.

Erik Bernhardsson 18:34
Yeah, yeah. Yeah. At the end of the neck, early hours of like this, the second day or so out of it sits down, though, when used to be like, fun every year.

Eric Dodds 18:46
Yeah, super interesting. All right. Well, give us just a brief overview of what Modal is. And then I want to hand the mic off to Kostas because I know he has a bunch of questions about Modal but Yeah, can you just give us a brief overview of what it is?

Erik Bernhardsson 19:02
Yes, it is a way to run code in the cloud in a way that purpose focus on the ethics, it’s very deliberate contextualized I think it’s actually, sorry, this may not be quite as predictive since I always wanted to build better tools for data engineers, like our data scientists, like I don’t have a lot of different things as a CTO, so it’s very clear that I think data teams need better tools, right? They’re sort of behind I think other segments of software engineering. So now I looked at a lot in different parts of the stack, like I looked a lot of orchestration to begin with I as a mission, I’m the author of an open-source tool, the Legion, which nobody leaves yesterday, but like 10 years ago, some people use some sort of kind of a precursor, like kind of before Airflow is like sort of a simple Yes, sir. So I started working on that. I was like, this is really cool, like an orchestra because it’s the middle it controls a lot of the other stuff, what if you start burying because then will become this like nexus for all the other stuff. They don’t realize like at some point like Cool, like, you can orchestrate the code, but you have to run it somewhere. And like, where do you run it? And I started thinking, Okay, after build this integration layer with like, with like Kubernetes, or Docker or like Lambda or like, he’s not just like turned out about Braintree sites Blendo realized I have found that is like how hard it is, like, fundamentally like, the user experience of building an orchestrator will never be better than the sort of beauty experiences of lightning, the integration with the underlying substrate where you actually run code. Yep, sounds like water is one of those, like, fixed problems that and so I started focusing on this idea that like, what it’s like, throw out all the good stuff, right? Like, you look at data teams and how they run code today, like, there’s like, Minetta is Docker Terraform, helm like all this stuff, Airflow, and they’re like, what, what if you started from scratch, and like, built it in a truly combative way. And we’re like, you don’t have to think about resources, you don’t have to think about like setting up instances, installing infrastructure, it just like runs in the cloud for you can also up and down, you can schedule things like, Well, what that looks like. And so that’s what I started working on two years ago. And yet we have something that works reasonably well. But today, we’re still in sort of close to failure testing stage right now.

Eric Dodds 21:10
Kostas, all yours.

Kostas Pardalis 21:12
Thank you, Eric. Thank you. So, Erik, you mentioned theory, dude. So let’s start from that. Because it’s, you said, like, you have a bunch of like moving tools. We’re like, data engineers. And Luigi is probably one of the first Hadoop builds maybe, or at least have, yep, they’re wanting to go to Open Source. Right? So tell us a little bit more about military. And I’m not talking about multiple, like, the technology itself, but how did you get into building it?

Erik Bernhardsson 21:44
Yeah, so just kind of a recap of religious it’s like, it’s an orchestrator, aka workflow scheduler sort of saying, categories primarily are flow. Now there’s also there’s DAX, or creed, bank and flight bills. But when I started working on Spotify, there really wasn’t anything in that space. Right. And so I ended up having more and more complex data bike loss for the music recommendation system, and like lot in particular, a lot of Hadoop jobs, like seven-hour series of complicated, hundreds to do jobs that I had to chain together, and like run in a particular order. And I ended up like, realizing at some point, like, actually, this is a grab, we should model it as such. And because I’m an old-school person to be quoted below, my first thought was make this makes up. So I started looking at make file because I like the idea of just like, functional nature of like, how makefile works, through the mind, like targets and rules. I mean, they seem Texas may cause incredibly arcane, and super annoying. Yep, it’s like, dollar percentage is like, the exit code of the previous, whatever. It’s just like a certain because it was built in the 70s 70s. So anyway, so as I started, but like, I went to sort of idea but so I started building, Luigi, sort of loosely model night, and then kind of evolved, like, which was actually the third iteration of, of the same sort of idea. And the first one where I felt like this is good enough for I’m just gonna throw it out there. I’ll get up and see if anyone uses it. And I guess other people had similar problems, after just a few months, someone Foursquare reached out, and they’re like, Hey, we’re running lychee, we’d love to collaborate. And then from there, over the next few years, there were many more companies that start using it more and more. I never, like, thought Luigi was like, amazing. It’s kind of funny anyway, but to me, it was just like a thing that solve a particular problem. That was like good enough, where I felt like other people would also have the same problem. And I think they had to say, I think there were a couple of things that I’m really happy with leisure, like it’s sort of had this like functional nature of like I’ll use stress the dad. But the other states also don’t like about it in particular, I think the reason why people like Airflow LoRa was like Adam much better web interface. And most of all, is stuff that people actually want it. So I don’t know. It’s kind of a fun day. I mean, people still use it, but it’s kind of rare.

Kostas Pardalis 24:19
Yeah, yeah. So what’s like the current state of the project? Why is loser still above that? Like, you said, they like on it.

Erik Bernhardsson 24:26
It’s kind of dead, I need I don’t know, like, there aren’t as a mission, like the RV companies, like still using it. I haven’t touched it for six years. But like, I think Spotify still has like, a lot of stuff in G I heard David Ortiz which suddenly ended there are some cool places solo you said, right? Yeah, gotta like picks those in a bit too. It’s a very sticky product straight and hard to get out of, at least like we’re cool. Souter wants to use hymns. You don’t need to rewrite the poem. But yeah, it’s not been like actively maintained. I think for good reason. I think they need I mean, I almost like thought about it. Like, what if I’d actually like to kept working. Let’s see, what would it look like today? That’s actually kind of like when I started working on, I mentioned like, work straighter, like the core model. And I think there are many things he could have done, like so much better. And I still kind of don’t see her like today’s workflow scheduled at it. But yeah, just a question like, which is not under active right now.

Kostas Pardalis 25:25
Okay, so how does like orchestrator, like fits in like, the wards of today with like, the serverless cloud infrastructure that you have in mind when you’re building Modal?

Erik Bernhardsson 25:38
Yeah, I mean, what it was like, kind of the layer below, right, like, as a mission, like I started thinking about our groceries, and there was actually like, I don’t want to build a website, I want to build a layer below, right, like, because that ending is much bigger profit. And so I think it was actually it took quite a bit laters and we have a couple of people, we use modal together with orchestrator, like Airflow protector, Dexter. And those, by the way like orchestrator, like I feel like in a way, like there’s someone agnostic as to what like where the job actually rondels like, typically people run it up for bid edits, or, or something else or, but you can certainly integrate the surplus like I, I feel like in general thinking about where the cloud is going. And then it is a function of having seen this kind of firsthand like Volker like the old days of like, on-prem to like, now, like the cloud, I think we’re still kind of early in that cloud, the evolution of the cloud. And think, to me, the natural sort of inevitability of the cloud is to some extent, serverless, like, why, as an engineer start to think about these, like these abstractions, like clusters and instances, and its resources and like, get all this other stuff, right, like, computers, just do it for me so much better at this. So those types of things. And so I’m extremely bullish on serverless for that reason, and I especially in data land, that might actually be a better fit for serverless. Right, it’s, yeah, there’s a lot of like assets, serverless serverless, software, I think, has been like, mostly prevalent in front and back end type architectures. And you see a couple of vendors, like, all into, like, sort of, for salad, or like, NumPy, whatever, like doing that kind of stuff. But I think like, the sort of nature of what developers in data teams do, might actually benefit, you have just like very, like bursty workloads, you have, like esoteric, sort of, or heterogeneous hardware requirements, and you have sort of like, very, like exploratory things that require you to build things quickly. And try it out and stuff like that. So that idea that was started with Modal, one like—I don’t know if I’m answering your question—I ended up diverging a little bit, but think about one is this.

Kostas Pardalis 27:51
Okay, so like, if I start, like working with Modal today, like, what’s my experience will be like, Well, how do I interact with Modal? How is this different from the cloud or yesterday?

Erik Bernhardsson 28:05
Yeah, I think, in particular, the biggest difference is the speed at which you can work with infrastructure. So traditionally, infrastructure was always this, like, annoying seeing your ex do it? Or, like me, like, I think this is still very much like, it’s just like, you write code, you run it locally, as you’re developing code, and sort of writing it locally, like kind of test running it and like, eventually, you’re like, Okay, great. I’m gonna, like, deploy this to production side. And then you’re like, Oh, my God, Abdullah, container, Arsis have to, like, write what’s a Yamo? After like, whatever, they will ask the ops team to do this permission to, we have a being disliked this inordinate amount of like, sure, like, just like, oh, it’s like, super annoying process. And so, one of the things that I fundamentally believe in is what I used to think that the only way to like work with the cloud is like, you sort of have to like you to walk all the things like globally that you don’t, but then actually realize super, like, what have you actually bring in excited, like, once much early in that process, right? Like, what did he do, like, always run things on the cloud, as you’re writing code locally? What have you like, when you run it, it actually runs in the cloud, in that same environment that you’re gonna— I’ve actually deployed it to and so that’s what Modal does. And the only way to do that is to make it super, super fast and fast, not in the sense of like running and fast because there are upper limits to how best computers are that we can’t do much about. But it works. However, what we can do is we can start containers super fast. We can take code locally, like you’re writing a script just wanted to run it. And so what we can do is we take that code, stick index data in the cloud, we launch it in less than a second. And we launched 100 of those containers in less than a second and immediately just like Print to standard out, like whatever was happening, like you see it, and like you don’t have this, like annoying, you have to build a container and push it to ECR dangled triggered in this manner of days and Angola download the logs, whatever, like, Yeah, you don’t have this, like super long feedback loops, right. Like, when I think about developer productivity, to me, I think developer activities, like, best understood in terms of these, like feedback loops, so you have to make these feedback loops fast, in order to developers to feel happy and to feel productive, is the moral justice with the cloud, like you go from like, writing code locally, to actually executing it in the cloud in less than a second. Which is something we have to go very deep and like, kind of in the guts of containers and file systems and that kind of stuff to publish.

Kostas Pardalis 30:47
Okay, we’ll talk about that. But before that, you’re passionate about buildings, products, technologies, like that’s tools that’s going to be used by data engineers make data, let’s say, professionals, how is Modal used for that by data integrity? What is a data engineering team going to use Modal for today?

Erik Bernhardsson 31:15
Yeah. Yeah. Yeah. And like, yes. So what I think is interesting is, over the last 20 years—or whatever, 15 years—there’s been all this like, back and forth between like, yeah, of course, we’re just gonna do and then their warehouses came. And they said, like, no, no, SQL is like very dominant for a lot of workloads. I love SQL. But I think fundamentally, like my belief that there’s always going to be stuff you have to write code for, right? So SQL covers, like a very large, percentage of work workloads, it’s all stuff you’re gonna have to run record for, like you don’t do any sharding, or do simulations, or whatever. So that types of some model exclusively focuses on everything right now goals are Python, right? Like, eventually, we’ll probably have support for other languages, too. But the benefit of focusing on data teams specifically, it’s like, they’re almost always use Python. Yeah. So what are the problems that Modal primarily helps with, like, right now we’re targeting what I call what I think of as, like, embarrassingly parallel workflows. So those could be things like, I have 100 million images, and I need to compute embeddings for those, and I have this function that runs on a GPU that takes an image and produces vector and I just want to scale it running, like one disease, or I have 100 million satellite images, I want to run some computer vision. Or there could also be things like, I need to scrape professional websites using a headless browser, chromium, or playwright or whatever, and take screenshots or like you’re not, stuff like that, it could be financial backtesting, can be very tough simulations, multi Carlo simulations, like either pricing National Instruments or, I don’t know, like sampling procedure, things like that. So, we focus a lot on these two types of sort of, embarrassingly parallel use cases as a sort of starting point, but, and then another, slightly smaller bucket of things tend to be just like simple cron jobs. People have like tiny scripts, and they just want to run it every hour, like every midnight or something, I don’t know, it’s not in smaller pocket. But I think there’s a lot of super early-stage tech teams, and like, they just have like a tiny thing, doesn’t even have to be a tech team, there’s gonna run something as scheduled.

Kostas Pardalis 31:28
And have those like model, the first convert to something like AWS, Lambda, for example.

Erik Bernhardsson 33:31
The architecture is very similar in the sense that we’re all about, like, running containers in a way where you don’t have to think about the underlying infrastructure. I think the difference is primarily and I thought a lot about this, like, why has serverless not been more successful? myocarditis, I frankly, didn’t, that developer experience salts, like none of them know, I don’t know, if you like production as like Lambda functions, but it’s like, pretty bad experience. Like you have to sit up, like cloud watch, or like, whatever sky and remember, like, there’s so much stuff you have to do. And you don’t have to, like back over you just like run code. It’s like we’re saying cloud. So big part of what Modal does is really just offers a much better user experience. And I think because the latency between like writing code and washing containers is so critical to that developer experience. We basically have to start over and build their own infrastructure for that.

Kostas Pardalis 34:33
Mm-hmm. Okay. And while you were going through like the use cases that you have seen, like in models so far, you took like a lot about late, let’s say, working libraries, unstructured binary data. What about tabular data? I mean, the use of land use cases also there or this is not, let’s say…

Erik Bernhardsson 34:56
Yeah, I think there’s all this stuff that I like to get out. There’s always stuff that you can, if you can do things in pure SQL, I think you should do it. But if like, let’s say you have like a lot of like tabular data, and you want to fit, and that’s you boost on top of that might be hard to do and on database. I don’t know. I know there are some crazy Postgres extensions, whatever to do that today. But like, typically, you want to write code that you want to run it, like a containerized environment. And so I think that’d be like one example. We can take that data, operate on it, and like fit models, and then like, help you deploy those models because you can also take functions and model and deploy them as sort of either REST endpoints or foxes, cues and tartelette, from other apps inside of English translation. So you could also use this kind of a model, that model serving interface. But yeah, and so those are some of the things we can do with model and tabular data that you probably don’t want to use the MS forum. But again, to go back, I think there’s like plenty of software databases, amazingly, definitely start with database.

Kostas Pardalis 36:01
Yeah, it makes a lot of sense. And there’s like, another leg huge conversation about what the serverless database looks like, which is another interesting topic words, let’s look at Modal, unlike the architecture of Modal. What it takes to build a platform like Modal, like the serverless clouds, infrastructure service.

Erik Bernhardsson 36:25
Yeah, so I started looking at this a couple years ago, and immediately like, kind of run into these problems, like building Docker containers, and like pushing Docker containers around, it was kind of slow. And so how to fix that. And so so one of the things that was pretty easy, pretty early on decided I’m not going to be able to stop this. And so we ended up using a much lower level, primitive Romsey and I mentioned the Deaf switch to Parquet, Docker and trunking tears, which he had using that and you can spin up containers in, like, milliseconds, or less than five minutes, like, Okay, now you have this like, big container, like container images, like how did you? How did you like shipped around these like big container images. So like, you don’t want to like awkwardness, like sort of tradition like Docker method of like, okay, we’re just gonna push and pull it all image, although it does a little bit of deduplication all of their levels, slim and festive sometimes. But like, if you have a container image that’s like, 10 gigabytes large, you really want to avoid, like sending that back and forth over network between different nodes in your cluster. So, I basically realized, Okay, well, if you look for containers, sorry, to skinny by the textbook, by the way, so feel free to tell me to shut up, it’s like too low level, I realized, like, if you look at container images, first of all, most of the container images are like, never read the average, like container image, like, it’s really good, like a bare bones like Linux distribution, it’s like, all this like random, like timezone information from for, like, Uzbekistan, and like, random islands, like, no one lives off locale information and like Bash and stuff, right, like, so, why in central states in the first case, the other thing is, like, even 14 from for the files on the Linux, or on that image that are actually read, most of them tend to be the same, it tends to be the Python charter, a bunch of Python standard library, but the top is very, when you watch a documentary, like It read, like the same few 100 files, roughly. That doesn’t differ at watch across the difference across different Python versions. But so what we ended up doing was, like, we ended up building our own file system, that this is source all these container images, it I call it an address where we go through the old image, you compute checksums, for every image for every file in it, and then we release, you store those files literally once in an underlying network storage. And then we like, kind of create this like virtual file system and fuse, we ended up building a Ross for performance, that then exposes that to the container runtime as like an linnets route. And I actually worked super well, like everyone told me like, You’re crazy, we’re building and filesystem but we ended up doing it, it actually works really well. So we have to build a bunch of other stuff to like sort of wallet slides. But that’s kind of one of the things that we ended up building I think very technically challenging and kind of complex, but I think ultimately like also now lets us deliver this experience to consumers that I always wanted to just write code and it’s like immediately starts in the cloud.

Kostas Pardalis 39:32
So if I understand correctly, you are using right in terms of like the technologies that have been used like as part of like the models that you can use firecracker, right?

Erik Bernhardsson 39:42
Not yet. We were planning to move to Parquet has. This is what GPU is delicious. So rapid, that’s the plan. Okay, so what are you using now instead of like firecracker, we use, use run See, just like on lower level primitives, Docker actually use LCM to Algren just like a simpler. If you look at like, what is a Linux containers, like basically like ch truths and like set cough and like, cgroups and especially like those types of things. And that’s essentially what like Romsey gos, it’s like 1000 line, go binary or go script binary or program that the basics wrap those things into kind of unified OCI image compatible, roughly?

Kostas Pardalis 40:25
Yeah. And then you use like fuse to create, like the file system that helps you like, store your the image there. Correct?

Erik Bernhardsson 40:35
Yeah, yeah. And actually, I’ve a few. It makes it pretty easy to build false systems, we actually ended up in, like, the first prototype we built in Python, because there’s like Python bindings for cuse, which is like, terrible from a performance point of view. But it’s actually kind of nice. Like, because it was very simple to experiment with. Even secure what’s really nice.

Kostas Pardalis 40:54
And where do you run but stops like you’re using like bare metal servers locally? You have, like, on the clouds?

Erik Bernhardsson 41:00
Yeah, yeah, exactly. So we run it on this bare metal is two instances. And then we maintain that pool of workers, and we start and stop instances, as we need to need more resources. So from a user’s point of view, they never have to think about, like, we do that.

Kostas Pardalis 41:22
Yep. And, okay, the users do not have to think about the stuff but you have to do it for them. Right. So yeah, it was like, UPS rates of like SATs, the product and infrastructure, how it looks like how likes an SRE you’re like, well, they were old duyvil you have like in Modal look like?

Erik Bernhardsson 41:45
Alright, this might be like a controversial fit in but like, actually don’t think you should have like, separate off steam, like early on, I think you should have like people who are like maybe like more like, interested in it, who can like get on the thought board with it. But early on, like, actually really think you align incentives really nicely if everyone’s part of the firefighting at all times. It’s something breaks. Anyone that that team just like jumps in, ideally, the person you’re like, we wrote the code, we’re like, understand that part of the code to die. We’re early. We’re six people. So we don’t have like a dedicated offseason. Eventually, we’ll have but not right now.

Kostas Pardalis 42:21
Yep, yep. No, I totally understand that in life. But something my experience Lego building maker teams. So how it is, like, let’s say, different than what’s let’s say someone was doing operations, who might a company that’s relying on Kubernetes. And like other kinds of primitives, like for the infrastructure that has like different the operations between like, modeled in like a compound that is more cloud traditional, let’s say.

Erik Bernhardsson 42:49
You’re never going to have to write a single line of YAML. That’s the biggest difference.

Kostas Pardalis 42:53
I think you’re going to do like a very good job in like hiring released the podcast episode. But you just did. Yeah, no, I think we have to write all the YAML wants, and so that our users never have to write yell.

Erik Bernhardsson 43:13
Yeah, ideally you don’t have to even think about like, I think in your mind, I won’t even have to think about data engineering. Like, this might also be a controversial opinion, but like, I almost wonder if like, long term, like data engineering, as it is today, like, will go away, because I look at, like, all these startups, every tech company in the world, and so building their own internal data, popcorn, right, and yeah, they all kind of look the same. It’s like, you start out with, like, Kubernetes, and brush the stock, and then like, someone bumps in obstruction to make it easier to launch internal, like machine learning models, or change things in notebooks or whatever. And that, but like, what I’ve liked, someone just built that and then sold that as a service. I kind of thought would be a better place instead of 10,000 companies building it themselves. And so that’s sort of where, like logically where I see the world wall into this, a lot of mortal states should really just be like services that you use in the cloud. So you don’t even need to have a cell if that phone takes internally that does this.

Kostas Pardalis 44:22
Yeah, makes total sense. Cool. All right. I have like a question. While you were like, dodging and you were mentioning, like, the Docker images there. And like how big these images are in general and like, my opinion is that especially when it comes to infrastructure, there are still like a lot of primitives around that they just feel like not the right primitives yet. Well, like young dudes are late. Sometimes like what comes to my mind is that because we are probably like, middle farm in terms of age, but during I remember like the biking like led zeros when you have like to download the stupid like super bloated installers like to install something like this really bad, like experience was why I need this thing like, sometimes I get like the same kind of like the link but only the frustration level with SOAP isn’t like it’s much more complicated, right? But we’d have like Vidya said like, okay timezones forms Bidston, like, where do I need that like to run like a serverless Lambda function, but Docker will like, yeah, n plus one equals two. Right? So what does what’s like, what kind of evolution did you expect to happen? And what aren’t you like, expecting to see in the industry? Like, as new primitives that like, we can better infrastructure at the end?

Erik Bernhardsson 45:48
Yeah, I mean, I think I feel like there are a lot of like, tools that tried to wrap underlying layers, but they end up being kind of leaky abstractions in a way where, like, it doesn’t prevent you from actually having to learn about those underlying layers anyway. And now you’re almost I kind of made the problem worse, because now there’s like, another layer, they have to learn, right? Maybe like a good abstraction, like, if you build like an amazing data platform tool that wraps Kubernetes, and then like, ideally, if you use that tool, you should never have to learn a single thing about Kubernetes or Docker, like, whatever, but laying on those tools really work that way. And again, I feel like there was like, ton of leaks through in the end, right. So that, to me, that’s ah, I don’t know, like, my theory is like, the first generation tools, they’re all about, like, being able you to do something like that they sold like a heart probably, like a part technical problem, in a way that like, there was no tool before that saw. And, like, people then have to use those tools. By necessity. I think what ends up happening is the second generation then actually, like, kind of preservative functionality, but like, rewrite all the like abstraction, and actually presented and what better way, we’re, like, fundamentally, the technical, the enablement is like, the same, but you no longer have to, like, jump through all these hoops to, get it and stop. Like, instead was, like, I don’t know, like, I am looking at machine learning, back here, I started doing deep learning, like, 2014, those days, and it was like, incredibly hard back then, like, you had to like, build built, install tiano, install a bunch of like, random CUDA drivers, or whatever. Yeah, I guess she kind of still very much a problem. But to some extent, like now, you can also go to a puking face, like, download a model. And now you suddenly have a model, you can actually do like, very cool things. But no, I think the UK, it’s just, like, similar story to be told that like many different meals, like, sort of package stinks in a way that suddenly becomes a lot more accessible. And yeah, it makes a lot easier and a lot more fun to use.

Kostas Pardalis 48:04
Yeah, yeah. Yeah, I totally agree. All right. Last question for me, and then we’ll hand the microphone to Eric again. Dutch. So what kind of like opportunities you see out there, like business opportunities in light. Building new experiences, like over like, clouds, you mentioned at some point that we are still like, very early on? Yeah, in the clouds. So obviously, you’re believing that that’s why he builds Modal, but what else is out there? In your opinion, interesting problems, that’s can be also like, interesting business opportunities.

Erik Bernhardsson 48:45
Yeah, I mean, like, here’s, like, one thing I’ve been like, ranting about all on Twitter is CI, like, I see eyes like this, like super janky experience today, right in like, I want someone else to build it. But I’m just like ranting about it on Twitter. It’s like, no one built it in the next year, like, I went to myself because, like, I have no idea. You can even like us Modal some of it. But I don’t ideally, I don’t have to do that. But what’s like crazy about the eyes, first of all, I don’t know us like GitHub actions or whatever, like then you get a doctor I actually think it’s like a really cool product in itself. And I think you’d like sort of its same story of like, and enabled me to do things a new way. But today when I do get a vaccine, it’s like, I was like, yeah, no stop and then something breaks and then like, I have this like super slow feedback cycles where I have to like commit something to get and then like, wait like five minutes and then like see in the look so you have this like slow feedback loops, which is like the most like torture for lay into developers like the bugging problems with slow feedback loops. And then you think about all this extreme amount of like, resource like wasted resources, like you have 10 containers, each pulling down the same libraries over and over again and installing them like if IBM like, and I also think a lot about this, do unit tests, like they’re the ultimate like parallelizable thing, like, why can’t I just like, let’s say have this, like large in a project, and I have 1,000 unit tests, like, can you just like stitch every unit test in a Lambda function and just like run all of them at the same time because you have a human sitting there waiting for a test that’s very caught, expensive time. And also, another thing to think about is like, you have this like, super annoying lack of parity between CI environment and local testing, it being about that point of like, Why do developers even run things locally? It’s because it’s the IAM moment sauce. But if, if the CI environment was like, so good, that like, the experience was as good as writing locally, like, you wouldn’t even run the tests locally, you would like to develop code, and just like launched 1,000 tests to make sure running it, they’re all Lambda environment, then the container, whatever. And then you would immediately just like, see the failing tests? What if you can build that? I think that’d be like fantastic world. I was six years, there’s very few things. Where if someone said, “Would you spend $100,000 making your engineers not have to wait for CI?” I’d be like, take my money. So I think it’s a huge opportunity to do this. But if no one else does it in next year… Maybe I’ll have to do it myself. I don’t want to.

Kostas Pardalis 51:26
Sounds good. All right, Eric. It’s all yours.

Eric Dodds 51:30
Yes. This has been so fascinating. I’m interested to know, have you built anything, even if it’s just for you, or your small team to like, address some of those CI issues? Or are you still just using like a process that you largely hate?

Erik Bernhardsson 51:46
I generally tend to build things when I don’t like it. But ya know, so far, we vote for CI Golang.

Eric Dodds 51:52
Yeah, super interesting. Okay, well, we’re really close to time here. And so I have a, I have a question that is unrelated to technology at all, that I want to know if you still have any of your Spotify playlists back in 2008? And if so, what music is on them?

Erik Bernhardsson 52:15
I don’t know. I do a lot of playlists. And I’m sure you can, like find, like Google my name and browse around a little bit. I’m just like, the gender at like Detroit techno fan, grown up together, up and like, I lived in Berlin at some point. So my sort of take sense of skirt to this sort of that those sort of styles. But I haven’t been I don’t know, like, I’m actually I grew up listening to music, I listen to everything like jazz, hip hop, and classical music, like whatever, like I, I do like music overall. And that was actually a fantastic experience working at Spotify. I was almost like wondering, like, I built the music recommendation system purely for, like, selfish reasons. I discovered, like, a lot of usage throughout my own, through my own system, which kind of gave me some sort of pleasure.

Eric Dodds 53:09
Yeah, for sure. Yes, that’s very interesting. I mean, as a Spotify user, I can’t remember where I listened to it. But I, I listened to an interview with Daniel, and he was talking about how, at some point, you figured out that if you could recommend something new that a user liked every week, then they would essentially sort of say, was Spotify forever, right? Because you’re sort of providing this like discovery experience. That’s been true. Like, it doesn’t come down to like different things.

Erik Bernhardsson 53:40
That wasn’t me, personally. But it was definitely it was people in my team who came up with that idea of discovery weekly. Edward and Chris, the other people in my team, fantastic idea that I still use every week, it’s a good point.

Eric Dodds 53:57
Yeah, super interesting. All right. Well, Erik, this has been such a great conversation. So much more to cover. So we’d love to have you back on the show.

Erik Bernhardsson 54:05
Anytime, anytime. This is fun. I really appreciate it.

Eric Dodds 54:08
Kostas, this is going to be this may sound like an interesting takeaway. But I remember being so interested, I think I mentioned on the show that I listened to an interview with Daniel, one of the founders of Spotify. And this was a years ago, but for some reason that interview has really stuck with me. And I’m gonna draw connection that Eric didn’t draw, but that I think is interesting. But one thing they talked about in the early days of Spotify was that latency was a really big deal. And so they had to figure out ways to almost like, create a user experience that made users okay with latency because speed was a really big deal when you were trying to deliver a large file like an mp3 over the internet streaming. And it was really interesting to me that he used a lot of very similar language when talking about mobile and the developer experience, right? There’s like a latency challenge. And friction points. And it was fascinating to me to think about the similar nature of those two problems. So yeah, that’s, that’s my big takeaway. I think that’s really interesting.

Kostas Pardalis 55:23
Oh, yeah, that’s like a great point that you’re making here. It’s really interesting, like Collins coordinates the developer experience with, like, the user experience at the end, because yeah, yeah, I think we tend like to consider them as like, very different, but some massage versions of the apple one, they just manifest themselves, like, in a different way. Obviously, like a developer has latency in different things than like someone who listens to music. Sure. But yeah, that’s, that’s that’s an excellent point, actually. I mean, I don’t know I think like every, every part of the conversation with Eric Mills grades. I love like hearing about the staff that they are building and how they build them. And what I will keep is that it’s great to hear that you can have like startups today, but in order like to operate, they have like to create their own file systems. Yeah. That’s like, I think like a great indication of like, the progress that has happened, like in all these years, in terms of like, the first structure that we have, and limited is that we have out there to build upon. And yes, like, people like just shouldn’t be scared to even like, go and build their own file system evade often. So that’s what I keep. And yeah, let’s, hopefully we’re all going to copy bucks from so.

Eric Dodds 56:50
Yeah, I agree. Awesome. Well, thank you for listening to today’s tech show great conversation with Erik Bernhardsson, and we’ll catch you on the next one.

We hope you enjoyed this episode of The Data Stack Show. Be sure to subscribe on your favorite podcast app to get notified about new episodes every week. We’d also love your feedback. You can email me, Eric Dodds, at eric@datastackshow.com. That’s E-R-I-C at datastackshow.com. The show is brought to you by RudderStack, the CDP for developers. Learn how to build a CDP on your data warehouse at RudderStack.com.