End-of-Year Product Trends: The Cost of Rushing Features with The Cynical Data Guy

John Wessel  00:28

Welcome back to The Data Stack Show. We’re here for a very special show, a Christmas edition of the cynical data guy. So I’m here with Matt. Matt, welcome back to the show.

Matt Kelliher-Gibson  00:38

Welcome back here to bring the Festivus to your Christmas cheer. Excellent.

John Wessel  00:42

Yeah. Unfortunately, Eric can’t be with us today, so you’re stuck with myself and Matt. But we’ve got some fun topics we are going to start out with, which is a record for us. By the way, we’re going to start on the show talking about AI. It’s unusual. It’s Matt’s favorite topic. Oh, so much. All right, so one of the things that I’ve seen a lot, Matt, I think you’ve seen it as well, is we keep seeing these various forms that open. Ai calls it the 12 days of ship mess. And there’s basically, there’s several different software companies doing this where they’re attempting to brand, pushing really hard at the end of the year, essentially to get product out. What do you think about that? Matt, it

Matt Kelliher-Gibson  01:29

sounds like someone made promises for the end of the year, and they have not kept up with them at this flame. Then some people that are like crap. We promised investors. We do some things. We haven’t done Eddie,

John Wessel  01:40

yeah, and it’s such, it’s in such a juxtaposition to like the other the what I would be used to, I think what you’re used to, too, is like we’re gonna do like we’re gonna do code freezes so we don’t break anything. We’ll be on vacation and cute, and things are very stable at the New Year. So look, how do you think this is gonna work out?

Matt Kelliher-Gibson  02:00

I mean, I think, I think it partially, just garners attention, is one of the big things. But the unfortunate thing is, once one person does it, and they get attention for it, then everybody is going to try to do it next year, and it just becomes the new standard, and nobody really gets credit for it, right? It just becomes, oh. Now we have to ship 12 features every December. Why are we doing this?

John Wessel  02:27

So here’s an article. So open AI, let’s see their 12 days of ship. Miss, they shipped the oh one reasoning model, which, if you’d like to spend $200 a month on chat, GPT,

Matt Kelliher-Gibson  02:39

you can, I will pass

John Wessel  02:43

What else did they show Apple intelligence was day five with chat GPT.

Matt Kelliher-Gibson  02:47

My iPhone is good. I won’t

John Wessel  02:50

support it. That’s too bad. A couple of other features day four. Oh, and then Sora was the other big one. That’s their can create video, and I still don’t

Matt Kelliher-Gibson  03:01

have a use case for that personal life or professional, yeah, at this point, so good for them. Not something I’m going to be able to take advantage of at times soon. Yeah,

John Wessel  03:11

That’s fair. Okay, so we’ve got the 12 days of service, and Matt and I were talking before the show about and we’ve talked about this topic before, about this founder mode versus, I don’t actually have a good word for the ops. What’s the opposite of founder mode? I think they called it normal. Okay, okay, manager mode. So we’re talking about, like, these things like the 12 days of ship miss. It’s like, well, we don’t want people to, like, slack off at the end of the year, or maybe we’re late on some deadlines, but mainly, we don’t want people to slack off at the end of the year, so we’re gonna set a bunch of deadlines, because it

Matt Kelliher-Gibson  03:47

feels a little bit like the dad who hasn’t been paying attention, and then he sees his kids kind of slacking off or something, and he goes,

Matt Kelliher-Gibson  03:57

that’s it. We’re all buckled down. You guys are going to be locked in your rooms. Never coming out to you getting a or something.

Matt Kelliher-Gibson  04:06

It had a little bit of feeling, like detached, and then you came back and you’re over corrected, yeah. Well,

John Wessel  04:13

it actually reminds me I had this, this guy I worked for years and years ago, and he would intentionally it was a, it was one of these things where we, you know, pretty much a 24/7 operation, but he would intentionally come in, like, Christmas Day, New Year’s Eve, New Year’s Day, and just to, like, you know, check and see who’s working, type of thing. And I, you know, I don’t know, this kind of reminds me of that, of like, All right, let’s make sure. Let’s make sure we get everybody putting in the extra hours. Yeah.

Matt Kelliher-Gibson  04:40

The only other thing with this is it kind of risks eliminating one of these unknown, like, not unknown, but under the radar things that kind of keep a lot of these places going, which is when everything kind of slows down. Suddenly you can sneak in there between Christmas and New. Years and all that technical debt, you can actually make a dent in it, right? Nobody was letting you touch right? We’re going to miss 12 days of shit miss. When is anyone gonna secretly do all the tech debt remediation? I mean, you never, but you’re at least gonna pay it down a little bit. Yeah? Now we’re just throwing everything on the credit card. Yeah? Till we’re next year. Just gonna keep throwing on there. We gotta pay it down, yeah? I just let the interest keep rolling.

John Wessel  05:25

Yeah, no. I mean, that was something that I actually scheduled. I’d never been on a team that did this, but we actually scheduled quarterly, like, cleaning stuff up, paycheck down, stuff. It was like a weak quarter or something like that. We do, like, security reviews that, like, nobody got around to. We’d clean stuff up. We’d actually, like, you know, pay down some debt or or just, you know, review stuff, stop, you know, for a minute. And I don’t think many teams do that. Yeah,

Matt Kelliher-Gibson  05:55

That’s unique being able to do that. Yeah, most of the time you get very hung up on it but I have another data request. Why can you just do that? Right?

John Wessel  06:05

For sure? All right, so it wouldn’t be a good episode without some LinkedIn post, right?

Matt Kelliher-Gibson  06:10

Of course, that is what I’m very happy about for this past all the LinkedIn posts that we get to, then bring on here. All right,

John Wessel  06:19

big Matt Scott, one for us here, maybe even a couple.

Matt Kelliher-Gibson  06:23

So this one was out there, and it’s a very short one. It just says data engineers can do what analysts do. Analysts can’t do what data engineers do. Some fighting word, yeah,

John Wessel  06:40

man, don’t tell me about some of the comments. First, that does not sound like a dangerous place to be in that comment section. DAG

Matt Kelliher-Gibson  06:48

got some fiery comments, though. My favorite one was a person who was not in the thread, but separately, just thrown. A data analyst can’t do what a data engineer can do because it’s boring.

John Wessel  07:02

Data a data analyst can do what a data engineer can do because it’s boring.

Matt Kelliher-Gibson  07:09

Well, honestly, I think if you are an analyst, a lot of times you look at what they’re doing. As a data engineer, I never want to

John Wessel  07:16

do that. Yeah, it’s just the plumbing, like, that’s boring, yeah?

Matt Kelliher-Gibson  07:19

I mean, it’s different personalities, for it usually, sure, but I mean, as a former data analyst, then those are fighting words, right there.

John Wessel  07:27

Same, yeah, I spent a number of years as a data analyst, and I also here’s, actually here’s a really interesting thing, depending on the company I worked for. I’d be curious if this was true for you, the value of a data analyst was drastically different than the engineering so, like, I remember one of the first companies I worked for, the analyst, I would say, kind of like in the middle, and then actually the people above analysts tended to be project managers, okay, which is totally unique. Another company that I worked for as an analyst was kind of a lower tier thing. And then there was, like, kind of the traditional IT heart RP and the analyst were kind of lower. And then a third company that I worked for, there was, it was just a bigger company, so it was, like, more split out, and there were kind of like levels of analysts versus intelligence, you know. So it really, I think it really depends on the culture, because there is that, like, there’s some cultures where it’s like, Well, these guys actually, like, drive the business for they have the business knowledge. Like, we value them the most. Like, we can just replace the IT people, and then other people. It’s like, Well, I actually like the, you know, data engineering skill set. It’s really hard to find somebody good and, you know, we pay them a lot of money, so we value them a lot. I mean, what have you seen? Well,

Matt Kelliher-Gibson  08:41

so we’re gonna go back to the old in time data world when I first started, I think the most common thing you saw was people would hire data analysts. Yes, I want whatever it is. I need an Excel monkey. I want someone to go right find me a nerd to do whatever those types of things. And then there was this brief error where they said, First Data hire shouldn’t be an analyst, it should be a data engineer. The problem with that was, you would hire a data engineer, and they would come in here and they’d be like, I’m gonna go make the plumbing or whatever, and they would immediately start getting data analyst requests. Yeah, of course, yeah. So you’d have this data engineer who’s now spending over half of his time trying to use data requests, which, to be honest, most of them were not very good at Sure. That was not their thing. I have. I’ve worked with some very good data engineers. I have also worked with data engineers that literally didn’t know what we did as a company for sure, like you would sit in a meeting, they’d be like, why are we doing this? Why do we have all of this information on that, on these customers? Because we’re giving out loans. Do you. Not understand New Year, like, why do

Matt Kelliher-Gibson  10:03

Do we even need this? Yeah,

Matt Kelliher-Gibson  10:04

Why is this all secured? Because it’s personal information that I felt was always a problem. I think there is this temptation to say, well, we got to do it sequentially, right? Data engineering is the first step. We have to do that first, and we’ll build up. But it doesn’t really work, because people want something tangible from it. That’s why Android typically was the first high walk, right? So you’ve got to, kind of, it’s like this cold start problem. You have to figure it

John Wessel  10:35

out well. And I think we, I think it was the episode we did with the team from LED. We talked. We went way, way deep on, like, housing and plumbing and like analogies between that and data. And I think that applies again here where it’s like, okay, I mean, hey, we hire a data engineer. With first hire, you can end up with a house with like seven, seven bathrooms all plumb, with like three sinks each. Yeah, you know, yes, multiple, you know, showers, it’s all the bathtubs and weird places.

Matt Kelliher-Gibson  11:05

And the hard part with that is, the better the data engineer is, the more they’re susceptible to this idea. When you say, Hey, we’re doing this data migration. We need to get this data from what they said, like, we can set up a staging area, and we’ll do this. This is gonna last four months, right? I don’t think this is a full term thing, right? I don’t know you’re wanting to lay down a railroad track as we’re just gonna pick it up behind you, right? Not gonna be elbowing right now.

John Wessel  11:34

I mean, that’s actually a really interesting topic, because, again, if we were talking homes, like, there are situations where, like, Hey, this is an RV situation, right? We literally want to, like, park this here for like, a couple weeks. Needs to be livable, but doesn’t need to be perfect, and then we’re going to move it. And then in data, like, the category is, is typically either fully overbuilt, like we’re going to build an empire worth to last a lifetime, right? Or, like, you know, a tent, yes, there’s not much in between. As far as philosophy, right?

Matt Kelliher-Gibson  12:07

There’s the one end, which is that we have to get it right front, yep. And it needs to be the Taj Mahal that we’re doing. And then there’s the other one that’s like, there’s a canvas over there, and I can hang a string and we can do it. And being it’s neither of those, because you need to have something that’s a little sturdier most of the time. I mean, there’s also, there’s different situations required from things, but you generally need something that’s going to be sturdier, but that can evolve, right? And sometimes that means you have to redo things, and that pisses you off. A lot of people, sure, who built it, like, I don’t want to build this. I’m just going to have to rebuild it in two years, right? I get that also, that’s the best way to go about this. Yeah.

John Wessel  12:53

I mean, I think, I mean, the rework thing is challenging, right? Because especially when you’re pitching projects or talking to your projects like that, that’ll come up, like, Okay, we have free workforce later in the honest, if you keep saying no, like, that’s and you’re going through, like, a complex project, like, that’s a little bit of a red flag over any, like, large time frame, yeah. Because the reality is, if you’re gonna invest in such modularity and flexibility that you will never have to, like, re, like, rework anything, then that’s not necessarily the right answer, yeah, nor even if you do invest in that like, there’s always gonna be some amount of rework.

Matt Kelliher-Gibson  13:37

And the worst one is when you’re sitting on that edge between an old system and a possible new system, right? I remember one place I worked, it would clear things we knew would make our ability to, like, track user data and stuff better. And we’re going to sunset that out, we’re going to have a new app and like, okay, so when is it going to come? Three to five years, right? Five years later, they were three to five years from it, and all of these problems had piled up to the point where it was causing customer problems, right? And it was one of like, if we had just done the work, not just little bits here and there over that, we wouldn’t be in this situation where, instead they had to fire a whole team just to modernize this app that they were still working on track in place.

John Wessel  14:26

Totally, yeah. Well, and the interesting part there too is it can be a good strategy. There’s a company that I was speaking to recently where, essentially, the company had been around 20 or 30 years. They got acquired by, you know, by a much larger company. And they made it. They made it on sticks and stones and older technology, and this, that and the other, and sold. And, you know, whoever, you know, whatever owners were part of that company, was it the right decision for them to, just like, make it happen and keep the lights on? On for, you know, for 20 years, and, yeah, bare minimum. And maybe now it’s also easy to think through, like, well, could they have done more if they’d, you know, invested more here and there? Like, maybe too, like, they may have missed out on some things. Like, it’s hard to it’s hard to

Matt Kelliher-Gibson  15:15

quantify, it’s hard to quantify, which makes all of these things tough to just

John Wessel  15:19

get but because people, because these modernization efforts, usually happen after a hard loss, not like an opportunity loss.

Matt Kelliher-Gibson  15:29

Yeah, that’s yeah. That’s very true. It’s usually once you there’s a perception you get a wall of what you can do, yes, or that there is some feeling that it’s holding you back, or you get new leadership in and they have that moment of where they’re horrified. You’re doing what? No, we have to stop this, right? But it always has those risks. It never goes smoothly, and sometimes the writing, the right answer is not keep it, we’re getting rid of it. It’s kind of like we talked about a brief so hollow it out, right? And just use the frame of it and just put everything else together on Yes,

John Wessel  16:04

yeah, definitely. All right, so we’ve got to talk about the Festivus and the cheer and the Christmas what? So we’ve got the 12 days to show us, yeah, what are 12 days now? I’ll do that. You. Say, what are 12 things that you’re grateful for this year? No, what are one or two things that you can look back on, that you’re as Festivus or cheer whichever category you’d like to choose, but I’ll think of one or two as well.

Matt Kelliher-Gibson  16:38

So we talking like day

Matt Kelliher-Gibson  16:41

and data, and yeah, and data. And I’ll

Matt Kelliher-Gibson  16:44

Say one thing that I am happy for is the final kind of break away from all of these SaaS walled gardens, and putting the warehouse kind of like in the center. Yeah, I always keep working on it when it was like, Oh, we got all this data, and it’s in a SaaS application, and they won’t let it out. I hated that. Yes, they moved it to the warehouse, and were very grateful for it. Comes with a whole bunch of other challenges, but I’d rather deal with those, you know, hostage, yeah, by the vendors.

Matt Kelliher-Gibson  17:19

that’s a good one other

Matt Kelliher-Gibson  17:21

thing I am, you know, we’re now a couple years away from the peak insanity of the COVID tech evaluation and stuff that we’re starting to, you know, we’re seeing more and more tech companies are having to act more like real companies now that money is free, right? And I would just like to show my appreciation for all this, to say, Welcome to my world where money doesn’t just fall out of the vents every time you want to do something. Well,

John Wessel  17:54

I mean, if you’re an AI, I would argue that it’s still kind of falling out of the sky. But other than that,

Matt Kelliher-Gibson  18:01

yes, yes, okay, but I try to pretend that’s not really. I know you see, you pretend that’s not real,

John Wessel  18:06

all right, my, my number one, I would have to agree and even kind of expand on there’s an awesome trend and data of a company like centering what they’re doing around the warehouse, but like, broader scope than that, just seeing all the growth in in open source data formats like iceberg, yeah, for example, AWS, you know, released some cool things with as three and iceberg at the conference this year. And then, you know, all the other major vendors are playing with it as well. But I think that’s a really exciting trend where, if there’s some future where essentially the all of a company’s data can live in some commodity storage, yeah, and then applications that get access to it, there is a flip of like, I have my data and I allow application access to the data, versus Like I store all of my data with an application, right? And they’re required by law to give me the data if I leave but you know, it can be 111, CSV file at a time if they want it to be,

Matt Kelliher-Gibson  19:13

like when I put them all in Excel, yeah, or

John Wessel  19:16

Yeah. So I, I hope that philosophy keeps trending in that direction where it really is more about we have all of our company data stored together, and we’re allowing these software vendors to to, quote, use it, or to be part of their product, and then when we’re ready to leave, we just cut off access. We’re not, like, quote, migrating necessarily. I think that would be a hugely positive thing. Yeah,

Matt Kelliher-Gibson  19:41

I think the big key for that will be having the open formats not be kind of CO opted, I’m sure, you know, oh, we’ve got 17 vintages of iceberg, depending on your

John Wessel  19:53

different flavors that are really compatible with each other, right?

Matt Kelliher-Gibson  19:57

You know, well, they have it, but you can only. Use their catalog. It doesn’t play nice with anybody else and those types of things. Yeah, i this will all be great as we kind of the couple the warehouse up until we reach the absurdity point of that, and then someone sells a completely coupled warehouse, right,

John Wessel  20:16

right? Or just a couple, you know, solution that includes, you know, X, Y and Z, other things with the warehouse, right? So which, which is the trend, right? Like this is the type of thing that will get de coupled, further decoupled, then, like the group together, which, my absolute pet peeve, frustration of the example of this is Cable TV and streaming like, there was such a hard sell of like, cut the you know, cut the cord, save money, move to like streaming platform, and then essentially, we’re at the point where everybody’s actually paying more than they ever would have, and you have to pay one per thing, and it’s insane.

Matt Kelliher-Gibson  20:59

We’re seeing consolidation, and there’s attempts at more of it. Will have to pay for a streaming service you want to pay more money than you want to, and it’ll come with 100 channels or content they don’t want, you don’t want, or write back the cable eliminated the cable box,

John Wessel  21:15

right, right, right, which is wild, like it’s just not, not something. I mean, it makes sense retrospectively, but it’s like, man, we really were. We really failed on that initial promise. Do you know, pay for what you want to use, etc, like that is not a card.

Matt Kelliher-Gibson  21:29

It’s going to be great. Oh, wait no, no, it doesn’t work. It doesn’t work. And,

John Wessel  21:35

yeah, and my fear is that, you know, some version of that very well may happen with a lot of this data space de consolidation. But I don’t know. Maybe it’ll be different at this time,

Matt Kelliher-Gibson  21:45

maybe. But I think as we look forward to another year of commentary on the ridiculous things that happen, the fun stuff and the ridiculous things that we get out of data, I think it’ll be an interesting time all around especially with all the stuff going on world, as

John Wessel  22:03

They say, May you live in interesting times.

Matt Kelliher-Gibson  22:08

Oh, that’s not a problem. We’ll

John Wessel  22:10

be prom All right, thanks for joining us. Merry Christmas. Happy New Year to everybody.

Matt Kelliher-Gibson  22:15

Happy Festivus. Stay cynical. See ya.

Eric Dodds  22:19

Data Stack Show is brought to you by RudderStack, the warehouse native customer data platform. RudderStack is purpose built to help data teams turn customer data into competitive advantage. Learn more at rudderstack.com.