DHH Left the Cloud. You Shouldn’t.

Transcript

Hilary Doyle: Oh man, Rahul, I hope you are ready for some hot takes.

Rahul Subramaniam: Always, Hilary. Always.

Hilary Doyle: Good. Good. Because on this episode, we are speaking with David Heinemeier Hansson. He is the controversial tech executive who recently posted, “Renting computers is mostly a bad deal for medium-sized companies like ours with stable growth. The savings promised in reduced complexity never materialized. So we are making our plans to leave.” And by leave, he means leaving the cloud.

Rahul Subramaniam: Safe to say, Hilary, I have a lot of opinions on this myself …

Hilary Doyle: Do you?

Rahul Subramaniam: … about why the cloud is essential for innovation. I mean, for instance-

Hilary Doyle: No, hold on. Hold on because we are going to get to all of these opinions in today’s episode, and I know that they will be spicy.

Rahul Subramaniam: This is AWS Insiders, an original podcast from CloudFix, bringing you what you need to know about AWS through the people and the companies that know it best. I’m Rahul Subramaniam and I’m the founder and CEO at CloudFix.

Hilary Doyle: CloudFix is the non-stop automated way to find and fix AWS recommended cost savings. It never stops working. I’m Hilary Doyle. I’m the co-founder of Wealthie Works Daily. Okay, let’s do this.

As you know, Rahul, this show is all about making the AWS experience better, but we’re also open-minded people. We don’t mind a contrarian view. So today we’re actually digging in to learn more about leaving the cloud.

Rahul Subramaniam: That’s right. I’m guessing that most listeners who see that we are talking with David Heinemeier Hansson, better known in the industry as DHH, might be saying to themselves, “Why have him on the show that celebrates AWS?”

Hilary Doyle: Right. Well, DHH is no stranger to controversy, happy to throw himself into the ring of most any conversation. He’s the co-owner and CTO of 37signals. That’s the software company behind Basecamp. Well, Heinemeier Hansson announced last fall that 37signals would start divesting itself of AWS and the cloud more generally. The cloud community’s initial shock and awe seems to have evolved into a sort of bemusement about what looks to a lot of people like a real step back into ancient history. So we wanted to hear a little more about how it’s going in real time from the man himself.

Rahul Subramaniam: I think this is an absolutely fascinating story, Hilary, with lots of useful and helpful information for the rest of us who are committed to staying and innovating in the cloud.

I believe that David’s perspective is molded by two things. First, he and his company, 37signals, are what I would call serial foundation builders. They have a history of creating these fundamental building blocks for applications with frameworks like Rails and Hotwire. And when you do that for 20 plus years, you can’t get away from the, “Hey, it’s got to be built here” or, “It’s got to be built from the ground up approach.” 

The second is the perspective that when you operate at these lower layers of the stack, you have a different flavor of what you might call innovation. For 37signals, they think reinventing email, which has been around for about 50 years and ubiquitous across the planet, is innovation. And that definition of innovation creates a different set of trade-offs when it comes to their platform decisions.

For the vast majority of us, innovation is creating something new from existing building blocks, and I believe that the cloud is the only place where you will find these building blocks. To be clear, I’m really excited about this conversation because it is really important for the audience to get these different perspectives so that they can make up their own minds.

Hilary Doyle: Absolutely. And we have DHH here for what is bound to be an extremely lively discussion. But first we will get you caught up with what’s fresh off the presses. Here are this week’s AWS News headlines.

Well, it looks like AWS’s own Hercule Poirot, or was it Inspector Gadget, or maybe now it’s Sherlock Holmes. Anyway, it’s back. Amazon Detective has been expanded and can now do more comprehensive investigations. So it’s not just about who got Professor Plum with the candlestick in the ballroom, it can now help you identify resources, patterns, and the scope of potential security issues. Rahul, since we’re probably not solving any crimes here yet, what does this mean for security?

Rahul Subramaniam: Okay, so Hilary, dealing with security across a large cloud footprint can pretty much feel like a game of whack-a-mole.

Hilary Doyle: Oof.

Rahul Subramaniam: Right? Organizations are constantly trying to balance risk, which means tight controls, and on the other hand, development velocity, which means you don’t want risk and compliance to become the roadblock in every direction. So the good news is that AWS understands this better than anyone else. AWS wants you to have the highest level of governance possible, and they’ve created tools like Amazon Detective that do all the hard work for you and identify the issues that you need to flag. With this new announcement, they have basically added some significant breadth to what Amazon Detective can now find for you.

Hilary Doyle: Okay, keep up the good work, Detective. All right, anyone out there who is running latency sensitive event driven applications is going to love this next story. Amazon EventBridge event buses now delivers events with lower latencies enabling you to power an even bigger set of latency sensitive applications, especially in medical and industrial applications. And that is big because we are talking about 80% faster.

Rahul Subramaniam: And I know that this gets a little bit too technical, but for anyone building cloud native or event driven systems on AWS, Eventbridge is what is at the heart of these solutions. I mean, in simple terms, Eventbridge is an event router taking messages from one subsystem and dispatching them to other subsystems based on some rules. And that routing just got 80% faster, which means that you can build even more responsive and efficient solutions. The thing about news items like this for me isn’t the technical details around this, but the fact that when you make a bet on an AWS service, you get all the amazing improvements like this one for absolutely free.

Hilary Doyle: Ooh.

Rahul Subramaniam: This is one thing that almost everyone ignores when they make an argument against betting on the cloud based on costs.

Hilary Doyle: All right. Every once in a while you get a freebie. Okay, Rahul, I am looking at you for this story. Ah, ah, ah. Look, AWS just announced the general availability of eye gaze direction detection in recognition’s face APIs. This means it can now support accessibility and safety. It can validate photos. It can even help identify where users focus on the screen. It sounds both amazing and utterly frightening. Tell us your thoughts.

Rahul Subramaniam: It’s all about perspectives, Hilary.

Hilary Doyle: Oh, is it always?

Rahul Subramaniam: All about perspectives. Okay. So I’m probably one person who’s quite excited about this. I mean, haven’t you ever felt like you were talking into an abyss when you’re on these Zoom calls…

Hilary Doyle: Every day of my life.

Rahul Subramaniam: … with a whole number of people just staring blankly at you or actually looking at their screens doing something else?

Hilary Doyle: What?

Rahul Subramaniam: Yeah, I mean, there are times when I really wish I had some real time measure of how engaged my audience was so that I could improve and actually be more effective. I totally see this being useful in areas where engagement needs to be measured and of course in education as well. Again, it’s just a matter of time before developers will prove my predictions to be inadequate and come up with something amazing that we’ve never even thought about.

Hilary Doyle: Nothing about you is inadequate, yeah. All right. No, I’m excited about the possibilities here as well. That is it for this week’s news headline. Joining us now is David Heinemeier Hansson. Welcome to the show.

David Heinemeier Hansson: Thanks for having me.

Hilary Doyle: In your book Rework, you write that the real world isn’t a place, it’s an excuse, it’s a justification for not trying, it has nothing to do with you. Okay, briefly to start us off, would you describe the world that you’re living in?

David Heinemeier Hansson: It is this place where you can have new ideas, you can try new things, you can throw it against reality. And then reality’s going to tell you whether it’s going to work or not. Not someone’s opinion, not someone’s quick take, not someone’s review, not someone’s enthusiasm either. 

I mean, it goes on both ends of it that I’ve tried a lot of things in my career and some of the things I’ve tried have been met with applause, and then nothing because the thing failed. Reality rejected it. And then I’ve tried just as many things where people were like, “That’s never going to work.” And then reality has validated it and it worked tremendously well. 

So I’ve learned not to trust the feedback of even myself that I am a terrible judge of whether something is going to work or not. I think all people are terrible judges of whether something’s going to work or not. Reality’s going to tell either way. So spend less time thinking about, planning about whatever. Just get the thing into the world, let reality be the judge.

Hilary Doyle: But you sound disappointed by where the internet has landed. Is that a fair assessment?

David Heinemeier Hansson: Yes, I am. I mean, disappointed is perhaps a big word, but I am a little… No, let’s use that word, disappointed.

Hilary Doyle: Got it.

David Heinemeier Hansson: About not just where it’s landed, but where it’s going. To me, the magic of the internet was it was a place to communicate, to connect, to do commerce without having to ask anyone for permission. It is the independence, the decentralization, the freedom that comes with all of that. It’s almost – it gets sappy pretty quick about sort of a space, almost like America. “This is where you go to make your dreams.” But literally, that was true for me. This was the place in the world, the virtual world, where I did to make my career and I did make my dream and I did make all these other things.

And I’ve seen large chunks of that get taken away. I became extra radicalized on this point. When we launched our new email service hey.com in 2020 and we thought, “Hey, we’re going to enter into this really competitive space with email. We’re going to compete against one of the largest tech companies in the world who’s giving away the product for free and we’re showing up here, ‘Hey, would you like to pay us $100 a year for email?’ And we’re like, ‘Oh, wow, those have long odds’.”

So we thought the big boss was going to be Google, and then we realized, “No, damn it. First we have to clear the boss before that, which is Apple,” which is to even have permission to exist. And I just went like, “Man, this is a bummer. This is really a bad turn.” Which gets us to sort of the topic I’ve been interested in lately, which is cloud and the fact that there are too many echos of what’s happening with cloud and the consolidation in the hands of a small number of hyperscalers that rings bells that sound similar to what’s been happening on mobile. And I go like, “Do you know what? I’m not too thrilled. Not too thrilled about where that’s going.”

Rahul Subramaniam: Follow up question for you, David. How do you and Jason as leaders of the organization make decisions within 37signals? Do you drive it by numbers? Do you drive it by feel or beliefs or do you have a set of tenets?

David Heinemeier Hansson: Yeah, let me try to keep it as medium spicy as possible, not maximum spicy. The medium spicy version here is that we do whatever the hell we want when we want because we want. Now that sounds so egotistical and it sounds so almost egomaniacal.

Hilary Doyle: No, you built it, you can do what you want.

David Heinemeier Hansson: Yes, but it sounds like you have no regard for other people or what they care what they think. And that’s not true at all. All those things inform it, but we’re not going to ask anyone for permission. And this is the main reason why after 22 years I’m still working with Jason at this specific company is because I don’t have to ask anyone for permission for anything. I will inform my gut, my opinions, my take from all sources, and they will absolutely weigh on it what employees think, what customers think, what credits think. 

It all gets filtered in and then we’ll make a decision and it’ll be ours and we’ll go with it. And I refuse, absolutely refuse to let the company be run by numbers. That is just… It is the most unappealing part of capitalism for me, is this illusion that capitalism forces you to run everything by the numbers and somehow the numbers are oracles that can tell everyone what to do in all different circumstances.

Hilary Doyle: Don’t say Oracle in front of Rahul. He won’t [inaudible 00:12:57].

David Heinemeier Hansson: Trigger word.

Rahul Subramaniam: Basically.

David Heinemeier Hansson: Trigger warning. You don’t need to do that. You can take it in and then you can make a decision based on gut, based on values, based on a fair diluted sense of what other people think. And that’s the only way and reason I’m still here. Again, I’ll turn it up a little bit more spicy. We had a bit of a traumatic experience at our company in April of 2021 when we made a controversial decision about internal corporate culture. Now, I’m really euphemism in it all the way to the moon here.

Hilary Doyle: Yeah, tell us what it was.

David Heinemeier Hansson: So we essentially banned discussion about hot button political issues on our work spaces. You don’t even need to go into the right or wrong or this, that, and the other thing of it. It blew up. It was a huge thing. We were trending on Twitter. A third of the company left the company when we offered them a very generous buyout to do so if they couldn’t see themself in this vision. Very difficult. 

If we’d asked permission to do that, “Hey, board, what do you think about us having a controversial take care stance on political issues?” They’d go like, “No, no, no, no, no. Have you seen what just happened to the company that got canceled last week? No, no, no. You are not walking within a thousand yards of this question. You will say the right thing. Just repeat whatever all the other corporate leaders are saying right now. ‘Here’s five soundbytes. Yes, we’re doing the work,’ that was one of them, and about 50 other ones you can repeat. And if you repeat those, you’ll be safe.”

And I’m just like, “Do you know what? I’d rather burn this place to the ground. I’d rather not work here. I’d rather sell. I’d rather pay to get rid of this company than to continue to work at a place where I feel like I’m fake, that I am not embodying my ideals about truth and, whatever, community and all the other things, the highest values you can incorporate into it that are going to subservient to that.”

Hilary Doyle: I am going to pause things here for a second because while the cloud may be giving DHH some pause, it is essential for the rest of us. And if you, like me, want to have your toughest AWS questions answered by the experts, Rahul has the live stream for you.

Rahul Subramaniam: That’s absolutely right. Every week I’m joined by fellow AWS enthusiast, Stephen Barr as we break down the latest AWS news and answer those tough questions along with some amazing guests from AWS.

Hilary Doyle: It’s called AWS Made Easy and you can find out more about it at cloudfix.com/livestream. Unsurprisingly, they stream live on LinkedIn, YouTube, Twitch, Facebook, even on Twitter. They are just waiting for their Bluesky invite code. All right, back to our talk with DHH.

Rahul Subramaniam: So David, I’d like to jump to your view of the cloud. When you talk about democratization of the cloud, there is the other side to that story as well, and that view is of the sheer pace that is enabled by these higher order services that startups and innovators just love because it makes it easier for them to get started, of course put their ideas out there really quickly, and like you said, get a reality check. How do you think about that aspect of the cloud?

David Heinemeier Hansson: I think there’s aspects of that where there’s overlap. We’ve used the cloud a bunch at 37signals. We’ve lately come to some conclusions in part because the rate of progress have been so slow on the cloud when it comes to passing on the savings of advanced technology, of speed, of drive storage, CPU, whatever. It seems like the old adage… And full disclosure here, Jeff Bezos is a part owner of 37signals. So I say this with love, but his original mantra that “Your margin is my opportunity” seems to have stopped applying to the cloud in large instances. And now they see the cloud as, “This is an opportunity for margin.”

So I think there’s some problems here where it works a little bit like a trap. And the trap metaphor becomes even more pertinent when you look at the lure that a lot of these cloud providers are offering startups. Google Cloud for example, “Here’s a hundred thousand in free credits. It’s free.” Yeah, it’s free and you start integrating with all these services and all of a sudden you found yourself in a roach motel. It’s only possible to check in and it’s almost impossible to get out. And suddenly you find yourself quite early actually in your startup journey if we’ll use that loath word, it’s too expensive. You’re just sucking up your whole runway incredibly early in the process.

So you are trading and it is real. I mean it’s incredible that in five minutes you can spin up a bunch of boxes and you have this whole fleet of servers that otherwise would take you two weeks. I mean, there’s some immediacy where we get tricked and we think like, “Because I can instantiate something in not even five minutes, but barely 30 seconds, this is revolutionary.” And you know what? Most things don’t run on the timeframe of five seconds versus two weeks. That is not the relevant difference.

Rahul Subramaniam: But it’s basically the difference between the lower order services that these cloud providers offer versus the higher order services, right? I mean, the lower order services are your basic compute storage, networking, and stuff like that. Totally get that. 

But when it comes to higher order services that make life so much easier, there’s so much software that we used to write 20 years ago that now you wouldn’t really need to write anymore. I mean, look at services like DynamoDB or even Aurora Serverless for that matter. There’s so much code you have to write and the management overhead associated with that, all that just goes away.

David Heinemeier Hansson: So that’s the premise I went into the cloud with. So we went into the cloud in full force about five or six years ago, brought over a bunch of services. We launched hey.com, our new email service entirely in the cloud on AWS, right?

Hilary Doyle: Right.

David Heinemeier Hansson: On that premise. On the premise it was going to be easier, that we were going to need fewer people, that there was all this stuff that we didn’t have to do. Know what? After five years of trying very hard with very smart people with full dedication and backing from me as a CTO, believing this is the future, it wasn’t true. It didn’t happen. The savings, all the software we weren’t supposed to write or all the management we weren’t supposed to do did not materialize.

Hilary Doyle: There’s a stat from Synergy Research. By the end of 2020, it had taken nine quarters for the cloud market to double in size. So in a sense, I mean many would say the horse has left the barn and then it’s blow torched the barn on its way out. So you want the internet to do better. Why not work with the big providers to help solve for interoperability as well as their energy bloat at scale? You’re uniquely positioned to do that.

David Heinemeier Hansson: And I’m doing that. So the tool that we are using to bring our cloud stuff to our own servers is called MRSK. It is a tool that built on top of Docker. Basic open source technology. We give MRSK away as an MIT licensed piece of technology. You can do whatever you want with it. Docker itself, a few asterisks about that story and open source, but by and large is cheap and easy to use and there are alternatives to do it. So we built MRSK. I built MRSK with the intention that someone could use a tool like that, whether it’s MRSK or it’s something else, where you get true interoperability and not the Kubernetes kind. 

This is a point I want to make. So all the focus in the cloud world has been on Kubernetes because it is a standard of sorts and it is being adopted quite broadly, but it’s so hideously complicated to actually run yourself that no one dares unless they have a large team or some frequently talented people to do it, which means that you end up with a defacto lock in nonetheless.

Were we able to just take our stuff from GCP and move it straight to Amazon with a “boop, boop, boop. Deploy”? Absolutely not. That was months of efforts. More effort, I would argue, perhaps than pulling our stuff out of the cloud was to migrate between two cloud providers who were using Kubernetes both. So that’s not too slag on Kubernetes. I actually think it’s a good standard. It solves a bunch of problems, blah, blah, blah. It’s great for big companies running infrastructure as a service. It’s not an answer for this. I am trying to build some technology here, but I’m doing it primarily in the service of giving people the option to run on their own hardware secondarily from making it easier on the cloud providers.

Rahul Subramaniam: But isn’t that just moving the abstraction to another place? As I think about it, we are locked in at so many different levels. You’re locked in right from the processor where if you pick Intel and want to switch to ARM, well, you might be in for some trouble. Sometimes even moving to AMD is a pain even though it’s really not supposed to be. And then of course you’ve got languages. You’re writing code in one language and then you’re tied to that ecosystem of libraries and utilities. Isn’t this just another layer of abstraction?

David Heinemeier Hansson: I think the difference is, first of all, if you look at the individual layers. And I’ll agree, you’re going to write your application to something specific. First of all, I’d say, and we’ve learned this lesson, and apparently we need to learn it more than once. We will never, ever depend on a piece of core technology that’s vital to running our business that is not open source. 

I’ve been burned now many times over whether the proprietary comes from the lock-in of the clouds or it comes from sort of consulting wear software that’s so complicated that you can only run it if you hire a bunch of consultants to guide you through it. So the lock-in you have to do or should do in my opinion is lock yourself towards generic open source simple tooling. Once you get down to the layer of the CPU, lock yourself into a place that has competition.

Hilary Doyle: This retooling in a sense though sounds less like a business strategy and more like a lifestyle, which in a sense has always been your brand of tech, right? But how does this approach mesh with the pressures of capitalism? So namely growth and the competition that comes along with that.

David Heinemeier Hansson: Oh, so the impetus of this was capitalism 101. It was looking at the build and going, “Wait a minute. We just did our accounting a few months ago for how much we spent in 2022 on cloud, on basically all of it on AWS, a little bit of Google Cloud. 3.2 million.” And I went like, “Oh, okay. I guess we are a large or mid-sized company. We’re making good money, whatever.” Then we started pricing out what it would cost to replace it, and I went like, “Wait, wait, wait, wait, wait. You’re telling me that we’re spending $3.2 million in the cloud on something that would cost us maybe half a million, three quarters of a million to buy? What?”

Hilary Doyle: The cloud has enabled so much growth for business. The conversation at this point is pretty blithe about cloud being the future. So where in history can you look back and point to someone who managed to succeed in halting or reversing human or business progress?

David Heinemeier Hansson: Yeah, that’s a losing bet. Don’t do that. Don’t try to halt progress in any way. It will run you over. And I think this is actually the folly. I was about to say stupidity, but that implies something else, and I don’t think these people are stupid. In fact, I think they’re very smart. The folly of what’s currently being proposed with this six month pause on AI development, I’m like, “No, that ship has sailed.”

Hilary Doyle: Sailed, yeah.

David Heinemeier Hansson: That is going to happen. Someone is going to do that work. So I don’t believe in that at all. I believe in riding the wave of progress, but not being infatuated by it. And reaping the benefits of progress, but not thinking that invariably comes with complexity. There’s a lot of people who confuse those two things, that complexity is the same thing as progress. It’s absolutely not. Sometimes we have to go through a phase of complexity or increased cost to get to a better place, but it’s never the final destination. The final destination is always simpler, better, cheaper.

Hilary Doyle: Well, unfortunately, this is our final destination, simpler, better, and also out of time. Thank you so much for joining us, David.

David Heinemeier Hansson: Thank you so much. This was amazing.

Rahul Subramaniam: Thank you so much, David.

Hilary Doyle: Well, we knew it would be an energizing chat with DHH. And in that regard, he does not disappoint. Two key points stand out for me. One, the businesses he’s talking about moving on-prem are Goldilocks businesses. They’re medium-sized with very stable, predictable growth. That is an extremely specific kind of business. And two, it’s the point he makes about not being driven by the numbers that really does it for me. I mean, that is an uncommon reality to say the least in business. And if you pair that with a similarly unusual ability to actually build what you need at scale, then sure. 

I mean, he’s in a position where he can test out a counterintuitive choice. He may be saving $7 million over five years, but he’s also telling us that he can afford to fail because he would prefer to burn his company to the ground than not live his politics. He’s a Le Mans car racing victor, a former hacker of the year, 2005. This is a guy who is obviously very comfortable with risk. Risk is just the last thing that I want in my business’ tech stack.

Rahul Subramaniam: I think I have similar takeaways from the conversation, Hilary. David has a certain worldview that comes from his experience and philosophy that he’s basically driven at 37signals. That perspective isn’t the same as that of any large enterprise nor of a startup or company that’s trying to find product market fit or growth, or even just step function efficiency. 

This isn’t a one size fits all situation. And maybe we found another niche space where the cloud doesn’t make sense. I’ve said that before. If you have an old school app that you have no intention of rearchitecting or modernizing or even innovating on, then just let it be where it is running right now. Moving to the cloud just doesn’t make sense in that case. And like you said at the beginning, it’s really important to have an open mind, hear the other perspectives on the topic, and then make the most informed decision for your company or situation.

Hilary Doyle: And of course, we want to hear your perspectives. You can reach us anytime day or night at podcast@cloudfix.com. We cover all the time zones.

Rahul Subramaniam: And hey, leave us a review and tell your friends about us.

Hilary Doyle: AWS Insiders is brought to you by CloudFix. They are an AWS cost optimization tool and you can learn more about them at cloudfix.com.

Rahul Subramaniam: Thanks for listening. Bye-bye.