Microservices? Or Macro Disaster?

Transcript

Rahul Subramaniam: Kubernetes took a very DIY approach and then came the evolution of the CNCF Landscape.

Carlisia Thompson: Kubernetes enables bad habits, is that what you’re saying?

Rahul Subramaniam: I would say that.

Carlisia Thompson: Oh, man. This is not a Kubernetes issue. This is a culture issue.

Hilary Doyle: This is AWS Insiders, an original podcast by CloudFix about the services, patterns, and future of cloud computing at AWS. CloudFix is a tool that finds and implements 100% safe AWS recommended account fixes. I’m Hilary Doyle, and this is Rahul Subramaniam. Hi, Rahul.

Rahul Subramaniam: Hey, Hilary. How are you doing?

Hilary Doyle: I’m well, thanks.

Today, we’re going macro on micro services. Who should be using them? How should we be using them? And has Kubernetes, as a platform, become the Etsy of web development? We’re getting into it. Starting with Cube, aka K8s, aka the OG, Kubernetes and the microservices contained within the containers within Kubernetes. Some might say it’s all Greek to me, which would be convenient since Kubernetes is actually Greek for helmsman or pilot. Thanks Wikipedia. So as always, we’ll look to you Rahul to lead the way. Let’s start with some definitions, namely Kubernetes. How about containers and microservices? Throw those in there too.

Rahul Subramaniam: Great. So here’s the short version.

Hilary Doyle: Like short.

Rahul Subramaniam: Yeah. Containers are basically the way that you package up all of your code and applications. Right?

Hilary Doyle: Got it.

Rahul Subramaniam: It’s actually really neat. Kubernetes takes these containers and manages the entire life cycle of their deployments. So typically, a unit of deployment on Kubernetes is what we call a pod, or you could call it a microservice in a rough sense. And that pod or microservice can contain one or more containers. Now, depending on the load that you have for your applications, Kubernetes can either scale up or down that particular pod or microservice, and that is the core functionality of the Kubernetes platform.

Hilary Doyle: Thanks for the primer. I want “containers are awesome” on a t-shirt. I’m looking forward to your thoughts on Kubernetes because I think there’s a little disdain there.

Rahul Subramaniam: Maybe.

Hilary Doyle: We will cover microservices. We’ve got Kubernetes all star, Carlisia Thompson plus your hot takes, your tips, your tricks, and a use case I think our listeners will love in every language. It’s a very packed show for something so micro. So let’s get it started with the news in the cloud.

Rahul, straight from the source, AWS has announced a new capability to create allow lists in Amazon Macie, what’s the significance of this?

Rahul Subramaniam: So just for the record, AWS Macie is one of those services that really lets me sleep well at night. One of the big struggles that enterprises have is scrubbing the data of all sensitive information so that it can meet requirements like GDPR. The allow lists help you work around false positives in Macie analysis.

Hilary Doyle: Testify.

Rahul Subramaniam: So say you have a random ID that looks like a phone number, Macie might log it as sensitive, personally identifiable information or PII. With the allow list, you can tell Macie to ignore that field or column and move on with the others.

Hilary Doyle: This sounds great. We could all use more insurance against sensitive data creep. That’s a hard pivot to this next news item. New York Life, third largest life insurance company in the US. They’re going to the cloud, or part of them is anyway. How do you feel about companies taking this hybrid approach where they send some things to the cloud, they keep some systems in the mainframe.

Rahul Subramaniam: So, the question really is how do they feel about this kind of low level infrastructure lock in that they have had for decades with mainframes? It is a classic example of a long-term relationship that you just cannot get out of.

Hilary Doyle: I don’t know what you’re talking about.

Rahul Subramaniam: So the fact of the matter is that the higher the level of abstraction that you operate at in higher auto services, the less locked in you are. That’s what most companies don’t get when they talk about cloud vendor lock in.

Hilary Doyle: Got it. Okay. So you’re saying lose the hardware and jump, but don’t jump ship because you can now enjoy the cloud on the high seas with Royal Caribbean International. They have taken a multi-cloud approach to building their architecture, in part, because obviously they can’t always get reliable service out on the water. Rahul, what are your thoughts on a multi-cloud approach?

Rahul Subramaniam: I have to tell you that I’m a diver.

Hilary Doyle: Impressive.

Rahul Subramaniam: But this really makes me seasick. I mean, multi-cloud just adds way too much complexity, especially if you’re using it for a single solution. And I don’t think that they’re choosing multiple cloud providers just because of connectivity issues. I think they need an architecture that’ll work in offline mode as well and I can bet that they’re going to use or they already use AWS Outpost, which is a great solution.

Hilary Doyle: I’ll take that bet and I’ll just send you the money.

Rahul, when I first heard about the Borg system, I naturally thought of Bjorn Borg, the Swedish tennis sensation. Then I thought about Star Trek. But by all accounts, there’s this third Borg lurking about the Borg that birthed Kubernetes. What is that origin story?

Rahul Subramaniam: So this is the early two thousands. Google had a bunch of bare metal instances lying around.

Hilary Doyle: Bare metal, the closest this show will come to pornography. Go on.

Rahul Subramaniam: So engineers used those servers to create a managed private cloud. That was known as Borg, as you said, and slowly, as word got out about how Google was managing their container deployments, everyone wanted to jump in on the action. So Google open sourced it, and that’s what became Kubernetes.

Hilary Doyle: That was very kind of them. What was it about this container management at the time that made K8s so attractive?

Rahul Subramaniam: So imagine this scenario where dev teams were battling these really badly architected monoliths at the time.

Hilary Doyle: Sounds gladiatorial.

Rahul Subramaniam: And they were also trying to migrate to the cloud, right? Because the cloud was just taking off at that time. That led to both DevOps and microservices becoming popular, and Kubernetes became theoretically a greater vehicle to orchestrate these workloads.

Hilary Doyle: I love these stories about the moments when everyone comes together on one platform, they never last. So how were you using Kubernetes and how did it change your approach to development?

Rahul Subramaniam: So I started using container orchestration way before Kubernetes was a thing.

Hilary Doyle: Of course you did.

Rahul Subramaniam: But as Kubernetes was evolving as a leader in the container orchestration space, we found that it was simpler and better than any of the alternatives out there. And I can’t emphasize how revolutionary this was. Everyone was talking about it. Kubernetes became a tech darling, but then we started running into issues.

Hilary Doyle: Oh, sounds like a real Hollywood epic. The rise, the fall of the open source system. I think our special guest is going to have a lot to say about this in a few minutes, but before we hear from her, I want to tell you an equally dramatic epic story about your COVID hobbies.

Rahul, what were some of your early COVID hobbies? But you strike me as a sourdough kind of guy.

Rahul Subramaniam: I definitely was on that sourdough bandwagon and yep, staying up, keeping track of the starter was, well, okay, it’s a nightmare I don’t want to go down and discuss. But I did get down to going back to a lot of the tech stuff that I used to do earlier. So yeah, that is a lot of fun. What about you, Hilary?

Hilary Doyle: I did not return to tech roots, but I did pick up piano. Immediately reminded me of why I left piano behind as a child. But I will say one of the most successful things I did during COVID was work on my French using an app called Babbel.

Hilary Doyle: It turns out I was definitely not alone in this pursuit because that sudden surge of interest from people who wanted to sit down and learn a new language was a once in a lifetime opportunity for the app and also an existential threat. So Rahul. [foreign language] I think that means fill in the blanks in French. Let’s move on.

Rahul Subramaniam: So here was the problem, Hilary.

Hilary Doyle: I just presented one to you as well. My French. Babbel, this is all on you.

Rahul Subramaniam: Babbel’s entire learning platform is digital. I mean, they have something like, what, 10 million subscribers and 60,000 courses that are taught in 14 different languages. So when the pandemic happened, the app’s reliability and scalability took a hit and they had to rethink their entire architecture because they were running on EC2 instances and ops work, which is grandfathered today. So what they did next is a really good example of how you upgrade to manage services and serverless.

Hilary Doyle: Ah, there it is. Plug from the Wizard of Serverless. If I had Babbel in front of me, I would say, “How’s that for a teaser?” And I would say it in Arabic, but instead, I’m just going to say more on that story later, after we speak with our featured guest about the evolution of Kubernetes from tech sensation to, well, let’s let Carlisia Thompson and Rahul hash it out. Did Kubernetes get left behind or is it still thriving?

Carlisia Thompson, welcome. We are so happy to have you with us.

Carlisia Thompson: Hello.

Hilary Doyle: We know you are busy. You are one of the hosts of the Podlets podcast. You talk a lot about Kubernetes. You also host the San Diego Kubernetes Roundup, and among other things, at VMware you work with Kubernetes networking. K8s seems like a calling for you, more of a community, and my question is, why?

Carlisia Thompson: Being a developer, I have struggled with deployment, tooling or mostly lack of tooling, and when I started working with Kubernetes, I said, this is solving multiple problems. Every single job I had, I had to learn a new set of tools and develop skills to push things through production. And yeah, quite painful.

Rahul Subramaniam: I totally get where you’re coming from, Carlisia. I mean, we were one of the early adopters and jumped right in with our DevOn test environments. Everything was great until we started pushing our deployments to production. The production environment is where it all fell apart for us. We saw failures and gaps in networking, in storage, monitoring, and even scaling. I mean, we were all running around like our heads were on fire trying to close the gaps in all these areas. It was a game of whack-a-mole with these issues. We were constantly searching for extensions that either someone else had written or often writing them ourselves to overcome these limitations of Kubernetes. It got so hard that we started questioning a bet on Kubernetes and asked ourselves if Kubernetes was the hill we really wanted to die on.

Carlisia Thompson: I think a lot has been changing and people are aware that there are best practices that you should follow, and that requires a lot of work and trade offs that you might not get anything from if you’re not doing that work. I think there is going to be a point where things are ready to consolidate and the tools that are more reliable and solid, they are going to surface more. And I see a lot of change coming in that direction.

Rahul Subramaniam: Yes, things improved, but my take is that they actually improved in a way that wasn’t very developer friendly from a standpoint of deploying applications. Kubernetes took a very DIY approach where they were like, oh, we are going to rely on this ecosystem to solve all these problems. And then came the evolution of this CNCF landscape. And to be very frank, that has evolved into an absolute nightmare for someone like me.

Hilary Doyle: It haunts your dreams. Yeah.

Carlisia Thompson: Yes.

Rahul Subramaniam: It actually haunts my dream. If you open that, I think the last time I checked on it, there were over a thousand cards across what, 30, 40 different categories. It’s basically paralysis that comes from being overwhelmed by all of these choices.

Carlisia Thompson: I completely agree with you. It is, absolutely. It’s quite insane if you look at that with the perspective that you have to know every single thing in there to then make a decision. I wouldn’t approach things that way, but if I did, I would very much share your pain. The way I would approach it is I need a tool that does X and like I would go on Amazon and look for a toothbrush, I’m not going to read the reveals for every single one because if you try to find one of the tools that’s going to do everything you need, you’re not going to find it. With Kubernetes, you can build your own and there is no reason why you can’t use multiple tools either in some cases.

Hilary Doyle: I want to jump in there because I am a co-founder of a company. We’re scaling very quickly, and in the early days of our company’s evolution, we worked with a dev who loved to hack together solutions. He was passionate about writing code and it was intoxicating because he would go away and he would just make things work with whatever we had. And the challenge was that over time we discovered that the only areas that were breaking were the ones that he’d hacked.

Carlisia Thompson: So what I am hearing, I think is that you are saying Kubernetes enables bad habits.

Rahul Subramaniam: I would say that. Developers in general like to create stuff, and so when there is a solution that they haven’t created or they don’t have their stamp on it, they feel like they’ve not made a contribution to their employer or to their work. So I think there’s this strong sense that you want to go build something from scratch, even if there is an off the shelf solution out there that you can just pick and use.

Carlisia Thompson: Oh man, this is not a Kubernetes issue. This is a culture issue. The issue here is more one of having proper heuristics to choosing what you need, knowing what that is and sticking to it. Having a culture that follows best practices, I think that can mitigate a lot of these issues. If you want to restrict your work to building apps, I don’t know, maybe you need to make sure that you’re not hiring developers who want to build a platform if that’s not what you’re building.

Hilary Doyle: If you did not enjoy coding, if you weren’t a developer, would you be encouraging your teams to build on Kubernetes?

Carlisia Thompson: Hilary this is not fair. This is a soul searching exercise. What problem are you trying to solve?

Hilary Doyle: When you talk about tools that are more reliable and certain, I mean, what about serverless?

Carlisia Thompson: I built a tool that’s serverless. It’s just that it runs on Kubernetes. I use a lot of tools to get my codes running and tested, and to look at things from different perspectives, to look at, like, what is the architecture of this thing that’s here.

Hilary Doyle: For you it’s just not a replacement for Kubernetes. It’s an addition.

Carlisia Thompson: Absolutely, yes.

Rahul Subramaniam: For me, serverless is something which is a complete end-to-end managed service where you have everything from your monitoring, to alerts, to tracing all your observability built into it, and of course your execution, scalability, all of that baked in without you having to deal with the nitty gritty infrastructure under the hood. I don’t think Kubernetes serves that purpose.

Carlisia Thompson: Sorry, we didn’t touch on the fact that there are managed offerings of Kubernetes. You use some managed serverless or even not serverless, anything that’s managed, you have an interface that you have to learn, et cetera. And then if you move to another provider, then it’s a different interface. And so now all your developers have to learn this other thing. And if you are using Kubernetes, even if you are using tools that are tooling on top of Kubernetes, they’re going to have different interfaces. But one thing that’s beautiful about Kubernetes is you can always look under the hoods. So I love that.

Hilary Doyle: Can we talk briefly about this fear of vendor lock in?

Rahul Subramaniam: By the way, I’m a huge fan of containerization, but that’s completely separate from Kubernetes. I mean, today there are over 20 different ways of deploying your containers, whether it be in different cloud services or from everything from Lambda to ECS to EKS, to any other Kubernetes based deployment. You can do all of that. So for me, there is a separation between containers and Kubernetes. Almost everyone that I speak to who talks about why they picked Kubernetes, they say we wanted to prevent vendor lock in. And I struggle to understand that argument from a simple standpoint that in Kubernetes you’re instead locked into your own architecture and your own decision to have tons of decisions being made, and then constantly figuring out whether that’s right, swapping them out constantly, learning more about their entire ecosystem, which is growing at an insane pace that you really can’t keep up with it.

Carlisia Thompson: You ask, why is vendor lock-in even something to think about? It’s like, what is the big deal? I mean, I don’t know, how much time do we have? What if the vendor doesn’t want to do something I need? It’s not going to have the latest version, so if I want to get the latest of whatever Kubernetes has, I’m not going to get that with the EKS. So now all your developers have to learn this other thing, and you might want to pick a different provider at some point if you’re not happy with the one you currently have.

One way that Kubernetes solves for that is by the platform itself, being flexible in allowing people to build on top of it. You can mix and match and replace things as you need. So you locked in into Kubernetes in a sense. I mean, you can take your apps out of Kubernetes, and as long as you want to run the containerized apps, you can run them elsewhere.

Hilary Doyle: Okay, let’s pull the threads of this conversation together. Rahul, what do you want listeners to take away?

Rahul Subramaniam: When I put on my developer or geek hat.

Hilary Doyle: Such a good looking hat.

Rahul Subramaniam: Yeah. Kubernetes feels like manna from heaven. When I put on my business owner hat, I realized very clearly that there is a disconnect in the perception of business value and costs. Developers or technologists focus on costs and value of the technology is looking at it holistically, accounting for time spent in making that technology work. They also see value in this false sense of control and ownership, which from a business standpoint, screams more liabilities.

Hilary Doyle: Carlisia, it sounds like control over the platform has been an important part of your success. Any final thoughts for Rahul?

Carlisia Thompson: So I absolutely get the archetype of the developer who gets a rush from building things, and especially building things that they want to versus the things that they have to. Totally get that. That’s totally me too. The life in the Kubernetes ecosystem is so vibrant. People are experimenting and innovating. It hasn’t slowed down. People are very focused on usability. People are very focused on coming together. But listen, having developers that have the maturity and follow whatever your cultural guidelines are to make decisions based on need versus whim, this is a cultural thing. I’m going to keep saying this. Every time you say developers want to do this. It’s like, well, yeah, but how is your culture?

Hilary Doyle: It comes back to, as you said, community and culture, and it sounds like both are thriving within the Kubernetes ecosystem. So thank you for giving us some insight into what that community looks like and how that ecosystem is moving forward.

Carlisia Thompson: Of course. Thank you.

Rahul Subramaniam: Yeah, thanks again Carlisia.

Hilary Doyle: That was a fascinating conversation with Carlisia. One of the biggest takeaways for me is this notion of company culture being the way to reign in developers on Kubernetes. Our company ended up building out its early stack at the onset of COVID. What that meant was obviously we were suddenly building remote, that had not been part of the plan. What that also meant was that we were building culture remotely as well, and we were figuring that out in real time. So this idea that Kubernetes might rely on a corporate culture to keep developers in check and working toward the same goal. And no disrespect to developers, they do not need to be reigned in. But if this is the conversation we’re having, the idea that company culture might become a success factor with Kubernetes, I think that’s going to be problematic for many businesses in a corporate world that are trying to figure out culture building in real time right now, in a new hybrid world.

Rahul Subramaniam: That’s exactly right. I think Kubernetes feeds the culture of it’s not built here, and that is a real problem. You can enforce a more business and value aligned culture with a managed serverless service.

Hilary Doyle: Oh, what a surprise. Serverless.

Rahul Subramaniam: I had to bring that in.

Hilary Doyle: Rahul, every episode you share your tech secrets. The commander of containers, the sorcerer of serverless. We look to you for your top tips and tricks within the container landscape. Please proceed.

Rahul Subramaniam: So here’s the things that you really need to know if I’ve convinced you to step away from both Kubernetes and microservices.

Hilary Doyle: I’m ready.

Rahul Subramaniam: For most tasks, Lambda is usually a great answer. You can run most general purpose workloads that need up to 15 minutes of compute time, you can optimize that even further by using the compute optimizer for Lambda that can right size your Lambdas for memory and basically your costs. Now, if you really need more memory or compute or just need to run long running jobs, use Fargate. Basically, just stay away from the headache of managing a Kubernetes cluster.

Hilary Doyle: I appreciate this enthusiasm for seed.

Rahul Subramaniam: Great. Next, developers really undervalue simplicity and don’t understand the cost associated with that complexity. So if you weigh it all in, serverless is incredibly cheap. Next, focus on good architecture. You can achieve it with even monolithic setups. You don’t need to do microservices for everything. I have this thumb rule for developers and microservices. You can only have as many microservices as the number of real balls that you can juggle at the same time.

Hilary Doyle: That’s terrific.

Rahul Subramaniam: For me, that’s two.

Hilary Doyle: Okay. For me, I think that’s one. I’m just kidding. I’m an excellent juggler. It’s three. It’s three. Go on.

Rahul Subramaniam: And lastly, ask yourself if you’re solving the business problem. Kubernetes is an answer to a problem that developers created for themselves and just kept themselves busy. Rarely is it an answer to a business problem.

Hilary Doyle: The man has spoken, the sorcerer has spake. Mark him.

Rahul, how many languages do you know?

Rahul Subramaniam: Oh, that’s a tough one. Let’s count Python, Java, C, C++, PHP.

Hilary Doyle: I did not mean programming languages, but please don’t change a thing. When we left our Babbel case study, Babbel was struggling with a sudden surge in demand for millions of people stuck at home during COVID, no longer able to travel so doing the next best thing and picking up a world language

Speaker 4: [foreign language]

Hilary Doyle: On the one hand, it’s a once in a lifetime opportunity for the app to multiply its user base, right? On the other hand, it was exposing in real time just how unprepared they were to handle this kind of volume. One, they were running OpsWorks. So, what happened?

Rahul Subramaniam: Well, they cut down their maintenance times, achieve faster releases and roll back times.

Hilary Doyle: Oh, easy.

Rahul Subramaniam: They got instant deployments and rapid scalability.

Hilary Doyle: Great.

Rahul Subramaniam: Which met their pandemic demand.

Hilary Doyle: Solved. Done.

Rahul Subramaniam: Exactly. And in the process, they reduced their costs dramatically.

Hilary Doyle: Boom.

Rahul Subramaniam: And yes, their setup was a real problem. Their old OpsWorks based deployment that used Chef to configure EC2 instances was a real headache. They really needed to change their architecture completely and quickly.

Hilary Doyle: So how did they do it?

Rahul Subramaniam: Well, they went with managed services and serverless. Full stop.

Hilary Doyle: [foreign language]

Rahul Subramaniam: They dropped OpsWorks, replaced it with Compute, replaced all of the Compute and hosting. They chose Lambda for automatic scaling, and they had no instances, no manual upkeep, no Chef scripts. And for anyone who’s dealt with Chefs scripts, you know how much of a nightmare that one is. But Babbel, again, couldn’t completely rely on Lambda so they did have some services that required Docker. And again, I’m a big fan of containerization, I think that’s the right way to do it. For that they went with Fargate on ECS and not a version of Kubernetes that they were managing themselves. So kudos to them. They made all the right decisions.

Hilary Doyle: That is it for us. You’ve been listening to AWS Insiders by CloudFix. I’m Hilary Doyle.

Rahul Subramaniam: And I’m Rahul Subramaniam.

Hilary Doyle: CloudFix is an AWS cost optimization tool. Learn about them at cloudfix.com.

Rahul Subramaniam: And please check out the show notes for Carlisia’s information and more. Leave a review and please follow us.

Hilary Doyle: We will catch you later.

Rahul Subramaniam: Bye.