I Don’t Give A Lift And Shift

Transcript

Rahul: Where I see value from being truly cloud native is really the fact that you can replace on an average every million lines of code with about 10,000 lines of code.

David Mee: You’re going to be spending money, you’re going to be hiring consultants, you’re going to be transforming your business while you’re paying for your business on-prem. It will take more than a year.

Rahul: I just don’t think lift and shift is a good cloud migration strategy.

David Mee: It’s quicker, faster, cheaper in the long run.

Hilary: This is AWS Insiders, an original podcast by CloudFix about the services, patterns and future of cloud computing at AWS. CloudFix is a tool that finds and implements 100% safe AWS recommended cost savings. That’s fixes not just analytics. I’m Hilary Doyle, joined as always by Rahul Subramaniam. Rahul, I see you’re drinking something over there. I’m going to assume it’s whiskey.

Rahul: Hey, Hilary, really excited me back on this new show. As you can see, I’m supercharged up.

Hilary: Yes. Drunk, drunk is the word. I’m just kidding. He’s not drinking anything, just trying to create some drama. In this episode, we are moving stacks. That’s right. We’re bouncing. We’re leaving the data center behind for a new fresh life in the cloud. The only problem: you have to choose between two ways to do it. Do you want to lift and shift? Sounds very Schwarzenegger or do you want to go fully cloud native? Sounds very Marianne Williamson. Rahul, please help me out. What are we talking about here?

Rahul: Okay, so this is going to be such a fun episode, Hilary, let’s just start with lift and shift.

Hilary: Go on.

Rahul: So as the name suggests, you are just moving your servers from your data center to servers at the cloud provider.

Hilary: Got it.

Rahul: And there aren’t significant changes to how the application works, it just move locations. The second approach, often referred to as modernization, replaces some or all of the existing functionality with cloud native solutions or alternatives. For all practical purposes, this is like open-heart surgery, but for your applications.

Hilary: Okay. Well, no one likes open-heart surgery except for the surgeons. I have a sense of where you’re going to land with this, Rahul, and I am looking very forward to the chat we’re having with David Mee. He is a migration expert at Carbonite. We are also talking TV ratings for our migration use case.

You’re going to love that use case. We’ve got Rahul’s hot tips, we’ve got even hotter hot takes. But first, your AWS headlines. Rahul, Amazon EC2 now offers an automated connection setup solution between EC2 instances and RDS database. Can you even handle it? What are your thoughts?

Rahul: Oh man, this is going to make life a breeze.

Hilary: Oh.

Rahul: For so many developers, AWS is basically taking a step back and simplifying how some of the services are put together by their customers. I mean, you heard me talk about stitching AWS services together time and again, connectors are AWS’s new way of stitching together some of these services.

They take away all that complexity that you hear me cribbing about all the time and they just make it really simple to connect two services together. I mean, this is the latest of the connectors, which is EC2 and RDS going together. These are two services that go together so often and I’m really glad AWS has put this one out.

Hilary: Now so am I. AWS is taking Neptune serverless, which will automatically scale to support variable graph database workloads. This comes five years after the launch of Neptune and it’s getting a lot of attention. Is it getting any attention from you?

Rahul: Okay, so here is the Insiders story and we are calling this Insiders.

Hilary: Oh my God, yes, please.

Rahul: So this serverless announcement, well, it was originally slated to be announced at Reinvent last year.

Hilary: Oh.

Rahul: But I guess they just missed their deadline to get the release out on time.

Hilary: Oh my god. Scandal.

Rahul: And then they ran into a whole lot of issues. So it’s taken them a year to iron out all of those issues and to get it working well. It just goes to show how hard it can be to build a service that caters to millions of customers for mission critical workloads. So coming back to the news item, I mean, this is great because you can now scale up or down your Neptune instances just automatically depending on the size of the workload.

I think the only thing that I am kind of not so pleased about is the fact that you need to have a minimum of 2.5 NCUs, and that’s Neptune Compute Units, and it scales in increments of one NCU. Now while Aurora serverless on the other hand scales up or down in increments of half an ACU or Aurora Compute Units. So that’s my only negative on this.

Hilary: Great.

Rahul: But that said, I love this team. There’s some amazing people and they’ve actually built an amazing service, so kudos to them.

Hilary: Yes, kudos to them. Rahul, I’ve seen the cars you make out of Lego, so this next one’s for you. AWS is partnering with BMW for its new cloud connected cars. The “strategic collaboration,” and I say that in quotes, will develop a customizable cloud-based vehicle data platform. This will allow other auto companies to integrate the jointly created software into their own vehicles. Is this open source automotive automation? I just made that up.

Rahul: That’s an interesting take. But I think the reality is that modern car companies really need to be data companies. And Tesla kind of leads that back. The data tells you how drivers drive. You can customize insurance policies based on that data. Cities can use that data to do better city planning.

With 5G available now, cars can pick the most optimum roots or even avoid each other if they could just talk to each other, right? The possibilities with this data are just endless and I couldn’t be more excited to see some amazing use cases come out of this. I also would not be surprised if this raises a number of privacy issues.

Hilary: What is top of mind?

Rahul: Well, the fact that every driver’s driving information is potentially going to be available to all of these manufacturers is something to look out for.

Hilary: Oh my God, that is horrifying. I’ll never be allowed on the road again. Oh, it was a good run. Okay, let’s get back to this lift and shift versus cloud native apps debate for the ages. What are the critical things to understand about what should go into making this decision?

Rahul: So let’s start with lift and shift. I mean, if you really know your application inside out, it is probably the fastest way to get to the cloud, though you really aren’t leveraging the cloud for all it has to offer. I mean, you can basically replace each data center machine with a cloud instance and you’re all done.

Hilary: Right.

Rahul: So unfortunately more often than not, you have code bases that are over a decade old and they’re trying to lift and shift these. For these cases, the lift and shift is more like the show Flip or Flop on HGTV. I don’t know whether you watched that one.

Hilary: I am not familiar, but I will look it up.

Rahul: So it is the equivalent of buying a property kind of site unseen and you just don’t know what you’re going to uncover. Okay? The lift fin shift exercise helps you discover all the dependencies that make your code base and application more portable.

Hilary: I’m sorry to hear about that experience. But assuming there aren’t a bunch of dusty skeletons in your software closet, then it sounds like lift and shift can actually be relatively fast. Does rewriting your applications as cloud native apps take a lot longer?

Rahul: So here’s the tricky part, Hilary.

Hilary: Oh god, I thought we’d already been through the tricky part.

Rahul: There are always speed bumps and tricky potholes to overcome. 

Hilary: Okay.

Rahul: Recreating enterprise software with all of its intricacies, nuances and idiosyncrasies that were built into it over decades is very, very hard to replicate again. In some ways it is like recreating one of my grandmother’s recipes. I once asked her for a recipe for a dish that I really love and her description meant something like this.

She said, “Well, when your father is around, I add a dash more salt. Otherwise, it’s a pinch. And if your aunt is around, then she likes it a bit more tangy. So I add some tamarind to it. But then when your mother is there, then she prefers tomatoes over tamarind. So that adds the tangy taste and so on.”

And I don’t think I’ve ever got all the permutations or variations of that recipe even today. And enterprise software isn’t very different. Over years they undergo customizations and evolve with their own little quirks. Not recognizing this fact, while undertaking a complete rewrite of the application is probably where most organizations falter. Most of these migrations take forever or just end up unsuccessful.

Hilary: Okay. But what’s the recipe for, just quickly, come on.

Rahul: Okay, so this is a dish called Rasam.

Hilary: Ah.

Rahul: It’s kind of like a soup. If only I can get my grandmother to give me the real recipe.

Hilary: Oh, you give me her number and I’ll call her. I hear what you’re saying. Recreating the past has some real challenges. Also, a dash isn’t a pinch. That is news to me. We’re going to continue this discussion with David Mee, but first I need an audience for a use case. Rahul, you mentioned Flip or Flop, so you obviously know what’s up in the world of hot TV. As a freshly minted television expert, what is the number one show in India and how many people are watching it?

Rahul: Well, I really don’t know much about TV shows or soaps.

Hilary: Oh, come on.

Rahul: But I can tell you this.

Hilary: Yes.

Rahul: India just played Pakistan on October 23rd and 288 million people watched the match just on the Star network.

Hilary: Wow.

Rahul: And on Doordarshan, which is India’s state broadcaster. And this was the data according to TAM, the ratings agency. Now, Disney+ Hotstar, which is the streaming solution from Disney over here, they alone had 18 million concurrent viewers on their live stream. And I know that they were able to do this successfully only because they moved everything to AWS.

Hilary: Okay. Now I’m assuming we’re talking about cricket here?

Rahul: Absolutely.

Hilary: Right.

Rahul: What else?

Hilary: Okay. I want to talk about TAM’s co-owner Nielsen, which is the main measurement service in the US of how many people watch TV shows and how many people watch which ads. Nielsen used to send little boxes to record their viewing habits, which was not that long ago, but still seems incredibly quaint.

Rahul: Yes. But that obviously moved online.

Hilary: Not a moment too soon.

Rahul: Right? No more trouble getting those respondents. But Nielsen then had another problem, which is processing all of this data. I mean, its IT infrastructure could only support surveys from something like 44,000 daily viewers, which is not a lot when you really consider the kind of measurement requirements for online streaming. Take a look at what Netflix or Disney or any of these OTTs got to get. So Nielsen decided it was time, way overdue to head to the cloud and they chose AWS. But then the decision came down to just two things.

Hilary: Okay, let me guess. Lift and shift or go and build cloud native apps.

Rahul: Yes. And they chose the latter of course.

Hilary: Hey.

Rahul: And we are going to hear about how that migration went right after we talk to David Mee.

Hilary: Oh, thank you for the segue. David Mee is the principal consultant for Carbonite’s availability, recovery and migration team. Carbonite is a cloud backup service owned by OpenText. David has spent over 25 years designing and implementing highly complex geographically dispersed availability, disaster recovery and cross infrastructure server migration projects for some of the world’s largest companies. He has never fit that all onto a business card, his one failure. David Mee, welcome to the show.

David Mee: Thank you. Great to be here.

Hilary: We are delighted to have you. You help companies migrate to the cloud. What are the advantages that you associate with lift and shift?

David Mee: It’s quicker, faster, cheaper. In the long run, you could move quickly, get there fast and start utilizing the cloud resources like AWS. From that point, you’ve removed those costs from the on-premise environment where I’m no longer paying for footprint and environmental units and everything else. I’m now in the cloud and I could start refunding my environment so I could start doing the cloud migration or the cloud transformation as needed.

Hilary: Rahul, what do you say to that?

Rahul: I just don’t think lift and shift is a good cloud migration strategy. There are more efficient and more cost effective ways to get there. And that is to pick the competence which are cloud native compliant and just move those to the clouds or go back to the core value proposition of your product and rebuild it to be completely cloud native. Don’t try to rebuild a like-to-like product. That’s never going to happen. And that’s the way you go about it.

Hilary: So if I was considering a cloud native approach, David, what would you tell me?

David Mee: You’re going to be spending money, you’re going to be hiring consultants, you’re going to be transforming your business while you’re paying for your business on-prem. It will take more than a year. There is no conversion transformation that I’ve ever seen, even in the smallest sense that didn’t take months of planning, months of prep, months of coding, months of testing. At the end, you’re not going to transform your business in an annual business cycle.

Your business is not going to stop the day you sign the contracts to transform. You’ll be adding, changing, adjusting, controlling. But at the same time, you’re still paying all this money to run the data center. Oh, by the way, “Dell just called. I need to hardware refresh to maintain my maintenance.” Okay, there’s another $50 million for the new kit. I got a great idea, why don’t we just get rid of the kit, put it in the cloud, and now move to that transformation.

Rahul: If you were to go completely cloud native to leverage cloud technologies, what does that migration plan look like for your customers?

David Mee: My product doesn’t do that, so my product moves lift and shift, we don’t transform. However, I’ve been in a lot of projects where they did the transformation. And how that looks is very simple. Step one, stop spending money on footprint. Step two, plan to convert this to the services. You need a step one and a half.

I need to get out of that platform and onto the cloud. Once I’ve done that, I have this capital expenditure where I can now start transforming and using Kubernetes and using Docker and getting everything using the proper services of the cloud. I don’t want to waste that effort. If I was running my business, I would have a goal.

The goal is very simple, where will my business be in two years, three years, five years? If I start transforming my application and system environment, I better be ready to transform my business. If I’m not transforming my business, if I’m just picking up and moving it to the cloud, I better be prepared to move it back to on-prem once we’ve rebuilt my data center with all new environmentals.

Hilary: Do people actually do that, move their system back after moving it to the cloud?

David Mee: Yeah.

Hilary: That sounds so expensive and inefficient.

David Mee: That’s a very good point. I can’t give you a solid number, but I’ve done migrations where I lifted and shifted into AWS. The clients running an AWS, a year later, they’re seeing that cost go astronomically high because they did not convert. So they move certain portions of their environment back to on-prem.

Hilary: I want to ask you a question, Rahul, because I understand the lift and shift as a step one, but when it comes to this sort of, for lack of a better phrase, posthumous transformation that then needs to happen in the cloud.

Rahul: That’s a good way to put it.

David Mee: I’m writing that one down, Hilary.

Hilary: Rahul, is there a way of doing that efficiently?

Rahul: Yeah. I mean, more often than not, you are tied into these long-term contracts with your data centers and it’s a pain to kind of get out of them. So whatever their end date is, set that as your target for taking out big chunks of your existing product. Figure out how you’re going to carve out things that can be turned into cloud native solutions and then move just those to the cloud. and the other stuff that shouldn’t be there, either downsize your data center footprint or sure, run it wherever else that it might be more efficient to run that stuff.

But lift and shift just feels like a big waste. Let me move to the next aspect of it, which is rebuilding your app for the cloud. My contention really is that that’s kind of the easiest way to spend an inordinate amount of money and time. Because most enterprise workloads are such that they’ve built so many nuances and nitty gritty idiosyncrasies over the years that it’s almost impossible to replicate the application as is. And for most enterprises, that is their goal. That’s what they plan for, that everything will be as is when they’re in the cloud, but cheaper.

Hilary: Rahul says lift and shift doesn’t work. One of the colorful examples he uses is, you wouldn’t build a skyscraper on the foundations of a log cabin, and that’s basically the active description for lift and shift. How and can you refute that, David?

David Mee: Oh, certainly. I don’t know of anybody who wants to build a log cabin in the middle of Manhattan.

Hilary: I know of a couple, but we run in different circles, so that makes sense.

David Mee: Well, yeah. There’s desire and a need. Okay, good point. There’s a desire and a need. But you have to look at your infrastructure and foundation. 100% correct, not everything needs to be lift-and-shifted. However, if I’m building a skyscraper, I’m making sure I have that infrastructure. That’s going to be an expensive long-term goal. Do I want to continue spending for my on-prem or do I just want to move it to the cloud and use that savings to start building the extra? That’s the kind of concept. The decision is not one size fits all.

Hilary: Right. But the response, Rahul, from you is listen, you’ve lifted and shifted and now you’re forced to refactor these apps in the cloud in real time and that has a massive failure rate. Do you see any circumstance where that outcome is different?

Rahul: In most cases, the money that you’re spending on your data center or on your on-prem, whatever the setup might be, it’s typically money that’s already been spent or committed.

Hilary: You’ve both talked about this idea of investing today to save for tomorrow. Is that the wrong way to look at this? Because it sounds like there are lots of cases where it’s just as expensive tomorrow or maybe even more so than it was today, but at least you’re now positioned for the future because you’re in the cloud or that’s the thinking anyway. Do we need to re-conceive of the way we’re thinking about this move in the first place?

Rahul: So I can take the first pass at this one. So…

Hilary: Please try.

Rahul: So in our world, again, dealing with a lot of enterprise software, most of which happens to be B2B, there is a general agreement that if done right being cloud native is orders of magnitude cheaper and more efficient and scalable and elastic as compared to any other solution that’s either on-prem or even just lifted and shifted at the infrastructure level. So that is the end goal done right.

Hilary: Yeah.

Rahul: Now, what does done right mean? Okay.

Hilary: It’s just that done-rightness is hard, right?

Rahul: I am very…

Hilary: Yeah.

Rahul: Strongly opinionated about the fact that trying to recreate your solution to be an apples to apples equivalent in a cloud native way is a no-go. It just does not work. So the general practice of going back and assessing at the business level, what is the core value proposition of your solution, build a completely different cloud-native version of your product that is super efficient, super cheap, and bring that out as a separate solution. While you have this captive customer base, use that to fund this new development work.

David Mee: I do believe the cloud will give you the long-term adaptation of future technologies. If you transform into a microservice, that microservice will progress over time and you could take advantage of it.

Hilary: Right?

David Mee: The difference is where you start spending that cost. I’m a firm believer is let’s get rid of that data center concept. I might be able to convert everything to a data closet because I moved everything to the cloud while you start your transformation. You make a business decision early, that business decision has to include all the tools in the basket. Are you going to one cloud? Are you going to multiple clouds? What service works best? You also have to consider this in different aspects. You might make a decision because you do B2B. You might make a completely different decision if you’re doing B2C.

Hilary: Rahul, your MO is always about getting rid of code as a liability. If you can bring it down to 10,000 lines of code. Rahul, I mean, is it fair to say that’s your sort of your magic threshold?

Rahul: Correct. I think where I see value from being truly cloud native is really the fact that you can replace on an average every million lines of code with about 10,000 lines of code. That’s been my experience with all of these large enterprise offers. So I would say that is the core value proposition you’re after. At the end of the day, it all boils down to costs or dollar value that you associate with it, but that is the business case I would make.

David Mee: Optimizing your code, whether it’s 10,000 lines or 100,000 lines, optimizing your code, reducing that code overhead has benefits. However, getting back to the era of ERP migrations, there was value in those million lines of code that separated you from your competition. If everybody gets into a homogeneous view of the environment, you’re not going to have that competition.

And the first thing that’s going to happen is somebody’s going to say, “Can you make this bluer or can you make this little widget better?” So you’re going to start writing code again. It’s the sprawl, application sprawl, server sprawl. It happens. Over time, you need to separate yourself from the competition. If everybody’s doing the same thing, you’re not separating yourself.

Hilary: David, thanks so much for joining us.

Rahul: It’s been an absolute pleasure, David.

David Mee: My pleasure.

Hilary: Rahul, let’s say you’re eyeing a migration to the cloud, what are your top tips?

Rahul: Okay, so here it goes. First, lift and shift is not a cloud migration strategy. It’s a data center migration strategy. And at best, it’s an audit tool. Second, there’s a saying in enterprise software that the easiest way to go bankrupt is to rewrite your app from scratch. And they aren’t wrong about it.

So rewrite the app from scratch only if you’re okay with bankruptcy being a very probable outcome. And then the third is that the best way to leverage the cloud is to rebuild the core value proposition that you had of your app in a cloud native way or replace the core competence of your app with managed AWS services.

Hilary: Fire. Rahul, obviously life started with this podcast, but prior to this podcasting utopia, prior to launching the startup that I’m working on, I don’t know if I’ve ever told you that I used to work in TV back when TV existed. 

Hilary: (Old news clip) What makes a face easier to recognize for you?

News guest: (Old news clip) If it was very distinctive.

Rahul: I’m familiar with your work, Hilary. I mean, you were a reporter and a producer on many shows, weren’t you?

Hilary: Many shows. And every morning we would get the overnights. Those are a measurement of how many people were watching provided by Nielsen, and those numbers could literally make or break a show.

Rahul: I could quite imagine.

Hilary: Yeah, you wanted those numbers to be high and people would crowd around them every morning. In our use case, Nielsen found itself playing catch-up when its IT infrastructure couldn’t process more than 44,000 daily surveys as you pointed out. So it turned to AWS. And Rahul, we’re going to skip through the commercials and just get right into it.

Rahul: Spoiler alert, they’re now processing data from over tens, if not hundreds of millions of daily survey respondents. That translates to something like 250 billion events per day, and that is about 55 terabytes of data. And the data lake can hold something like 30 petabytes at this point.

Hilary: Oh.

Rahul: So yeah, I mean, they did pretty good. The data lake lives basically in S3 and the analytics platform is driven by Redshift, Elastic Map Reduce, or EMR and Lambdas. And the system can automatically scale up or down, thanks to the capabilities of serverless architecture. And processing from one terabyte to six terabytes of data per hour and costing…

Hilary: Wow.

Rahul: Only a thousand dollars per day.

Hilary: Hey.

Rahul: I mean, I can’t imagine how we would ever process this kind of data if it weren’t for these AWS services. And the whole package allows Nielsen’s engineers to spend less time on infrastructure management and more time building new products that can get more and more granular data for Nielsen’s customers.

Hilary: We should probably ask Nielsen to track our show numbers. But frankly, I don’t know that they have the petabytes available. That’s it for us, for now. We’ll be back. You have been listening to AWS Insiders from CloudFix. I’m Hilary Doyle.

Rahul: And I’m Rahul Subramaniam

Hilary: CloudFix is an AWS cost optimization tool. You can learn more about them at cloudfix.com. Check out the show notes.

Rahul: Leave us a review, and please follow us.

Hilary: Oh, and you can reach out to us directly at podcast@cloudfix.com. We’d like to hear from you and we’ll catch you later.

Rahul: Bye-bye.