April 28th, 2022 | 16 min read
Why your Data Management Platform should be cloud-native
The phrase cloud-native gets banded around a lot. If we go by the standard Wikipedia definition, cloud-native computing is an approach in software development that utilizes cloud computing to "build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds".
Does this definition really capture what it means to be cloud-native? We don’t think so. We believe there is a fundamental difference between this definition and what it is to be truly cloud-native.
Cloud-native goes so much deeper than just scale. If we take CluedIn, a cloud-native Master Data Management platform, we can identify ten characteristics that identify it as cloud-native. We would argue that if your data management platform cannot meet these requirements, then not only is it not cloud-native, it is also falling short of delivering against modern MDM requirements. Let’s examine each in turn.
1. Getting your data-initiatives started is faster and easier
Your first experience of a cloud-native platform should be how easy it is to purchase, install, setup, and get started. A cloud-native platform installs in your own cloud tenant (e.g. Azure) with one click, a few details on a form, and that is it. You can install custom settings such as your own HTTPS certificate, or custom domains, but the experience is straightforward and simple. Long gone are the days of tedious installation processes where you need to worry about what operating systems to choose or to wait for your hardware to arrive.
Cloud-native doesn't mean you are on docker containers and Kubernetes. You can take the majority of platforms, even the old dinosaurs and have them containerised. You can then quite easily have it composed with Kubernetes. This is not cloud-native.
Installing CluedIn will setup the environment in an Azure Kubernetes Service. This makes sure it uses an Azure Key Vault, a firewall and all the best practices that you would expect - all rolled up into a single price. CluedIn can then be scaled in many different ways, including automatically, manually through the Azure Portal, or by redeploying the application with new Helm Chart updates (replicas).
Specifying your environment is all done using box sizes with a known number of boxes and associated cost. You want things to happen more quickly? You pay more – it’s as simple as that.
Cloud-native essentially means that the entire backbone of operations in your platform can run off a cloud provider’s stack. For example, being able to utilize cloud Platform-as-a-Service (PaaS) services to scale out the processing or size or storage amount. Cloud-“nativeness” should feel like using a Software-as-a-Service (SaaS) platform, just in your own tenant.
At the technical level, a cloud-native platform doesn't require containers, but this clearly is the direction that cloud is taking and investments will be made that expect you to be on containers to reap the full benefits of the cloud. Having Kubernetes as the orchestration framework is only the start. To get the best out of Kubernetes, your application needs to be designed in a way that can take advantage of it.
Cloud-native means that costs should be transparent and predictable. Like most companies, you’re probably more than happy to spend money when the costs are clear and the ROI is obvious. What you probably don’t want are hidden or unpredictable costs.
Cloud-native does not necessarily have to mean SaaS. We believe that your most important data should stay in your environment, but to take full advantage of the cloud cost model, multi-tenancy is needed. The future of data sovereignty is having cloud providers offer smart ways of slicing the same digital assets and guarantee the tenancy and isolation. There is no easy technical way to solve this, it is more about trust - i.e. do you trust a particular vendor – say CluedIn - to host your data on your behalf? This is not the same as asking you to trust Microsoft with your data. Microsoft have thousands of full time Infrastructure and DevOps people, CluedIn does not (yet!).
When asked what it means to be cloud-native, an industry veteran replied that "it needs to be multitenant." This is interesting, and not necessarily wrong. Rather it is a clear indication that cloud is about an architecture where data is stored once and consumed separately. Even from different companies.
At CluedIn we endeavour to deliver a SaaS experience, with the benefits of PaaS “under the hood”/ The critical difference being that your master data always stays in your environment - even if it is used in a public cloud.
2. Access to elastic scale is at your fingertips (if your technology stack allows it)
Cloud quotas aside, companies theoretically have access to endless cloud resources on demand. This is immensely powerful in delivering accelerated time to value. But just because you installed an application in the cloud, it does not mean that the technology will take advantage of it (quite the opposite, in fact). With easy access to elastic scale comes a warning, as we have seen some customers’ budgets spiral out of control because they took advantage of the scaling possibilities. There are tools to help with this, but the temptation can be to ignore these rules because of a desire to just get the job done.
3. Other cloud-native services will play nicely with you
Very similar to the architectural wins we saw pre-cloud, the same principle of realising greater efficiencies from services that are co-located applies in the cloud.
Although companies embark on many different data-driven initiatives, they typically tend to fall into a handful of buckets. These include Reporting/Business Intelligence, Data Science and Process Automation. The majority of these services are all fuelled by data. Having the sovereignty of your data on the same backbone means that the insight-generating systems can get to the data more easily.
4. Cloud-native systems work with the economics of the cloud.
Moving to the cloud can be a double-edged sword, with scalability, flexibility and interoperability gains being counterbalanced by escalating costs. The cloud only makes economic sense if it is being used in the way it was intended to be. For example, elastic compute is only more efficient when you have the ability to scale down as well as up. You should be wary of tools that have made their way to the cloud rather than having being designed for it in the first instance. This is because for an existing on-premises platform to truly become cloud-native, it will inevitably need a major system overhaul.
One of the benefits of choosing a cloud-native platform is there is a better chance that the architecture of the platform is more sound and suited to that environment. That is a huge generalisation, and it does not mean that architectures that were built pre-cloud are less robust. Rather it means that architectures built for the cloud should inherently be more robust as a result of the economic model imposed by cloud platforms. The first fundamental element of designing for the cloud is to properly separate state from stateless. This is critical to allowing parts of your application to scale up and down. Your application needs to have parts of the platform which can work with 100 instances or none at all. This is the stateless part of your application. As for the state part, it is very cloud-like to have your data persisted in cheap storage. The benefit of this is that when your platform is not being used at all, you pay either little or no money. A truly cloud-native solution will cater for:
- Unexpected network latency (the time taken for a service request to travel to the receiver and back).
- Transient faults (short-lived network connectivity errors).
- Blockages by long-running synchronous operations.
- A host process that has crashed and is being restarted or moved.
- An overloaded microservice that can’t respond for a short time.
- An in-flight orchestrator operation such as a rolling upgrade or moving a service from one node to another.
- Hardware failures.
5. The majority, if not all, of the R&D investment is going into the cloud
The majority of technology vendors today are investing in making sure that their offering is suited to and available via the popular cloud providers. In fact, many providers are dropping support for their on-premises and non-cloud-native solutions. This is because the cloud offers the best possible benefits for all players involved, including the customer, the vendor and the market. A properly cloud-native platform is working towards providing customers with a solution that is easier, cheaper and faster.
6. Cloud-native embraces and welcomes the idea of huge data.
For years companies and vendors alike have grappled with the challenge of managing huge amounts of data. One of the benefits of major technological advances is that they don’t just make something easier or better, they eradicate the need to even think about it in the first place. The properly architected cloud solution acknowledges and takes advantage of the fact that data storage is cheap. If you look at the breakdown of the costs associated with CluedIn, 2% is storage. Why? Because storage size and the amount of data should not hinder your ambitions.
7. Cloud-native lets you focus on the things that are important.
Since going cloud-native, we have not spent a single day worrying about the security of our virtual machines in the cloud. We have not spent a single day hardening the boxes that are used in processing of our data. And we have seen the same thing happening with our customers. We realise at CluedIn that our speciality is data and hence we rely on cloud services to manage our firewalls, networks, and more. We focus 100% on providing the best Master Data Management platform on the market, allowing our customers focus on doing whatever it is that they do best too.
8. Cloud-native delivers the required services, as part of the service.
Because CluedIn is cloud-native, our customers inherit several cloud services at no extra cost. Something as simple as cost management is natively available in all services, if they have been designed to take advantage of the tooling. Industry analysts Gartner have developed a full FinOps category, but in our opinion a cloud-native service provides this inherently. In addition, there are numerous services required for security, policy management, performance efficiency, alerts and more. All of these are essential services, that a cloud-native platform should deliver as standard, not as an afterthought.
9. Security, authentication and authorization form part of the ecosystem.
Whether it has come as a result of advances in the technology itself, or it is cloud-native specific, security, authentication and authorization should be ingrained inside the services. Genuinely cloud-native solutions have embraced the idea that access, permissions, authentication and authorization are often provided by third party services. This "context" provides more free flowing access to services, connectivity and discovery once properly authenticated. For CluedIn, being cloud-native means being cloud-aware. This means that the CluedIn platform is aware that it is being hosted in the cloud and sits inside a specific context of security - providing access to the services that sit underneath the platform.
10. Dependencies can become services, allowing products to move faster.
A cloud-native solution supports the idea that users should be able to take full advantage of the native services provided by cloud providers. When CluedIn is deployed into Microsoft Azure it uses the underlying managed disks, the underlying logging, the underlying security and has the option of using the respective database services. This allows our product team to walk in on a daily basis feeling like a team of 400+ developers, because we know that many of the wins we are achieving in our platform come as a knock-on effect of us using the services which huge teams at Microsoft are working on, on a daily basis.
There’s a lot more to being cloud-native than simply being able to scale and run scalable applications. Scalability is a big part of it, but true cloud-nativeness is a mindset and a attribute that runs through the very core of a platform. Most companies are already well on their way to realising the advantages the cloud offers, and there is no turning back the tide. Those who will see the greatest scalability, efficiency and flexibility gains will embrace truly cloud-native platforms and reap the benefits of a cloud-first approach.