<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=4011258&amp;fmt=gif">

Articles

Cluedin articles

Posts about:

Data Modelling

The Future of Master Data Management: Insights and Innovations with CluedIn

CluedIn

Executive Summary

The ability to effectively manage and utilize data is more important than ever. Master Data Management (MDM) has become a cornerstone for organizations seeking to achieve a unified, accurate, and comprehensive view of their critical data assets. The recent Gartner Market Guide for Master Data Management Solutions 2024 provided valuable insights into the current trends and strategic recommendations that are shaping the future of Master Data Management.

As a prominent player in the MDM space, CluedIn is uniquely equipped to address these trends and deliver innovative solutions that drive business success.

  • Embracing Cloud-Native Architectures:
    • Cloud-native MDM solutions offer unparalleled flexibility, scalability, and integration capabilities.
    • CluedIn's graph-based MDM solution ensures seamless integration and real-time data synchronization.

  • Leveraging AI and Generative AI:

    • AI and GenAI revolutionize data management by automating tasks, enhancing data quality, and providing predictive analytics.
    • CluedIn's AI-powered features reduce manual intervention and generate valuable insights.

  • Reducing Deployment Time Frames:

    • Rapid deployment is critical, with modern MDM solutions offering streamlined processes and agile implementation.
    • CluedIn's efficient deployment process ensures quick time to value, with MVPs ready in under 60 days.

  • Supporting Multiple Domains and Use Cases:

    • Versatile MDM solutions must support various domains and use cases, from customer and product data to ESG reporting.
    • CluedIn's multidomain capabilities provide a unified view of critical data across different domains.
       
  • Enhancing ESG Reporting:

    • Integrating ESG metrics into MDM strategies enhances transparency and accountability.
    • CluedIn's solution supports accurate and comprehensive ESG reporting, meeting regulatory requirements.
       
  • Driving Business Agility with Augmented MDM:

    • Augmented MDM combines traditional capabilities with AI and machine learning, reducing manual tasks and generating insights.
    • CluedIn's augmented MDM capabilities enable adaptive and context-centric data management.
       
  • Ensuring Data Quality and Governance:

    • High data quality is maintained through automated cleansing, standardization, and enrichment processes.
    • CluedIn's robust governance framework supports policy enforcement, compliance, and data stewardship.
       
  • Facilitating Seamless Integration:

    • Seamless integration with other data management and analytics tools is essential for a unified data ecosystem.
    • CluedIn's extensive library of connectors and APIs ensures interoperability and enhances collaboration.
       

By focusing on these key areas, organizations can navigate the dynamic landscape of Master Data Management and unlock the full potential of their data. The insights and recommendations provided in the Gartner Market Guide serve as a valuable resource for businesses looking to adopt innovative solutions and drive strategic initiatives with confidence.

Read More

MDM as the Foundation for ESG Data

CluedIn

In the era of sustainability and social responsibility, organizations are increasingly focusing on Environmental, Social, and Governance (ESG) initiatives. These initiatives not only demonstrate a commitment to ethical practices but also offer a pathway to long-term value creation. However, to effectively manage ESG strategies, accurate, consistent, and comprehensive data is crucial. This is where Master Data Management (MDM) plays a transformative role, serving as the backbone for successful ESG initiatives.

Read More

The Most Common MDM Myths. Debunked.

CluedIn

During its 30-year history, many myths have arisen around Master Data Management (MDM), some of which may have been valid in the early days but are no longer true today. MDM has come a long way, and is experiencing something of a renaissance as progressive platforms take advantage of technical innovations like Graph databases and Generative AI, and offer increased commercial flexibility via the major CSPs.

Join us as we expose the eight most common MDM myths below, and discover how MDM has developed to become more accessible, manageable, and impactful.

Read More

Upfront versus Zero Upfront Data Modelling

CluedIn

Traditional Upfront Data Modeling

Traditional upfront data modeling in MDM involves the creation of a comprehensive data model that defines the structure, relationships, and constraints of master data entities. This approach requires a thorough understanding of the business domain, data requirements, and anticipated data usage scenarios. The key characteristics of traditional upfront data modeling include:

Read More
green Data Mesh graphic

Data Mesh: The Next Frontier in Master Data Management?

CluedIn

Data is difficult to manage, especially master data. Master Data Management is essentially the process of controlling the way you define, store, organize and track information related to customers, products and partners across your organization. Traditional data modeling requires that all business entities are defined in advance and cannot change throughout the life of the enterprise system; this limits flexibility and prevents you from easily integrating new systems with existing ones. What if you could handle master data management in a more flexible and agile way? Could it make managing master data much easier?

Data Mesh is a relatively new approach to mastering data, and if its advocates are to believed, could represent the future of master data management.

What is Data Mesh?

Data Mesh is a platform architecture that takes a decentralized approach to managing data. Fundamentally, it’s about treating as something that is separate from any single platform or application. Under the Data Mesh philosophy, data is treated as a democratized product, owned by the entire organisation and managed by domain-specific teams. Each team is responsible for the quality, governance and sovereignty of its own domain, and this data is then provided for use by the rest of the business.

What problems is Data Mesh trying to solve?

The theory behind Data Mesh is a noble one. In essence, it is seeking to solve the fundamental and pervasive issues that have dogged traditional Master Data Management systems for years. In order to realise any value from these initiatives, organisations have been forced to wrestle their data into rigid structures and compositions before they can even get started. Cleaning, preparing and integrating data is hard, time-consuming and expensive, which is why so many Master Data Management projects fail before they've even begun. It's not uncommon in a traditional MDM project for it to take six months to get just one domain operational. Six months! Considering the speed at which customers, markets and competitors change, this timeline simply isn't acceptable. By rejecting the heavily centralized and monolithic models of the past, Data Mesh is trying to do a good thing. The question is whether Data Mesh is a practical alternative.

The challenges of Data Mesh

Despite the buzz around Data Mesh, there are some fundamental questions that need to be answered before you can decide if it's right for you. Firstly, you'll need to consider whether your organisation is operating at a scale at which Data Mesh makes sense. Complete decentralization of data brings its own challenges and risks, as it depends on each of those domain-based teams having the necessary skills and experience to manage the data they are responsible for. Without some level of centralised management, data silos, duplication and governance issues will inevitably arise.

Data Mesh is also a more expensive option. If we accept that every domain-based team will need people with some level of data management experience, and that there needs to be another team to oversee them, suddenly this looks very expensive. For all but the largest of organisations, Data Mesh is most likely cost-prohibitive.

It's not just about cost, it's also about value and building a compelling business case. In theory, federated ownership should lead to quicker learning cycles and accelerated ROI. In practice, if each domain-based team needs to invest in some level of data analytics and engineering expertise before they've even procured any technology, it's going to be very hard for some departments - let’s say the HR team which is responsible for Employee Data - to justify and build a compelling business case.

Which brings us to another point. You cannot buy a Data Mesh. There is no off-the-shelf product that will enable a federated approach to data ownership. It is about so much more than technology and tools. It is a topology, a guiding principle, and in order to realise its value it requires a mindset change and cultural shift that many organisations simply are not ready for.

A means to an end

We've established that Data Mesh is not right for everyone. But that doesn't mean what it seeks to achieve is wrong. Quite the opposite, in fact. There is only way to allow organisations to use their data in ways that will actually help them to adapt to market shifts and serve their customers and stakeholders better. And that is to rip up the current master data management "rulebook" and start again.

It is possible to share data ownership between the IT team and business users without requiring the latter to be data engineers or scientists. It is possible to automate manual tasks like data cleaning, enrichment and integration and save hours of time and significant sums of money. Most importantly, it is possible for sets of data to be treated as products which are universally useful across the business. Truly democratized data can only come from a platform that benefits the many, not only the few - as is the case with Data Mesh. The future therefore lies in a modern approach to Master Data Management that has the same ambition as Data Mesh, but which makes the means of achieving it accessible to all.

Read More
paper aeroplane icon on pink and green gradient background

Breaking down the barriers to entry for MDM

CluedIn

There are always hurdles (many are necessary) to starting any data initiative. Do you have the right team, the right technology, the right budget, and more. In fact, we are really underplaying it here, there are literally 100s of decisions that need to be made before kicking off an internal data project. One of our big focusses at CluedIn is to help you limit the number of hurdles in a positive and constructive manner.

Here’s a couple of good examples. Imagine trying to buy technology and with a budget of $10,000, and the technology you want is $9,000. This is a good example of a hurdle that doesn't exist. If it was $11,000, then suddenly, many months of effort has potentially been added while you figure out how to secure a bigger budget or how to negotiate with the vendor to discount the price. This isn't always a quick process.

At CluedIn, we considered the entire sales process from the point of view of the customer, and put in a strategy to make sure that WE, as the vendor, are removing as many hurdles as possible. Let's dive in and look at some of the options we’ve put in place to make this possible.

Self-install, start when you’re comfortable.

I am an engineer at heart. I have been a software engineer for the last 15 years and I can speak only for myself when I say that I need to use software before deciding whether it’s a good fit. I also realise that when I do this, I really only get a 25% view of what that software is actually capable of. At CluedIn, we have recently added support to deploy CluedIn directly through the Microsoft Azure Marketplace in a Managed Application Offering. This makes CluedIn dead easy to get started with by offering the data sovereignty of PAAS, with the ease and scalability of SAAS. I can get a rough idea of the platform, but am quite happy to accept that some things might not work as I expect or it may not reflect the final nature of what I will be getting.

Start sending data to CluedIn straightaway - no need for upfront modelling.

Let's be clear, having a plan makes so much sense - you should plan. But, you do not need your data model to be perfect and future-proofed before you implement MDM. If you wait until you do, you will literally never start because the perfect data model is a flawed and impossible concept. CluedIn's data model was designed around the idea of source control. This might get a bit technical, but it is literally the best analogy to equate to the way CluedIn stores, changes and processes data. You don't build code with perfection in mind. You evolve it and sometimes you might even fundamentally rewrite something. Without a doubt, source control systems like GIT have proven that they can manage huge repositories in any type of change that you could expect now and not even expect to happen in the future. We provide the same with data. At CluedIn, you do not need to perfect your data upfront, you can delay it until a point when it makes sense. This cannot be said about the majority of MDM systems. At CluedIn, although you will benefit from mapping the data on entry, and mapping the primary and foreign keys, it is not enforced. What is the value of this? Should this not be the time to actually map this data? The answer is, categorically, no. There are so many benefits one can get from simply placing data into CluedIn such as Data Quality Metrics, Data Lineage, Sensitive Data scanning and Data Sharing. Once you are ready, going back to the mapping of data and updating it will simply require the data to be reprocessed and CluedIn will handle the change on your behalf.

No need to complete your Data Governance program before you start.

CluedIn is different – think of it like the Agile alternative to Project Management. You can build and discover and modify and adapt along the way.

Let me reiterate, a plan is a good thing. But at some point, overplanning leads to a lack of agility to change in the face of necessary change. CluedIn is an Agile MDM platform, in that it expects change, it expects things to go wrong and is prepared for that. Which means that bumps in the road will not fundamentally kill a project with CluedIn.

Let business rules evolve over time.

We have mentioned this in previous posts, but CluedIn removes a huge hurdle from common MDM initiatives, which is to develop business rules to either detect and identify possible data quality issues or to setup rules to invoke an action once a certain condition is met. Instead of asking you upfront to manually develop these rules, it turns out that most of the rules that you actually want to build will come from working with the data, allowing the issues to surface and then putting the proper fixes in place. In the majority of cases, you won't be able to develop these proactively, but reactively. CluedIn embraces this idea, by onboarding data into the platform, and then using surfacing tools to help detect and automatically place business rules in place to fix the existing issues and prevent that problem from making its way through the system ever again.

Zero downtime upgrades.

Let’s face it, upgrading software is a massive ****ache in an enterprise environment. That complexity is escalated when you have software that is more of a "platform" that allows you to extend. Now that CluedIn offers generic, REST-based extension points in the latest version, it makes the process of upgrading painless. CluedIn can be setup to auto-update or you can opt-in and manage it yourself, giving you the choice. Considering the core of CluedIn is based off a schemaless data model, with support for reprocessing, then any actions that need to be triggered on new updates can be automated as well.

Auto-scaling across the entire cluster i.e. scales disks, CPU, RAM, network.

In the true spirit of CluedIn, we are not interested in providing a solution that is faster and better. We attempt to remove the need to actually do something in the first place. CluedIn is designed and setup to auto-scale according to your business drivers. Typically these factors will either be working towards a particular time and date, or a particular budget, or even "spend as much money as possible to get the job done as fast as possible." In saying that, although all of the above is possible, it still needs to make economic sense in most cases.

Native integration to 27 Azure services in just a few clicks.

Even with a multi-cloud strategy, native integration to the cloud provider you are hosting your platform in, is without a doubt, hugely valuable. CluedIn is focused on being the most native MDM solution on Microsoft Azure. Sure, we work and have many customers on the other cloud platforms, but on Azure we are easily the most native and obvious choice due to the number of native integrations we support. Want to use Azure Active Directory for authorization, SSO and authentication? One click away. Want to enable Azure Defender, Azure Sentinel? One click away. Want to share mastered and cleaned data from CluedIn to Azure Synapse, Azure DataBricks, Azure Machine Learning Studio? One click away. Want Azure Purview to register and govern all the data movement in CluedIn? One click away.

We want to provide a “think it, done!” type of experience for Microsoft Azure customers. If you have an idea, you should be able to make it a reality within moments, not weeks.

Kubernetes backbone means support for all environments.

It is well known that in the MDM space, many leading vendors take months to just install and setup. Without a doubt, the future of infrastructure is containers and Kubernetes. Kubernetes brings an abstraction that isolates environments, operating systems, and more. This essentially lowers the entry barrier, due to the abstraction, but also due to the native support for all cloud providers. In addition to this, Kubernetes brings some of the pieces expected for modern, enterprise applications such as auto-scaling, zero-downtime upgrades, and more.

Endorsed by Microsoft – already!

Just as we are investing in our Microsoft relationship, Microsoft is also heavily investing in CluedIn. CluedIn was one of the first applications to provide the Managed Application Offering for MDM on the Azure Marketplace. This provides a great combination of security and data sovereignty, combined with the beauty of a managed service.

Built for the enterprise - i.e. Logging, SSO, Telemtry, SSL, DNS, Inbuilt backups, Budget Allocation (scale to budget), Azure Defender, Azure Sentinel.

Just like knowledge of a particular industry will accelerate implementation, knowing what is expected by enterprise customers is also crucial to generating and sustaining momentum. At CluedIn, we know our customers intimately and have our finger on the pulse of what they will expect in the future from enterprise applications. CluedIn has native support to provide logging, SSO, Telemtry, SSL, DNS, Backup/Restore, and more. We have developed this not only from the teams experience, but also by monitoring what the cloud providers are enabling, as well as what customers are asking for during the purchase cycle - e.g. "What does CluedIn provide in terms of threat-detection?".

Accelerators for all industries, and partners that know your space.

Different sectors, industries and verticals require specific domain knowledge. After implementing and being part of over 40+ MDM implementations myself, I can say that each industry has specific identifiers that are known only to its industry e.g. NPI in Health. At CluedIn, we have vertically aligned partners that specialise in implementing MDM for particular sectors. These partners come with their own additions and pre-built packages for CluedIn in the shape of existing Domains, Vocabularies, Connectors to Systems, Enrichers (public datasets) - and that is just from the technology side.

A clear comparison between other MDM vendors and CluedIn.

All MDM vendors are different. Considering that MDM is a well-established industry, it is important to help our customers understand the revolution that Modern MDM has brought. For this, CluedIn provides many layers of research into what the main differences are between Modern MDM providers like CluedIn and traditional MDM providers. I would like to use another example of a shift in technology that has also drawn a clear line in the sand around modern and traditional approaches, and that is the Data Warehouse space.

The modern Data Warehouse makes a fundamental shift at a very low level that essentially optimises files for read access and then distributes jobs across multiple machines to answer a question. In addition to this, it has taken full economic advantage of the scalability of the cloud in that you can spin up a huge number of machines to run distributed computing at relatively low cost.

The same revolution has happened in the MDM space, driven predominantly by the shift to the cloud. I know this sounds like a cliche, but building for the cloud is a fundamental difference. The other big revolutions in MDM have been through either the automation or augmentation or processes that were complex with traditional MDM software.

Migrate from other MDM systems with ease.

CluedIn offers specific services for customers wanting to easily migrate from a traditional MDM solution to CluedIn. We can provide this through our partner network, in conjunction with our own team that has a plethora of experience and expertise in many MDM solutions. CluedIn has a generic framework for translating data, models, business rules, workflows, hierarchies and more.

Get started with a free trial.

Although not unique in the majority of categories, in the MDM space, free trials are really a rarity. Why is this? Well, I can only speculate, but our opinion is that MDM is hard to implement, and although I would like to stand here and say that has been solved, I don't think it has, it is still hard to implement ANY MDM system. It’s just not quite as hard with CluedIn!

Pricing that works for you.

Essentially there are two ways of purchasing software. It is either a Capital Expense (cap-ex) or an Operational Expense (op-ex). Each of these has their own advantages and disadvantages. This is why CluedIn offers both cap-ex pricing (upfront payment, yearly recurring.) and now per hour pricing (consumption based).

Hourly pricing means that you only pay when you are using the platform. This increasingly suits companies that want to avoid hefty upfront investments. On the flipside, with a capital footprint, you can often get quite heavy discounts. This is simply because vendors like CluedIn need some level of predictability. The operational footprint is fantastic for removing hurdles to getting started, but the caveat to this is that customers won’t be offered the same discounts. Why? Because there are literally 100 reasons why a project might not start, could or could be delayed. And the technology might only be one of them. In saying that, a combination of both the cap-ex and op-ex model can be very powerful, particularly when you start with a consumption and move to a commitment once trust has been established.

Buy CluedIn under your MACC agreement with Microsoft.

Finding budget is hard! It may be that your organization already has an agreement with Microsoft in regards to Azure spend. This commitment means that you have been given a discount on Azure services in return for a commitment on many years of agreed revenue. You can use this to purchase CluedIn.

Buy CluedIn under the standard Microsoft Ts&Cs.

When buying enterprise software, you can't just choose any software, it needs to meet the legal requirements of the business. Chances are you have already signed the Microsoft Standard Ts&Cs if you have bought off the Azure platform before. Hence, CluedIn can be bought under the same Ts&Cs.

Read More
light painting ripples

The past, present and future state of Master Data Management

CluedIn

Master Data Management (MDM) is a discipline that involves managing and governing master data in order to support decision-making, analytics, compliance, and customer engagement. MDM aims to ensure that master data is accurate, consistent, and complete across various systems and applications within an organization. In this article, we will explore the history of MDM, where we are today, and what the future holds.

Read More