How to Create Golden Records for Microsoft Fabric and OneLake
Fabric gives organisations a strong place to prepare and use trusted data. Golden records solve the more specific problem of deciding which real-world entity those records actually describe.
The word “gold” already has a very specific meaning in Microsoft Fabric. In the medallion architecture, Bronze is raw data, Silver is cleaned and conformed data, and Gold is the curated layer designed for analytics, reporting and AI.
Short answer
A golden record is the trusted representation of a real-world business entity created by combining and reconciling records from multiple sources.
In a Microsoft Fabric environment, CluedIn can perform that mastering work by identifying records that describe the same customer, product, supplier, asset or other entity, resolving conflicting values, improving quality and retaining the evidence behind the result.
Fabric provides the data platform. CluedIn establishes which business entity the records describe and what the trusted version should look like.
Gold is not the same as golden
A Gold layer and a golden record solve different problems
Microsoft recommends medallion architecture as one way of organising data in Fabric and OneLake. Raw data enters the Bronze layer. Silver data is cleaned and conformed. Gold contains curated, business-ready data designed for downstream consumption.
That is an architectural pattern for preparing data for use. Master Data Management is concerned with something more specific: identity.
CluedIn resolves CRM, ERP and billing records into a trusted golden record and publishes the mastered data to Microsoft Fabric and OneLake.
Imagine three records arrive from different systems:
CRM
Customer: Northwind Ltd
City: London
Email: info@northwind.co.uk
ERP
Customer: Northwind Limited
City: London
VAT Number: GB123456789
Billing
Customer: Northwind Ltd.
City: London
Email: accounts@northwind.co.uk
A data-engineering pipeline can standardise the formatting, clean the addresses and move all three records through Bronze, Silver and Gold. It still needs some way of deciding whether they describe one company or three.
If they are the same company, someone also needs to decide which values should survive and preserve the fact that those values originated from different systems. That is the golden-record problem.
More than a merged row
What actually goes into a golden record?
CluedIn does not simply overwrite several source records with one new row. The source records continue to matter. They provide evidence about where each value came from, when it changed and why the mastered entity currently looks the way it does.
CluedIn groups records that describe the same real-world entity into a golden record. When values disagree, survivorship rules determine which value should be presented as the mastered value. That can be straightforward. Perhaps the newest valid address should win. Or CRM should take priority for customer contact information, while ERP remains authoritative for legal and financial attributes.
Different attributes can have different rules because real enterprise systems rarely have one universal source of truth.
In CluedIn
CluedIn’s default survivorship behaviour prioritises manually added changes and otherwise selects the most recent value when different sources contribute conflicting values. Survivorship can then be tailored to the needs of the data domain.
Golden-record screen showing mastered properties, contributing sources and provenance.
The mastering process
How does CluedIn create golden records for Fabric?
The process starts with the records themselves. Data may originate in Fabric, ERP systems, CRM platforms, operational databases, SaaS applications, files or other enterprise sources.
Once those records reach CluedIn, they are mapped into business entities and identifiers. CluedIn then works out which records describe the same thing. Some matches are obvious because the records share a reliable identifier. Others need deduplication logic that considers combinations of attributes and similarity. When records are matched, they can be merged into the same golden record.
The golden record is then built from the contributing source records, with survivorship logic deciding which conflicting values should be presented. CluedIn keeps the source history rather than hiding it. Relationships between golden records can also be retained, which matters when the business context is more complicated than a flat customer or product table.
Context matters
A supplier can belong to a parent organisation. A product can sit in several hierarchies. An asset can relate to a site, contract and manufacturer. Those relationships become part of the trusted entity rather than something that has to be reconstructed later.
Golden records keep moving
What happens when the underlying data changes?
This is one of the areas where the idea of a golden record is often misunderstood. A golden record is not a project deliverable that gets created once and then forgotten. Source data keeps changing.
Customers move. Products gain new attributes. Suppliers merge. New systems are introduced. Existing systems disagree with each other. Better information arrives tomorrow than was available today. CluedIn can reprocess the golden record as new source records and values arrive.
Trusted master data needs to behave like an operational capability, not a one-off cleansing exercise.
Can OneLake store golden records?
Yes. OneLake can be where mastered data is made available for wider consumption. The important distinction is that OneLake stores and serves the trusted result, while the identity resolution, survivorship and mastering logic required to create that result happens upstream in an MDM platform such as CluedIn.
Back into the Microsoft estate
Publishing golden records into Fabric and OneLake
CluedIn provides Streams for selecting and publishing golden records to downstream systems. For Microsoft Fabric specifically, CluedIn also supports Open Mirroring as a route for making selected mastered data available inside Fabric. That makes the overall pattern fairly simple:
Source systems → Fabric and/or CluedIn → Entity resolution and mastering in CluedIn → Trusted golden records → Fabric / OneLake
The result is that downstream teams can work with the mastered entity instead of repeatedly rebuilding matching, deduplication and survivorship logic for themselves.
Enterprise source systems, CluedIn mastering and golden records, then Open Mirroring or Streams publishing the trusted result into Fabric and OneLake.
Build or manage
Why not build the golden record directly in Fabric?
You can build a lot of this logic yourself. Data engineers can write joins, matching rules, transformations and pipelines. They can create curated customer dimensions and build increasingly sophisticated logic around them. For some projects, that is perfectly reasonable.
The difficulty appears when the logic becomes an ongoing business capability rather than a pipeline. Someone needs to review uncertain matches. Survivorship rules change. New sources arrive. Data stewards need to investigate why a value was chosen. Records need to be split after an incorrect merge. Relationships need managing. Quality rules evolve. History and provenance become important.
CluedIn is designed around those jobs.
What CluedIn retains
A CluedIn golden-record page can expose mastered values, metadata, lineage, relationships, publishing information, duplicate suggestions, pending changes and full record history. That context is what makes the record manageable after the initial match has been made.
It should be possible to answer some fairly basic questions.
Where did this value come from?
Which source records contributed to this entity?
Why did this value win over another one?
What other business entities is this record related to?
Has this record been merged before?
Where is this mastered record being published?
What changed?
If the only thing left at the end of the mastering process is a flattened table with no explanation of how it was produced, some of the most valuable context has already been lost. That matters for governance, but it matters just as much for AI.
An AI agent working with a trusted customer entity is far more useful if that identity has already been reconciled and the business context behind it is understood.
Architecture
Where should golden records sit in a Fabric architecture?
There is no need to turn this into an argument about whether data belongs in Fabric or in an MDM platform. They are doing different jobs.
Fabric and OneLake provide the environment where enterprise data can be engineered, analysed, shared and consumed at scale. CluedIn provides the operational mastering layer that works out which records describe the same entity, what the trusted representation should be and how that entity should be maintained. The mastered result can then flow back into Fabric for wider consumption.
The useful distinction
Gold data and golden records belong together
A Gold layer tells downstream consumers that data has reached a curated stage in the analytical architecture. A golden record tells them that the underlying business entity has been resolved and mastered. You often want both. Fabric gives organisations a strong place to use trusted data. CluedIn provides the specialist Master Data Management capabilities required to create and maintain the trusted entities inside it.
For organisations bringing several operational systems together in Fabric, that distinction quickly stops being theoretical. It becomes the difference between having more data available and actually knowing what that data represents.
Fabric provides the data platform and OneLake provides the common data foundation. CluedIn resolves the identity behind the records, creates and maintains the golden record, and publishes the trusted result back into the Microsoft data estate for analytics, reporting and AI.
Create trusted golden records for your Fabric estate
See how CluedIn resolves fragmented enterprise records into trusted business entities and makes them available to Microsoft Fabric and OneLake.