by Chris Wray, Head of Engineering Growth
Data foundations are a roadmap for how your organisation’s data will be compiled, cleaned, governed and used. A good foundation means the difference between data that supports growth and data that causes headaches.
The best data foundations aren’t necessarily the most complex, nor do they always look perfect on paper. They’re designed around how your teams actually run; built to work in practice and create value for your company in the long-term.
If you’re pressure-testing your own setup, here’s a simple checklist of what good foundations tend to look like.
- Fit the organisation, not a fantasy org chart
There’s no universally correct foundational structure. The right choice depends on how decisions are made, how much autonomy teams need and how quickly the business needs to move.
Generally speaking, your options are:
- A centralised data model for more hierarchical organisations that rely on shared processes and systems
- A federated model that gives teams greater autonomy for more distributed organisations
- A data mesh model for large, complex businesses with highly independent teams
Choose your model based on the reality of how the business operates today. But do so carefully, as a mismatch between the organisation’s operating model and its data foundation model can create more problems than you’re trying to solve in the first place.
- Align across teams and job levels
Data foundations fail when they’re owned by only one layer of the organisation. A mix of seniority levels and departments must be involved in shaping your data foundations, so the metrics, definitions and responsibilities work in practice.
This not only ensures shared clarity on priorities, but it also means everyone – from the head of data to a junior analyst – understands and supports the mission.
- Empower teams to experiment, but protect production
Good data foundations build control into the delivery process from day one. Continuous integration and deployment should be in place for data pipelines and code. Automated tests should run on every push. Production standards should be enforced before deployment.
The goal is to enable experimentation in lower environments while keeping production controlled and governed to your organisation’s standards, giving teams room to test while ensuring only verified work reaches live environments.
- Lay the foundations for growth
A data foundation only proves its value if it works beyond the pilot stage. That means thinking about scalable security, permissions, resilience and auditability early on instead of treating them as problems to be dealt with post-launch.
Too often, these essentials get pushed down the priority list. But fixing them later is usually slower, more disruptive and far more expensive.
- Make governance visible and owned
There are certain characteristics we see repeated across trusted data foundations:
- A clear governance model
- Defined stewardship roles
- A documented data dictionary
- Assigned domain ownership
- Accountability mechanisms that make responsibilities actionable
Together, these elements create a governance framework that ensures data is managed consistently and responsibly across the organisation.
However, governance is about people as much as process. When it comes to your data foundations, everyone should know who owns what, who maintains quality and who steps in when standards slip. Transparent accountability resolves problems faster and builds trust naturally.
Consider using a data catalog to make this accountability visible. It allows users across your organisation to identify owners and see clear metadata about your datasets, such as when they were last updated and what their lineage looks like.
Once assigned, make sure that owned data stays that way. It’s easy to let a dataset or object become orphaned and go months or even years without being updated, perhaps due to a team being shuffled around, individuals leaving the organisation or priorities changing. Don’t be afraid to remove or archive orphaned and unused datasets.
- Measure data quality explicitly
Data quality cannot be assumed. It must be measured to ensure your foundations are holding up. That means having:
- Defined metrics
- Tracked null rates
- Validated value ranges
- Monitored anomalies
- A clear yardstick for good versus bad data
The principle is simple: if you cannot measure it, you cannot govern it. Without concrete signals, many issues stay invisible until they cause bigger problems downstream.
It’s worth considering building versus buying when setting up your measurement. Building means getting exactly what you want, but with a longer time to value; buying means seeing value earlier, but potentially not getting exactly what you need.
Consider tools you might already have available like Microsoft Purview (Azure) or the Knowledge Catalog for BigQuery (GCP); and there are other options, like Monte Carlo, if you want a full suite off the shelf.
- Think about the long-term
Governance is generally a negative word in a lot of organisations, so expect pushback. The key to countering this is to make the benefits clear without being overbearing.
The strongest data foundations don’t just support one dashboard, migration or AI initiative. They make the whole organisation easier to run by improving data quality, reducing duplication, speeding up onboarding and creating a more dependable base for whatever comes next.
That’s where the long-term value really sits: not in a single project, but a platform that keeps paying back over time.
A simple reality check
Data foundations are much easier to design than to repair. Adding governance and operating rules later often creates complexity that could have been avoided entirely.
Would your current data foundation still hold up if:
- The business doubled in size?
- Your operating model changed?
- Priorities shifted?
If the answer is yes, you’re probably in good shape. If the answer is no, the foundation may be helping today’s project, but it’s not yet supporting tomorrow’s business.
That’s where Optima comes in: we help organisations build data foundations that are practical, well-governed and made to last.
