Common Enterprise wide Data Governance Issues – #12. No Enterprise wide Data Dictionary.

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.    Assess the status of this issue in your Enterprise by clicking here:  Data Governance Issue Assessment Process

Anyone know what this means?

An excellent series of blog posts from Phil Wright (Balanced approach to scoring data quality) prompted me to restart this series.  Phil tells us that in his organisation, “a large amount of time and effort has been applied to ensure that the business community has a definitive business glossary, containing all the terminology and business rules that they use within their reporting and business processes. This has been published, and highly praised, throughout the organisation.” I wish other organisations were like Phil’s.

Not only do some organisations lack “a definitive business glossary” as Phil describes above, complete with business rules….
Some organisations have no Enterprise wide Data Dictionary.  What is worse – there is no appreciation within senior management of the need for an Enterprise wide Data Dictionary (and therefore no budget to develop one).

Impact(s):

• No business definition, or contradictory business definitions of the intended content of critical fields.
• There is an over dependence on a small number of staff with detailed knowledge of some databases.
• Incorrect or non-ideal sources of required data are identified – because the source of required data is determined by personnel with expertise in specific systems only.
• New projects, dependent on existing data, are left ‘flying blind’.  The impact is similar to landing in a foreign city, with no map and not speaking the language.
• Repeated re-invention of the wheel, duplication of work, with associated costs.

Solution:

CIO to define and implement the following Policy:  (in addition to the policies listed for Data Governance Issue #10):

• An Enterprise wide Data Dictionary will be developed covering critical Enterprise wide data, in accordance with industry best practice.

Does your organisation have an “Enterprise wide Data Dictionary” – if so, how did you achieve it?  If not, how do new projects that depend on existing data begin the process of locating that data?  Please share your experience.

Craig Newmark on Information Quality

Craig Newmark on Information Quality

Craig Newmark, the founder of Craigslist is visiting Dublin at the moment.  He is a keynote speaker at the Dublin Web Summit on Thursday 4th Feb 2010.

I spotted the following quote from Craig about information quality in an interview with the Sunday Business Post, Ireland’s leading business newspaper.

‘‘Large organisations are normally run in a way that people tell their boss what they think their boss wants to hear, and that continues right up the ladder,” he said. ‘‘Because of this, the result is that the people making decisions rarely get good-quality information. In small organisations, commentary and decision-making happens much closer to the ground.”

Craig’s comment struck a chord with me, due to a recent conversation I had with the Data Quality Manager of a large Enterprise.   Senior management in his organisation believe the information quality within the organisation is “OK”.  They believe this because the information “looks reasonable”, and the people who provide the information to them are “good people”.

Given Craig Newmark’s observation, I suspect that the information quality may not be “OK”.

In the future, with the swing from principles to rules based regulation, senior management will need to provide evidence of the quality of the information on which regulatory submissions are based.

I discuss this topic further in two related posts “Achieving Regulatory Compliance – The devil is in the data“, and “What questions will the regulator ask“.   If you get a chance, read some of the comments – they provide  helpful insights – and if you have the time, please comment yourself.

What questions will Regulators ask about data?

Given the failure of Financial Regulators to prevent the financial crisis, the pendulum of regulation is expected to swing from a “principles-based, light-touch” regulation, to a more restrictive rules-based philosophy.

Regulatory Requirements Pendulum swinging towards rules based

Regulation across all industries is likely to follow suit, as regulators seek to ensure their industry is not the next to collapse due to a failure to regulate.

Increasingly, regulators will ask questions about the data underpinning the regulatory submissions, and the governance processes applied to that data.

What questions do you believe regulators will ask?

Achieving Regulatory Compliance – the devil is in the data

I will be sharing my experience and ideas on “Achieving Regulatory Compliance – the devil is in the data” at an IDQ Seminar Series event in Dublin next month.  I would like you to help me prepare.

I would like you to share your past experience with me, your ideas on the current situation, and most important, your view of the future.

Is Regulatory Compliance a mere box ticking execise?

What industry do you work in?

Is regulation increasing in your industry?

Is regulation merely a box ticking exercise?  Does the regulator simply accept what you say.

What role does data quality play?

What role does data governance play?

My initial thoughts are as follows:

• Regulation is increasing across all industries
  e.g. Within Financial Services, the list includes:
  • SOLVENCY II
  • BASEL II
  • Anti Money Laundering AML
  • Anti Terrorist Financing AFT
  • Sarbanes Oxley SOX
  • MFID
• Regulatory compliance is often seen as a box ticking exercise, since it is physically impossible for the regulator to check all the information provided.
• Regulators will increasingly seek to challenge, audit and query the Data Governance processes used to gather the information, and critically the controls applied within those processes.  (I have written a series of posts on common Data Governance Issues – see Data Governance Issue Assessment Process)

I hope to write a number of posts expanding on the above ideas.  My argument is that “To achieve Regulatory Compliance, the devil is very definitely in the data, but the evidence is in the Data Governance process”.

Whether you agree, or disagree, I would like to hear from you.

The truth the whole truth and nothing but the truth

I enjoyed the good-natured contest (i.e., a blog-bout) between Henrik Liliendahl Sørensen, Charles Blyth and Jim Harris. The contest was a Blogging Olympics of sorts, with the Great Britain, United States and Denmark competing for the Gold, Silver, and Bronze medals in an event called “Three Single Versions of a Shared Version of the Truth.”

I read all three posts, and the excellent comments on them, and I then voted for Charles Blyth.  Here’s why:

I worked as a Systems Programmer in the ’80s in the airline industry. Remarkable as it sounds, system programmers in each airline used to modify the IBM supplied Operating System, which was known as the Airline Control Program (ACP), later renamed as Transaction Processing Facility (TPF).  Could you imagine each business in the world modifying Windows today? I’m not talking about a configuration change, I’m talking about an Assembler code change to the internals of the operating system. The late ‘80s saw the development of Global “Computer Reservations Systems” (CRS systems) including AMADEUS and GALILEO.  I moved from Aer Lingus, a small Irish airline, to work in London on the British Airways systems, to enable the British Airways systems share information and communicate with the new Global CRS systems. I learnt very important lessons during those years.

1. The criticality of standards
2. The drive for interoperability of systems
3. The drive towards information sharing
4. The drive away from bespoke development

What has the above got to do with MDM and the single version of the truth?

In the 70’s and 80’s, each airline was “re-inventing” the wheel by taking the base IBM Operating System, and then changing it.  Each airline started with the same base, but then Darwin’s theory of evolution kicked in (as it always does in Bespoke development environments).  This worked fine as long as each airline effectively worked in a standalone manner, and connecting flights required passengers to re-check in with a different airline etc.  This model was blown away with the arrival of Global CRS systems. Interconnectivity of airline reservation systems became critical, and this required all airlines to adhere to common standards.

We are still in the bespoke era of Master Data Management.  This will continue for some time.  Breaking out of this mode will require a major breakthrough.  The human genome project, in which DNA is being ‘deciphered’ is one of the finest examples of how the “open source” model can bring greater benefit to all.   The equivalent within the Data world could be the opening up of proprietary data models.  IBM developed the Financial Services Data Model (FSDM).   The FSDM became an ‘overnight success’ when BASEL II arrived.  Those Financial Institutions that had adopted the FSDM were in a position to find the data required by the Regulators relatively easily.

Imagine a world in which the Financial Regulator(s) used the same Data Model as the Financial Organisations?

Naturally, such a model would not be set in stone.  There would be incremental improvements – with new versions published on a regular (perhaps  yearly, biannually, or maybe every 5 years).

Back to the great “Truth” debate.  Charle’s view most closely aligns with mine, and I particularly liked his reference to granularity – keep going till one reaches the lowest granularity required – remember, one can always summarise up, but one cannot take apart what has been summarised up.

Most importantly, I would like to thank Henrik Liliendahl Sørensen, Charles Blyth and Jim Harris for holding this debate.  Debates like this signal the beginning of the move towards a “Single Version of the Truth” – and has already led to a major step forward.  Dean Groves suggested on Henrik’s blog that the word “version” be changed to “vision” – Suddenly, we had agreement – We all aspire to a “Single VISION of the Truth”.

I look forward to many more debates of this nature.

You can see the results of the vote here

Plug and Play Data – The future for Data Quality

The excellent IAIDQ World Quality Day webinar looked at what the Data Quality landscape might be like in 5 years time, in 2014.  This got me thinking.  Dylan Jones excellent article on The perils of procrastination made me think some more…

Plug and Play Data

I believe that we data quality professionals need a paradigm shift in the way we think about data.  We need to make “Get data right first time” and  ”Data Quality By Design” such no brainers that procrastination is not an option.   We need to promote a vision of the future in which all data is reusable and interchangeable – a world of “Plug and Play Data”.

Everybody, even senior business management, understand the concepts of “plug and play” and reusable play blocks.  For “plug and play” to succeed, interconnecting parts must be complete, fully moulded, and conform to clearly defined standards.  Hence “plug and play data” must be complete, fully populated, and conform to clearly defined standards (business rules).

How can organisations “get it right first time” and create “plug and play data”?
It is now relatively simple to invoke cloud based verification from any part of a system through which data enters.

For example, when opening a new “Student” bank account, cloud based verification might prompt the bank assistant with a message like “Mr. Jones’ date of birth suggests he is 48 years old.  Is his date of birth correct?  Is a “Student Account” appropriate for Mr. Jones”?

In conclusion:

We Data Quality Professionals need to educate both Business and IT on the need for, and the benefits of “plug and play data”.   We need to explain to senior management that data is no longer needed or used by only one application.  We need to explain that even tactical solutions within Lines of Business need to consider Enterprise demands for data such as:

1. Data feed into regulatory systems (e.g Anti Money Laundering, BASEL II, Solvency II)
2. Access from or data feed into CRM system
3. Access from or data feed into Business Intelligence system
4. Ad hoc provision of data to satisfy regulatory requests
5. Increasingly – feeds to and from other organisations in the supply chain
6. Ultimate replacement of application with newer generation system

We must educate the business on the increasingly dynamic information requirements of the Enterprise – which can only be satisfied by getting data “right first time” and by creating “plug and play data” that can be easily reused and interconnected.

What do you think?

Common Enterprise wide Data Governance Issues #11: No ownership of Cross Business Unit business rules

This post is one of a series dealing with common Enterprise Wide Data Governance Issues.  Assess the status of this issue in your Enterprise by clicking here:  Data Governance Issue Assessment Process

I'm right, he's wrong!

Different Business Units sometimes use different business rules to perform the same task.

Withing retail banking for example, Business Unit A might use “Account Type” to distinguish personal accounts from business accounts, while Business Unit B might use “Account Fee Rate”.

Impact(s) can include:

1. Undercharging of Business Accounts mistakenly identified as Personal Accounts, resulting in loss of revenue.
2. Overcharging of Personal Accounts mistakenly identified as Business Accounts, which could lead to a fine or other sanctions from the Financial Regulator.
3. Anti Money Laundering (AML) system generates false alerts on Business Accounts mistakenly identified as Personal Accounts.
4. AML system fails to generate alert on suspicious activity (e.g. large cash lodgements) on a personal account misidentified as a Business Account, which could lead to a regulatory fine.
5. Projects dependent on existing data (e.g. AML, CRM, BI) discover that the business rules they require are inconsistent.

Solution:
Agree and implement the following Policy:  (in addition to the policies listed for Data Governance Issue #10)

• Responsibility for resolving cross business unit business rule discrepancies lies with the Enterprise Data Architect.

For further details on Business rules – see Business Rules Case Study.

Your experience:
Have you faced a situation in which different business units use different business rules?   Please share your experience by posting a comment – Thank you – Ken.

Lego Blocks and data quality

Lego Plane

Lego blocks allow the average person to build practically anything, because they come in standard sizes, and interconnect with ease.

Having built a model, one may later take it apart and reuse the standard blocks to build other models.  One may do this time and again, giving hours of enjoyment.

Plane carved from wood

By contrast, few people have the skill to carve models from wood.

Once carved, it is practically impossible to ‘remodel’, and completely impossible to reuse any of the parts for other than firewood.

What has the above ‘common sense’ got to do with data quality?

Imagine trying to build a lego model using partially moulded lego blocks?  Imagine opening your lego model kit to discover that some of the pieces were missing.  Truly unimaginable.

We in the Data Quality Profession need to educate both Business and IT on the need to create “standard data components”, that can be easily interconnected to satisfy the information requirements of the business.

Currently, the focus of the Data Quality Industry is on data “Fixing” – remoulding data into parts that are more complete, and more useable.  I see this continuing for a long time, due to the vast quantity of legacy data.   I see the focus moving more towards “get it right first time’ with the emphasis on creating completely moulded, standard component parts from the outset.

My interview with Ajay Ohri

Ajay delves into my past, my present, and my vision for the future.

My interview with Dylan Jones

Dylan Jones of DataQualityPro interviews me about the process I use to assess common Enterprise wide data issues. Use this process to assess the status of data governance within your organisation or that of a client.