Netrics Signs TIBCO as an OEM Partner

We’re thrilled to announce that TIBCO Software (NASDAQ:TIBX) is our newest OEM Partner.

Data Migration - The lessons to be learned from NPfiT

A break out session from the cloud as I rage against incompetence and complacency.

The 10 Commandments … Or The 7 Deadly Sins?

With much amusement I read a blog entry the other day entitled the 10 commandments of data integration. Of course, I wish I’d thought of it myself.  Great idea and quite amusing.  Being the CMO for the world’s number one independent data integration company, I of course had to think of something witty to respond with!  So, how about the 7 deadly sins of data integration!  Lost in the midst of time and as a reminder, the 7 deadly sins are gluttony, lust, greed, envy, wrath, sloth and pride.  So here goes …

How to avoid the 7 deadly sins of data integration:

• Thou shall not give into data "gluttony" and instead live the lean consumption data integration lifestyle.
• Thou shall not show "greed" over the consumption of more and more and more data, but instead embrace data quality as nirvana.
• Thou shall not show "envy" if you notice your competiors have embraced integration on-demand. Don't worry, Informatica can guide you along your journey.
• Thou shall not show "wrath" when your hand coding projects come in over budget because you will see there is a better solution.
• Thou shall not exhibit "sloth" at having data to manage in multiple applications and databases, Informatica is the light that will show companies the way.
• Thou shall not exhibit "lust" when you complete your data integration merger milestones with Informatica in record time, under budget and with a single view of your customers.
• Thou shall not show "pride" when your company reaches the data integration pinnacle with Informatica, just beat the pants off your competitors and wink!

�

Survey: Who wants to see a regular UK Data Quality Pro Live Event?

Our registered membership is now a shade under 2,000 and we regular receive about 6,000 visitors to the site each month, a large proportion of whom are UK based.

We therefore feel it is time to take the show on the road and are considering a series of UK events, either annual or more frequent.

But what kind of event do you want us to hold?

What topics do you want us to discuss?

Would an annual event suit you or would you prefer more regular events?

 

To help us create the ideal event for our members we've put together a short survey (no more than 1-2 minutes) to help us gather feedback.

Here is the link: http://www.dataqualitypro.com/uk-events/

 

Based on your responses, we'll hopefully look to launch a series of UK events very soon.

Not based in the UK?

At the rate our membership is growing we'll certainly be looking for volunteers who are keen to hold local events in their neck of the woods so please stay tuned.

Tesco data quality issue is not illegal!

I don't know about you, but I personally have a trust issue with Tesco when it comes to the information they provide you to help you make your purchasing decision.

My big frustration is the so called ‘information’ they give you to help you work out which brand is the cheapest per 100g or 100 sheets.

They mix up the comparison information using Kg and g for the same range - I'm OK with this level of mental math’s, but I know that others can not work out which item really is the cheapest when faced with this ‘mis-information’.

Yesterday I was buying coffee and noticed this label:

The price for a 200g jar is £4.38, yet the price per 100g is £4.38!

At £4.38 per 100g the 200g jar should be £8.76 - Being an honest citizen and being concerned for Tesco's profits, I pointed the error out to a member of staff.

Over the next five minutes I was given explanations from four different members of staff that ranged from, 'That's the way the computer prints them' and 'These things are set by head office', through to the Managers view, 'It's not illegal'!

It may not be illegal, but that response did not help develop customer trust!

My free consultancy / 'quick and dirty' advise to Tesco's Price Integrity team is to put all the shelf edge label pricing information in to a database (if it’s not already in one) and run a daily report which identifies items where the price does not equal the price per 100g X its weight!

A simple check, which whilst not a legal obligation, sounds like good practice to me :-)

Data Quality Blog Roundup - June 2009 Edition

Another marked increase in online publishing this month for the data quality sector. A smattering of new entrants means there is a steady flow of fresh ideas and insight in this months blog roundup.

Are you interested in getting started with your own blog? We are about to launch a social media support hub here on Data Quality Pro to provide education and support that will help you create, launch and sustain a healthy data quality blog. Stay posted for an announcement very soon so feel free to contact us with your blogging questions.

 

Data Quality Blog Roundup - June 2009 Edition

 

"El Festival del IDQ Bloggers” - June Edition - Steve Sarsfield hosted this month's IAIDQ and did a great write-up of the various data quality bloggers who had submitted their June posts, great to see the older blogs still going strong and of course some great new bloggers coming on the scene too, submit your entries for next month to the IAIDQ.

Evil Dictators: You Can’t Rule the World without Data Governance : Steve Sarsfield argues the case that only with data governance can business leaders bring order to their business units (we also interviewed Steve recently to talk about his new book launch).

Bord Gais loses 75000 customer records - Daragh O Brien writes on his personal blog about the recent data security breach at Bord Gais in Ireland and the potential fallout for customers and the organisation.

DataFlux Still Leads in the Gartner Magic Quadrant for Data Quality Tools 2009 - Vincent McBurney carries out his annual assessment of the Gartner Magic Quadrant and highlights the transition of vendors from the previous year.

Guerilla Governance : Mike Meier presents a great example of how poorly designed processes can emerge and how his particular approach of "Guerilla Data Governance" can help break down silos and move the business forward. (We also featured Mike in this recent interview).

Identifying Duplicate Customer Records - A case study where Dalton Cervo (also interviewed in this post) writes about the challenges, techniques and outcomes of attempting to consolidate 800+ customer management databases at Sun Microsystems.

IT Challenges for Enterprise Data Quality : David Loshin presents 3 common challenges the IT community face when attempting to deliver an enterprise data quality framework.

Lightweight Data Governance : A Starting Point : Mark Goloboy kicks off a series of posts that will focus on his experience of launching Data Governance initiatives. This post provides 5 really useful pointers for those treading a similar path.

Modeling the MDM Blueprint – Part 6 : James Parnitzke continues his excellent series on the Hub Solution Designs Blog, this week he looks at the activities involved in creating an MDM solution specification.

Quadrant for Data Quality Initiatives : Dalton Cervo creates a useful tool for utilising data quality dimensions in order to target which data quality initiatives in your organisation will require closer attention, Dalton also featured in our recent interview on Data Quality Pro:  "Identifying Duplicate Customer Records"

Qualities in Data Architecture : Henrik Liliendahl Sørensen discusses the importance of creating attractive data and creates a compelling metaphor for the importance of data quality in architecture.

Sun Tzu and the Art of Data Quality : Daniel Gent continues to be busy, last month we featured two of his posts and this one cites several examples of exactly why organisations have to step up and begin addressing data quality.

There goes another one : Graham Rhind laments the acquisition of yet another international name and address specialist to a data quality technology vendor, namely the Informatica acquisition of AddressDoctor. Graham debates the possible outcomes of the deal and what it could mean to the industry.

The terror of the Terrorist Watch list : Daragh O Brien on IQ Trainwrecks.com reports on some of the ongoing failures being carried out by the FBI as they attempt to manage the terrorist watchlist.

The Three Musketeers of Data Quality : Jim Harris with another great post, this one examines the 4 vital roles on any data quality project, an entertaining (and memorable) way to explain the importance of team dynamics on a data quality initiative.

TSA "Secure Flight" will require more demographic information : Stefanos Damianakis also reports on the recent challenges faced by the authorities when attempting to match airline passengers against terrorist watchlists, he also cites the obvious drawbacks with the current process.

What’s the value of prospect data or ex-customer’s data? : Ron Mulderij opens up a debate that stale prospect data can clog up CRM systems and should be routinely deleted as its value is constantly diminishing.

 

Useful Resources

 

Find all posts in : DQ Blog Roundup

• El Festival del IDQ Bloggers (promote your DQ posts or why not host it on your site one month?)
• Data Quality Blog Roundup (May 2009 Edition)
• Data Quality Blog Roundup (April 2009 Edition)
• Data Quality Blog Roundup (March 2009 Edition)
• Data Quality Blog Roundup (February 2009 Edition)
• Data Quality Pro Blog Roundup (January 2009 Edition)
• Data Quality Blog Roundup (December '08 Edition)
• Data quality blog roundup (November '08 edition)
• Data Quality Blog Roundup (October '08 Edition)
• Who are the data quality blogging heroes?
• How to create a professional data quality blog (Part 1 - Planning)
• 20 simple tips to spice up your data quality blog

 

Data Governance and Data Quality

 

Regular readers know that I often blog about the common mistakes I have observed (and made) in my professional services and application development experience in data quality (for example, see my post: The Nine Circles of Data Quality Hell).

According to Wikipedia: “Data governance is an emerging discipline with an evolving definition.  The discipline embodies a convergence of data quality, data management, business process management, and risk management surrounding the handling of data in an organization.”

Since I have never formally used the term “data governance” with my clients, I have been researching what data governance is and how it specifically relates to data quality.

Thankfully, I found a great resource in Steve Sarsfield's excellent book The Data Governance Imperative, where he explains:

“Data governance is about changing the hearts and minds of your company to see the value of information quality...data governance is a set of processes that ensures that important data assets are formally managed throughout the enterprise...at the root of the problems with managing your data are data quality problems...data governance guarantees that data can be trusted...putting people in charge of fixing and preventing issues with data...to have fewer negative events as a result of poor data.”

Although the book covers data governance more comprehensively, I focused on three of my favorite data quality themes:

• Business-IT Collaboration
• Data Quality Assessments
• People Power

 

Business-IT Collaboration

Data governance establishes policies and procedures to align people throughout the organization.  Successful data quality initiatives require the Business and IT to forge an ongoing and iterative collaboration.  Neither the Business nor IT alone has all of the necessary knowledge and resources required to achieve data quality success.  The Business usually owns the data and understands its meaning and use in the day-to-day operation of the enterprise and must partner with IT in defining the necessary data quality standards and processes. 

Steve Sarsfield explains:

“Business users need to understand that data quality is everyone's job and not just an issue with technology...the mantra of data governance is that technologists and business users must work together to define what good data is...constantly leverage both business users, who know the value of the data, and technologists, who can apply what the business users know to the data.” 

Data Quality Assessments

Data quality assessments provide a much needed reality check for the perceptions and assumptions that the enterprise has about the quality of its data.  Data quality assessments help with many tasks including verifying metadata, preparing meaningful questions for subject matter experts, understanding how data is being used, and most importantly – evaluating the ROI of data quality improvements.  Building data quality monitoring functionality into the applications that support business processes provides the ability to measure the effect that poor data quality can have on decision-critical information.

Steve Sarsfield explains:

“In order to know if you're winning in the fight against poor data quality, you have to keep score...use data quality scorecards to understand the detail about quality of data...and aggregate those scores into business value metrics...solid metrics...give you a baseline against which you can measure improvement over time.” 

People Power

Although incredible advancements continue, technology alone cannot provide the solution.  Data governance and data quality both require a holistic approach involving people, process and technology.  However, by far the most important of the three is people.  In my experience, it is always the people involved that make projects successful.

Steve Sarsfield explains:

“The most important aspect of implementing data governance is that people power must be used to improve the processes within an organization.  Technology will have its place, but it's most importantly the people who set up new processes who make the biggest impact.”

Conclusion

Data governance provides the framework for evolving data quality from a project to an enterprise-wide initiative.  By facilitating the collaboration of business and technical stakeholders, aligning data usage with business metrics, and enabling people to be responsible for data ownership and data quality, data governance provides for the ongoing management of the decision-critical information that drives the tactical and strategic initiatives essential to the enterprise's mission to survive and thrive in today's highly competitive and rapidly evolving marketplace.

 

Related Posts

TDWI World Conference Chicago 2009

Not So Strange Case of Dr. Technology and Mr. Business

Schrödinger's Data Quality

The Three Musketeers of Data Quality

 

Additional Resources

Over on Data Quality Pro, read the following posts:

• INTERVIEW: Steve Sarsfield
• DEBATE: How should data governance and data quality work together?

From the IAIDQ publications portal, read the 2008 industry report: The State of Information and Data Governance

Read Steve Sarsfield's book: The Data Governance Imperative and read his blog: Data Governance and Data Quality Insider

How to use WinMerge to Manage DataStage Configuration Files

WinMerge is a free open source application that helps manage DataStage configuration text files and merge changes between DataStage environments.

Catching The Dodgers And Raising Taxes - Without "Raising Taxes"

What a year it’s been.  The financial markets have been in turmoil and we’re deep inside a recession.  I pay my taxes, as I’m sure you do, but I’m convinced that many others don’t.  I don’t mind paying – honest, I don’t.  What I object to is the thought that others are not paying their fair share.  Sometimes this may be through an innocent misunderstanding, but sometimes I'm sure it's through deliberate action.  Can I call these people "dodgers"?

So how can we catch the tax dodgers?  If we can catch them, then maybe our state and local government offices can continue to provide the level of service we’ve expected of them rather than having to cut-back in light of the recession we find ourselves in.

I was reading about a number of examples where state authorities have actually been raising taxes … without raising taxes.  Sounds implausible doesn’t it?

For example, the State of Texas uses data integration technology to recover $70 million per year in uncollected sales and use taxes.  How do they do this?  They use Identity Resolution, technology which enables companies and government organizations to search and match identity data in batch and real-time.  Have a look at our website to understand how Identity Resolution is helping organizations find criminals, uncover tax fraud and help ‘raise’ taxes without really raising taxes.

June’s "El Festival del IDQ Bloggers”

A Blog Carnival for Information/Data Quality Bloggers

June of 2009 is gone, so it’s time to look back at the month and recognized some of the very best data quality blog entries. Like other blog carnivals, this one is a collection of posts from different blogs on a specific theme.

If you’re a blogger and you missed out on this month’s data quality carnival, don’t worry. You can always submit your brilliant entries next month. So, here they are, in no particular order.

• Newcomer Jeremy Benson has a unique perspective of being an actuary – someone who deals with the financial impact of risk and uncertainty to a business. We know that improving data quality will certainly produce more accurate assessments when it comes to crunching numbers and calculating risk. This month’s blog entry describes how data quality is important to predictive modeling. More actuaries should understand the importance of data quality, so this is a positive step.

• Irish information quality expert Daragh O Brien was talking about his marriage problems this month – well, at least the data quality problems with his recording of his marriage. In this post he discusses a recent experience and how it made him think yet again about the influence of organizational culture and leadership attributes on information quality success and change management.

• Over at dataqualitypro.com, Dylan Jones was also having family data problems. The good news was the arrival of a beautiful baby boy. The bad news was there were some issues about getting his health records straight in the NHS database. See NHS unique ID failure leads to duplicate newborn patient records. Congratulations to Dylan on his new family member!

• Western Australian blogger Vince McBurney contributes his excellent analysis of the new Gartner Magic Quadrant for data quality tools. Vince’s analysis of the LAST Magic Quadrant (two years ago) was perhaps my biggest inspiration for getting involved in blogging, so it makes me happy to include his blog. “Tooling Around on the IBM InfoSphere” is focused on data integration topics from the perspective of an expert in the IBM suite of software tools.

• Jim Harris takes us into “The Data-Information Continuum” to remind us that data quality is usually both objective and subjective, making reaching the “single version of truth” more mystical. The post made it clear to me that our description of the data quality problem is evolving, and the language we must use to promote our successes must evolve, too.

• Henrik Sørensen sees the similarities between Roman architecture and data architecture in his blog post. He makes some great points about moving beyond functional architectures and making them more attractive.

• Dalton Cervo is the Customer Data Quality Lead at Sun Microsystems and a member of the Customer Data Governance team at Sun. Dalton takes us on a journey of depuplicating a customer database using a popular data quality tool. It’s great to see the detail of project like this so that we can better understand the challenges and benefits of using data quality tools.

• Daniel Gent delivers another brilliant post this month, drawing lines between SunTzu’s Art of War and the data quality. We’re looking forward to a continuation of this series.

Thanks to all the outstanding data quality bloggers this month!