Corporate concern

• Article 5 of 6
• Database Marketing, September 2006
• View a PDF of the original article ~ 1.2MB

Page 1 | Page 2 | Page 3 | Page 4 | All 4 Pages

Data cleansing is a job that needs to be done frequently and well. Marketing has had to learn this the hard way, and over time the tools and datasets necessary to clean data have become a familiar entry in most marketing budgets. But for some time now, other areas of the business have been learning that they too need clean data. From call centres to accounts departments, data cleansing tools are becoming an increasingly important part of the IT infrastructure.

Gartner analyst Debra Logan argues that organisations that consciously ignore or have a complacent attitudes toward enterprise information management (EIM) and data quality will “struggle to maintain their status in increasingly competitive and agile business environments”. Companies that follow best practices in EIM will achieve a competitive advantage by mid-2007, she argues, and she’s already seeing movement in that direction in financial services, life sciences and other companies with a high quantity of data to manage.

Certainly, some enterprise software vendors seem to agree with her that EIM is on the rise. Business intelligence company Business Objects acquired First Logic in June to add data cleansing to its EIM portfolio.

“Data quality is a pre-requisite and really important for units to consider before making decisions,” says Kristin McMahon, product marketing manager, data quality, of Business Objects. “Going forward, it’s something that organisations aren’t going to be able to ignore.”

Marcus Brook of Data Discoveries highlights, for example, problems facing companies in financial services. “Data quality used to be a pretty turgid subject, just about names and addresses – nothing to get excited about. But many companies in financial services don’t know where 21% of their customers live. That’s a huge drag on costs. When a pension matures, where do you send the cheque? There’s tens of millions of pounds sitting in bank accounts because the companies haven’t maintained relationships. But with proper data quality, you can find out where the costs are coming from and develop strategies around that.”

It’s this realisation that data quality is affecting business decisions as well as just mailshots that is causing the shift in attitude. Martin Doyle, CEO of DQ Global, says the pressure for improved, organisation-wide data quality is coming from the top. “There’s a top-down push from business execs. And the business execs are saying, ‘I want score-carding, I want business intelligence, I want to make decisions based upon facts.’ And then the IT people are running away in a panic, the marketing people are running away in a panic and they’re saying, ‘But our data’s rubbish.’ ‘Well, then you’d better sort it out, hadn’t you?’ And then they start looking at the wider aspects of sorting the data out.”

Nottingham-based Ikano, for example, realised it needed to improve its data quality, not just to avoid losses to the business, but to help save customers from themselves. The company specialises in store-cards, interest-free credit, personal loans and support services to retailers to underwrite credit and loyalty schemes.

Ikano uses credit-rating data to decide whether its products are suitable for individual customers. It also tries to identify holders of more than one card to determine if they’re at risk of building up unmanageable debt.

“In the past, we had developed our own software in-house but we suspected it wasn’t picking up all the records of matching names,” says marketing analyst John Duncombe. “If a name appeared as ‘Miss A Williams’, it wouldn’t recognise it as the same as ‘Amy Williams’. We can’t just list all the cards a person has on the same record because we have to keep a separate record for use of each card - we can’t share customer data between companies.”

Ikano ended up buying the matchIT suite from helpIT to solve its data cleansing problems. Like First Logic, though, helpIT already had functions built into its software that appealed to departments other than marketing. For the most part, it seems to be the rule that only those vendors that have deliberately addressed the needs of the rest of the business have been able to make real headway.

Hopewiser’s national sales manager Mark Dewell says that it’s had its feet in both camps for some time. “The prime users of Atlas tend to be marketing or IT. We tend to integrate into other business processes that deal with customer and prospect data.” The difference is in how the software fits into the organisation. “With marketing, with very conscious of Windows and GUIs. We provide set-up wizards. But while IT sometimes like GUIs, they prefer the functions that integrate it into a larger business. They use the API route, starting at the engine level. We’re providing IT with the flexibility for integrated workflow.”

Similarly, Hans Ruigrok, VP sales and marketing of Human Inference, says his company’s products have appeal for the larger organisation that transcends the marketing department. “In marketing, they tend to use the software as a stand-alone tool. But we can be implemented as a business service that can be called from every application.”

While name and address cleaning are the most well known DQ issues, corporate data quality requires other cleansing. Phone numbers, national insurance numbers, social security numbers of US residents and personal information of customers, prospects and employees might require validation; supplier and product information might need cleansing; customer lists could require checking against lists of known terrorists and money launderers. There may be fields that only have relevance to an individual organisation that need cleaning according to its rules.

Additionally, Human Inference’s Ruigrok says the software’s multi-lingual capabilities make it of particular interest to multi-national companies that want to standardise on a single tool throughout its different offices. Knowing how to work with individual countries’ different datasets and languages, which can each present unique problems and misspellings, is one of its main appeals.

Nevertheless, it’s the ability to be integrated with other applications that really makes or break a marketing application that has ambitions of being used by the rest of the enterprise. Smaller, more specific tools that can be integrated into larger applications, for instance, are also doing well.

Arc En Ciel’s data cleansing tool is integrated into various applications at the point of data capture. “Almost all estate agents use our software,” says technical manager Tony Reynolds.

“I never cease to be surprised where our products get to,” says David Dorricott, managing director of AFD. “Through the use of our API tools and through access to our cleaning engine, there’s the option to build both in enterprise systems.” If anything, says Dorricott the rest of the organisation tends to be more demanding in its requirements of data cleansing tools than marketing.

To be able to integrate with other business applications requires functions that many marketing applications do not have. As Hopewiser’s Dewell points out, being able to run more or less unattended and without a graphical interface is a necessary first step. Bureau in a Box’s Cygnus, while excellent for marketing work and seeming some interest from the rest of the organisation, will need further development work before it will truly appeal, says managing director Mark Dobson.

“At the moment, we’re a one-product company, about to embark on an 18-month development of some new products. There are certain limitations [to Cygnus] that might not fit with the rest of the organisation. It won’t run lights-out so it doesn’t interrupt activities. It’s also not client-server.”

Running “lights out” presents some obstacles that GUI-based apps don’t. While marketing can often have far looser criteria when dealing with names and addresses, operational data requires far more precise matching. This will either lead to more exceptions that require manual intervention or requires far more sophisticated matching algorithms and datasets. The formercosts far more in both time and wages so needs to be avoided. This means the latter option is far more popular with the rest of the enterprise: what’s “good enough” for marketing won’t do for everyone else, unless the marketing app is capable of far greater precision than marketing may demand of it.

In this service-oriented, loosely-coupled age, embedding directly into an application is neither attractive nor preferred. If data cleansing is to be done across the whole organisation, it needs to be accessible by all applications, rather than just the ones that can be embedded given a huge development budget and a team of programmers. So the ability for the software to function as a standalone server, offering its functions to any applications that asks, is becoming more and more important.

This adds an additional requirement as well: the ability to have different business rules for different cleansing requirements. Accounting may want to keep two records separate, but marketing may want to merge them to reduce costs during a mailout. So when marketing calls for a list of names from the database, the data cleansing tool needs to be able to merge names according to marketing’s requirements rather than the more conservative requirements of accounts.

Says Business Objects’ McMahan, “When you’re talking data quality throughout the enterprise, that’s a lot of work. It may be 50 different projects, say. You need a tool that provides a flexible architecture, a service-oriented architecture, where it can act as a centralised business rule repository. That makes the job of IT a lot simpler.”

Marketing apps that have these capabilities, such as those available from Trillium, DQ Global, Human Inference and Business Objects, are more than capable of appealing to the rest of the enterprise – and indeed many of their vendors are playing down the marketing origins of their applications in favour of these other functions. Yet they’re facing competition from vendors of mainstream enterprise applications who are adding data cleansing functions to their applications.

Unless those applications have been developed through acquisition, though, those mainstream applications won’t typically have the expertise necessary to take advantage of the full range of datasets available for data cleansing, without some end-user or consultant development work. For example, working with PAF will usually need to be done through integrating a smaller marketing-derived tool.

Using PAF and other datasets with these tools, when marketing or some other department might already license it for their own use, can also prove costly. Increasing use of the marketing tool to other parts of the enterprise can reduce dataset licensing costs. This inevitably means that fuller featured applications that can do address and postcode cleansing as well as other DQ functions and that can be accessed from anywhere are going to have an edge over the smaller specialised apps in the long-run.

Alternatively, some enterprises try to avoid double-licensing issues by using web services to access datasets hosted by third-parties. This also allows them to do data cleansing on the fly, paying per record cleaned. However, this doesn’t appeal to everyone, says Maurice Hickey, solutions consultant at Identex.

“Some companies are wary of doing DQ themselves, while others are wary of web services or giving data to a bureaux.” Although bulk deals can be negotiated, the variable cost of pay-per-clean puts off some companies; the high cost of a pay-per-clean run on an initial database of thousands or millions of records scares others, although an initial cleanse by a bureau followed by ongoing pay-per-click can appeal.

Marketing tools that provide more than just the algorithm, but also include datasets and the technological ‘skills’ necessary to use that dataset effectively therefore have something of an edge.

Data Discoveries’ Brooks goes one further: “The days of taking data out and sending it to third-parties is over. It’s Dickensian. Sure, the advent of web tech all over the world makes things much easier. But you’re still dealing with third-parties and none of them understand your business as well as you do.” To get a true grip on data quality means using in-house tools. And that in turn means using tools that have come from the marketing department and that have the functions needed to dispense with bureaux.

Although data cleansing is emerging from the marketing department to become an organisation-wide concern, only those applications that have adapted to meet the more technically advancements requirements of the rest of the organisation are going to succeed. More importantly, it will be their capabilities in dealing with standard datasets that will give them the edge over the tools emerging from enterprise software vendors. While some tools will no doubt become commodities in a short space of time, the more advanced and fully-featured data quality apps that marketing has become used to over the years are going to be the ones setting the pace.

Page 1 | Page 2 | Page 3 | Page 4 | All 4 Pages