The virtual shortcut

• Article 4 of 6
• Database Marketing, October 2005
• View a PDF of the original article ~ 1.9MB

Page 1 | Page 2 | Page 3 | All 3 Pages

The single customer view is a vital step on the way to effective marketing. A cleaned, de-duped view of all customer data and transactions provides benefits to both the marketer and the customer: better targeting and less waste during marketing campaigns; better analysis of customer behaviour and trends; and better customer service.

But achieving a single view is by no means easy. Data is often spread throughout a company in various databases, flat files and other sources. Some way has to be found to aggregate the data from these sources. Most companies opt for the traditional approach of ETL, extracting data at regular intervals from the databases, transforming it using a set of pre-established rules, before loading the cleaned data into a central repository where it’s merged with the other sources.

Yet some prefer a ‘virtual’ single customer view, extracting information from the different datasets, transforming it and merging it on the fly, arguing there are many advantages to it, including a real-time view of customers that is completely up to date. So is the single virtual view a viable option for marketing?

Bill Marjot, CMO of SmartFocus, says while he doesn’t think one approach is categorically better than the other, “if you can leave the data where it is – and there are very good reasons for doing that – it reduces the time necessary to deploy a solution and get marketing up and running. If you can do that, it’s very good news.”

It’s a viewpoint Steve Clarke of CDMS backs up, saying that with the virtual view system from Unica that his company resells, he can get companies putting together campaigns within eight weeks – “a lot of that is meetings, working out matching rules and making sure you provide exactly what the client wants. We can get it done in days.”

Done correctly, a virtual view will be identical to a single integrated database in capabilities. Marketers will be able to extract data from the view for analysis and campaigns, since single-view systems invariably can export into XML and other standard data formats. So for certain applications, virtual and consolidated approaches will produce similar results.

In fact, there will be some situations where the virtual view might be the only one possible. In many organisations, data intended for use in the single customer view is often contained in legacy systems or operational systems, such as accounts, call centres and ERP systems. Often it will be impossible to centralise the data from these systems; this might be because of technological shortcomings in the various systems, internal politics or legal issues, such as data protection restrictions. It may even be a deliberate security policy designed to stop a hacker that breaks into one database from having access to all customer data instantaneously.

Tim Pottinger, CRM divisional managing director of Identex, highlights for example, the problems of merged companies. “They can [combine datasets], provided they ask the right question, but often they haven’t got control over all their datasets, their partners and legacy systems. It’s a big project which may need more structural types of changes.”

Combining the different datasets into a virtual view is not a trivial operation. To link the databases and present this view will require a unique key that allows the systems to match the records in the different databases. This may be a customer number or similar ID. But where datasets refer to different products or have been acquired through merger and acquisition, for example, there may be no such ID.

In these cases, some kind of matching algorithm or tool will be needed, typically based on name, address and other data – or whatever data happens to be available in the source databases. Once the company has matched the equivalent records, tagging them with corresponding IDs for future matching will reduce overheads and matching time. It may not be possible to do this however, because of write restrictions in some databases, meaning that matching will need to be done on the fly in the virtual view.

With the records stored in separate databases, however, de-duping is a harder task since deleting a duplicate will require the permission of the dataset’s owner, something that may not be granted. There will also be the usual issues associated with deleting or merging duplicates in operations datasets: with a single, separate marketing database, it’s possible to delete and merge records incorrectly, usually with only minimal ill effects; in an operational database, these kinds of operations need to be done with absolute certainty.

Even cleaning is likely to be harder. Terry Hiles, MD of Capscan, points out that while a marketer may want an address stored in one format, “the individual database owners don’t want you mucking around with their records, might not want an address stored in that format and won’t thank you for deciding that you have the authority and wisdom to dump data from their records.”

As well as cleaning the databases at the beginning of the operation there needs to be a way to keep all the databases clean. Says CDMS’s Clarke, “If you’ve got different databases and they’re being updated by different means by the call centre, secretaries, those kind of groups, you do need a central cleaning place so the cleaning process is the same for all databases.” Without consistent maintenance of all the databases, the virtual single view will begin to lose its authority and cohesiveness.

Cleansing on the fly, in which these activities are done as the system pulls together records for the single view, may be possible, but Ed Wrazen, VP International of Trillium Software cautions that this can be complex, since it adds additional integration layers on top of the integration already required to pull together the databases. “Effectively, you’re creating spiders’ webs of complexity and information, that logistically are quite difficult to manage. There are also limitations in terms of the technology being able to provide the performance necessary.”

The virtual single view will also generally require a master-slave relationship between the databases, with one database – usually the cleanest and most complete – taking precedence in cleaning operations. This will typically also be the database that marketing has most control over and which is in the best format for marketing operations. When in doubt, the customer billing is usually the best master to pick.

The complexities involved in the virtual view mean that many companies eschew the virtues of real-time insight into customers in favour of the traditional single marketing database with regular ETL updates from other data sources.

Colm O'Hara, database manager at EuroDirect, says that regulatory and operational reasons prevent his firm, which creates and hosts single customer views for a number of companies, from using virtual views. “We’ve been able to use strong name and address matching capabilities since we don’t use a virtual view. We probably couldn’t even use a virtual view for compliance reasons: how could you provide evidence for your decisions in a virtual view?”

Similarly, Shelagh Regester of CACI client services director says that it’s operationally easier for CACI to go for a standard integration approach rather than the virtual view. “The issue with having a virtual view is that because of the hit that obviously makes on central processing, it makes it harder to plan operational work since you’re making ad hoc use of the ‘juice’. Frankly, the tools available for the front end are so good at holding massive datasets, it’s the just the same as creating a virtual view anyway.”

Similarly, Arthur Kay of Synaxis says that few of his clients are interested in virtual view technology once they see what a single integrated database is capable of doing. “The virtual view shouldn’t be seen as a way of achieving a single customer view, not if marketing messages are important to you.”

While CDMS’s Clarke claims that Unica’s universal data interconnect technology can interface with pretty much any database to create a virtual database, others are not so sure that a virtual view is so simple. Phil Good, founder and MD of Hopewiser, highlights the problems of accessing legacy systems that don’t provide standard interfaces and which will require coding to extract data. “Somewhere like Shell or BP, where the systems have been around 20 years and the guy who programmed them has left, if you ask them to stick a web wrapper round the systems so you can access them, they’ll just say, ‘I don’t think so, mate.’”

Equally, Chris Cuffe, MD of helpIT Systems, argues that virtual views that pull in data from a number of systems are liable to fail, since most big IT implementations fail. This can prove financially catastrophic for many organisations. He also suggests that few marketing applications need real-time data: “What’s the disadvantage to most marketers of having data a week old? If you’re looking back three years, what’s changed in a week? For the price you have to pay to achieve a virtual view, I can’t say it makes a good deal of difference.”

The virtual view is best suited to marketing applications where a small number of systems are involved, the company needs quick access to reasonably accurate data as quickly as possible and where it’s difficult or impossible to create a single integrated database from the various available datasets. For normal marketing activities, its complexity and the increased difficulty of standard data cleansing makes it a poor choice for many companies.

Page 1 | Page 2 | Page 3 | All 3 Pages