Logo Rob Buckley – Freelance Journalist and Editor

Watching brief

Watching brief

Web analytics is often the only way to understand who visits a web site and why

Page 1 | Page 2 | Page 3 | Page 4 | All 4 Pages

Web site analytics - analysing user behaviour and the information provided by their web browsers - has long been regarded as an inadequate activity.

While scooping up the Internet protocol (IP) addresses of web site visitors and the number of hits and page impressions might impress techies, turning that data into information that can sell advertising or more products has proved more challenging.

Yet Amazon.com shows just what can be done with a well-designed analytics function. In combination with a user profiling system, it personalises content and pushes relevant products to users, even telling them how much their past purchases might be worth second-hand.

The Amazon experience demonstrates that while the data generated by web analytics packages might be limited, it can go far in helping organisations understand what kind of people visit their web site and what they use it for. This makes it useful not just for commercial site operators that need to know more about their customers, whether to sell them products or to 'sell' their site's demographic profile to advertisers, but for a wide range of organisations, both public and private.

These include:

  • local authorities and other public sector organisations that are obliged to show that their web sites are available to citizens and that they do indeed use them;
  • organisations that want to provide information and need to be sure that the right people are able to read it;
  • service providers who want to ensure that as many people as possible can take advantage of a service;
  • any organisation that wants to ensure its web site is free of errors and is accessible.

Web analytics is potentially useful to any organisation that has put up a web site with a purpose. Indeed, research by web analytics software maker WebTrends suggests that nine-tenths of organisations now regard it as a key tool supporting their Internet strategies.

Who, where, why?
The range of data potentially available to organisations through web analytics at first glance seems small. Most web statistics revolve around the seemingly mundane figure of how many times people have downloaded particular pages.

But every web visit provides far more information than just requests for data. Each request also includes the user's Internet (IP) address, and the time and the time zone of the user's computer. So it is easily possible to locate the countries in which users are based. If they are corporate users, it can also identify their employers.

The data should also identify the web site the user has just surfed in from. This is usually stored automatically in web server logs and can then be analysed to create an historic view of exactly where the site's web traffic is coming from.

Any requests for missing pages, images and so on will also show up, highlighting broken links and references. Since the IP address is usually unique and constant throughout a visit to a site, it is possible to see which pages a particular user viewed and in what order.

This “click stream” analysis, either at the individual level or in aggregate, can provide insight into how well the navigation structure of the site is performing. It can show which areas are the most popular and which are the least popular; where users left the site through a link, or simply gave up on it altogether; and what other sites have linked to the site.

And since most search engines tend to encode search terms into their web addresses or URLs, it can show what search terms produce good results for the site and what people are most interested in when they come to the site.

There are a number of techniques that can augment server-side data. For example, cookies can be used to store unique codes on a user's computer that the web server can identify when they return to the site. This will reveal the repeat visitors and allow the site owner to monitor users' areas of interest over repeat visits, provided the cookies are not deleted by users; fortunately, most are not.

Inaccuracies
The problem with server-side data is that it can be inaccurate and is only available for requests made direct to the server. Often, Internet service providers (ISPs) and organisations will use proxy and caching servers to improve perceived network speeds. These store pages and images locally - nearer to the user - so that if the browser requests any of the stored data, it can be provided far more quickly than if the requests were sent to the organisation's web server.

However, it also means that there is no entry in the web server's log for the page or data view.

The server-side approach also fails to take into account the browser's own cache, which stores images and even whole web pages. By serving such data from the browser, end-users enjoy much faster progress through already downloaded pages when using the back and forward buttons of the browser. This cache also prevents requests reaching the web server.

A method developed by Clickstream Technologies uses a combination of cookies and a JavaScript-based program embedded in web pages in a bid to overcome this problem.

The programs measures how long someone has looked at a page and the actions they took, including forwards and back buttons. It then stores the information in a cookie. When the browser next makes a request to the web server, the browser passes all the data in the cookie to the web server.

The goal
But with all this information potentially available, before putting it to work, organisations need to consider carefully what they will actually do with it.

“The first step is identifying who in the company cares about the web site and then to get them all into one room,” says Jim Sterne, CEO of The Emetrics Summit and president of the newly formed Web Analytics Association (WAA). “Then have them identify very specific web goals. For example, what are they trying to accomplish? And how will they know if they've been successful?”

Typical goals might include selling more advertising space or to increase sales generated from the site. Public sector organisations will want to ensure that their services are easy to use for as many people as possible and often need to abide by e-government “priority outcomes”.

For example, priority outcome R25 states that councils must test and monitor their web services and publish figures to show that they are offering 24/7 availability, while outcome R26 says that they must perform web analytics and publish the results to show their services are being adopted by citizens. The organisation then needs to decide if it has the skills and the software to understand the information it gets - which is usually more than enough. Paul Dawson, head of user experience at consultancy Conchango, warns that the out-of-the-box analysis from most statistical packages is “a bit hit and miss”.

“What you tend to end up with is a statistical analysis package that is extremely wide ranging and complex,” he says. “Customers tend to find all the data extremely overwhelming and so focus on just one or two metrics, such as hits or page impressions.”

Initially, the best policy might be to hire someone with a web analytics background. This person can then set up the processes and analytics packages that will provide data back to the rest of the organisation on an ongoing basis. When the processes are in place, less specialised or highly trained users can be brought in.

Reliance on the standard output of software packages can also hide important information. Often packages will report information, such as the top 20 browsers and platforms used by visitors; they may even perform analytics based on what users with particular browsers did, to see if the web site is poorly configured for particular browsers and is causing their users to abandon the site.

However, assisted browsers for visually and physically impaired users will invariably appear very low down in the rankings because of the small number of users. As a result, investigations into site accessibility may be defective since many web site operators will ignore the reports from these browsers since they are so low down the list.

An appealing alternative to in-house analytics for many organisations is to deploy a third-party application service provider (ASP), such as WebTrends or RedSheriff, to analyse the data. However, passing information to ASPs can also cause problems.

Privacy
Simon Halberstam, partner and head of ecommerce law at Sprecher Grier & Halberstam, says that web site operators need to inform site visitors that they intend to pass data on to third-parties. “To be safe from the consequences of breach of the Data Protection Act, the web site owner needs to force the site visitor to scroll through and accept its privacy policy prior to registering personal details on the site,” he suggests.

Another issue is Microsoft's Internet Explorer 6 browser, which will refuse cookies that do not meet particular security requirements. IE6 is far more likely to reject third-party cookies than cookies provided by the site being visited, thus potentially ruining all validity to the analytics.

Other browsers may be set to delete cookies when the user closes the browser down.

Nevertheless, these are relatively minor issues. With the goal and metric decided, organisations can then implement the measuring system of their choice, taking care of the legalities when they do so.

Any cookie-based analytics system - not just ASP-based systems - potentially falls under the European Union directive on privacy and electronic communications. Conchango's Paul Dawson advises placing a comprehensive privacy policy on the site advising visitors of the use of cookies for web analytics to avoid problems.

With the analytics in place, the organisation then needs to consider what it has learnt before implementing any changes, large or small. That can be open to a very wide degree of interpretation.

“If lots of people go to your information page, for instance, and then to the 'contact us' page, one interpretation is that people wanted the contact number and they've been able to find it, so the site is a success. The other is they could not find what they wanted so looked for the number for more information,” says Dawson.

So Dawson advises combining web analytics with offline information gathering, including user focus groups: the analytics can highlight potentially interesting parts of the site, while user groups can help explain why those parts are particularly interesting.

Nick Trainor, director of analytics firm Trainor Thornton, advises a more directed approach to web site development. By devising pages and links through the site appropriately, it is possible to lead people through the site and get a clearer idea of what they want from it, particularly in conjunction with the search engine keywords used to drive traffic to particular areas.

Once possible changes are devised, implementation of these changes must be slow unless there's enough traffic for results to show up.

The WAA's Sterne advises against big redesigns unless they are absolutely necessary. “When you do a 'big bang' redesign, you lose all your benchmark material. All the pages are different, all the navigation is different so you're comparing apples with oranges,” he says.

“I've heard lots of stories of companies spending millions of pounds updating their web sites and the convergence rates going down. Your most loyal users show up at your brand new web site and they're confused. They bumble around the new web site, trying to learn how to use it and therefore don't buy as much.” The real competitive edge comes from implementing smaller, incremental changes combined with constant testing, he adds.

Compared to the business intelligence tools available to interrogate data warehouses of customer data, web site analytics is immature and limited. But there is nevertheless a real business benefit available to organisations that implement analytics for their sites - provided that it is implemented and managed correctly.

Page 1 | Page 2 | Page 3 | Page 4 | All 4 Pages

Interested in commissioning a similar article? Please contact me to discuss details. Alternatively, return to the main gallery or search for another article: