The Bayesian haze
- Article 1 of 25
- Infoconomist, March 2001
Knowledge management is an area in which Europe has been doing very well. The star of the sector is Autonomy, but is its heady valuation deserved?
Page 1 | Page 2 | Page 3 | All 3 Pages
Autonomy, the Cambridge, UK-based search engine company, is a software superstar — highly valued and admired on both sides of the Atlantic. Its founder, Mike Lynch, is fêted for his wit and vision as much in Silicon Valley as in Europe. But the market is jumpy about the company, as if investors can't quite believe how well it is doing; and Lynch, in turn, seems jumpy about the market.
In February 2001, for example, all it took was a leaked note from Merrill Lynch that said Microsoft was going to launch a competing product to Autonomy — using technology based on work by “Steve Robinson”, a supposed former Autonomy employee, and the company's share price dropped from £19 to £17. This was despite some analysts' suggestions that it should be nearer £40 — and despite inaccuracies in the note that Merrill Lynch later had to retract.
Autonomy, in fact, has been so successful – in growth, in profile, and in its valuation – that many in the search engine business cannot conceal their astonishment, if not their outright jealousy. This is particularly true of those Cambridge companies that use similar technology. Executives at SmartLogik, for example, frequently complain that the market capitalisation of its parent company, BrightStation, is too low compared to Autonomy's.
Is the company's technology that similar? Autonomy certainly argues not. But both Autonomy's and SmartLogik's systems draw on the same Cambridge University research into 'Bayesian probabilistic inference'. Research that was principally conducted by Professor Martin Porter – (now a consultant to SmartLogik), Karen Sparck Jones (still working for Cambridge University), and one Stephen Robertson – the real name of the person who now works in Microsoft's Cambridge research labs, and who gave Merrill Lynch and the market such jitters. He has, however, never worked for Autonomy.
The basic principle of Bayesian probability theory differs from normal probability theory because it does not automatically assume that the chance of something being true is independent of events that went before it. For example, under normal probability theory, a coin that comes up heads 100 times in a row would still have a 50/50 chance of landing tails on the next throw, whereas Bayesian theory would weight the chances in favour of a head, given the previous 100 results.
By feeding a system such as Autonomy's a set of documents that relate to a topic, and a set that do not, it can use Bayesian probability to work out which word-groups in a given document mean the text is more likely to be about that subject. It can also go on to work out what are the synonyms for particular terms — all without any knowledge of the language in which it is searching.
Although most of the theoretical research into Bayesian probability was carried out by Robertson, it was Porter who developed a working system for Cambridge University's geology department. “They had a huge collection of objects with a lot of data to collect and analyse,” says Porter. “It was a classic information retrieval problem.” After working at it for several years, he eventually managed to build a system called Muscat, which the University said he could keep when he left, provided it could continue to use it.
“Between 1984 and 1990, I carried on developing it and put together a demo. But by 1990, I'd run out of opportunities to market it,” says Porter. However, a chance meeting with John Snyder, who had been doing similar work, led to the two founding a company called Muscat to sell the software. And with Cambridge being the small place it is, they set up shop in the offices next door to those of a firm called Cambridge Neurodynamics, run by Mike Lynch, future founder and CEO of Autonomy.
“I was at college with a guy called Mike Lynch,” Snyder notes wryly. “He and I used to go around pitching Muscat. And I put Muscat on his machines. He even had the source code because we were porting it to some Unix platforms. Mike was going to sell Muscat and give us a royalty of 15%. But eventually he went off and developed his own technology based largely on my ideas and the like.” That, says Snyder, is how Autonomy was born. Porter recalls the initial friendliness of the two companies. “At one time, there was a lot of crossover... we even used to go to their Christmas parties.”
Autonomy persevered and developed its technology, but Muscat was bought out in 1997 by MAID, the UK-based online information service, which shortly after became Dialog. “A few months later,” says Snyder, Dialog bought information services giant Knight Ridder. “It had a huge amount of debt and so there was no money to invest in Muscat. We were frozen in the wilderness.” But Muscat did not die. After a firesale, the technology assets of Dialog became BrightStation, and it maintained its London stock market listing. BrightStation subsequently placed Muscat in the care of its SmartLogik division, currently being groomed for a spin-off IPO.
Page 1 | Page 2 | Page 3 | All 3 Pages