Analysing your website traffic

By now it is common knowledge that every time someone visits a website they leave fingerprints that you can be used to analyse their web-site traffic.

For small, specialist business-to-business (B2B) sites the number of hits to your site may be small enough to analyse by hand but for larger consumer facing (B2C) sites, you will need more specialist tools to automate much of the analysis (eg WebTrends or Visual Insights although there are some free resources too).

Many websites are now becoming so big new graphical ways of summarising and analysing the data are being used. We have a number of propriety methods for summarising website usage.


Web-traffic basics

Each web-site is held (hosted) on a computer called a web-server (probably at your ISP) that responds to the requests for pages from the visitors to your site.

As each visitor moves around your site, the web-server records who asked for what and when in a logfile. You should be able to easily get hold of this logfile from the web-server and some ISPs will also provide you with web-site statistics based on this logfile. Below is an example of what you will see in a log-file.

On the first line here you can see an IP address of a computer (216.72.94.70) who accessed the site on the 6th September at 20:24. They looked at the segmentation page and came from to the site from the Market Research Information Forum using Netscape (Mozilla).

The remaining lines are the graphics images supporting the page. In total this took 10 seconds for them to download. And if, they had gone to another page you would have been able to see how long they spent on this particular page before they switched somewhere else. Consequently, you can see who visited, when and what they looked for and for how long.

You can translate the IP address to a named organisation via the RIPE database for computers in Europe, ARIN in the US or APNIC for Asia Pacific regions (if you can't find it on one of these sites, try another). The IP 216.72.94.70 refers to a US company called Global One.

Note though you cannot identify an individual this way, if you are selling business-to-business it may be sufficient to know someone from an organisation has visited your site. If you have just sent out a mailing, it may be sufficient to put 2 and 2 together to identify the likely visitor.

Much simple web-analysis can be carried out using tools like Excel and Pivottables.

For more sophisticated tracking of customers (and sessions), to track which visitors come to your site and to see if they return, you can use "cookies"  to log different customers on the site at any one time. Unfortunately this needs a bit of HTML and Javascript experience. If you combine these cookies with a required user registration (which will put some people off your site), you can track specific individuals.

Although cookies allow the tracking of individuals between sessions on your site, they cannot track which other web sites your customers visit regularly. If you are placing banner ads this can be extremely useful to know, but to track this sort of information you must have the users permission, not least because it requires a special piece of software (or access to the users' history file). Some banner ad companies can go some way to solving this by keeping track of visitors seeing any of their ads, but they do not have information on the sites without their banners.

Cookies are also tied to machines and if a customer has a different machine at home to work, you can find that the information is lost. Amazon came unstuck when it was trying "price testing" with different prices for different users by this distinction. There is nothing hated more on the Internet than underhand dealings.

216.72.94.70 - - [06/Sep/2000:20:24:03 +0100] "GET /segmentation.htm HTTP/1.0" 200 7557 "http://www.marketresearchinfo.com/forum/index.cfm/fuseaction/thread/CFB/1/Tid/1150/DoOnePage/Yes.cfm" "Mozilla/4.7 [en] (Win95; I)"
216.72.94.70 - - [06/Sep/2000:20:24:05 +0100] "GET /_themes/water/wate1011.css HTTP/1.0" 200 14026 "-" "Mozilla/4.7 [en] (Win95; I)"
216.72.94.70 - - [06/Sep/2000:20:24:12 +0100] "GET /compass3.gif HTTP/1.0" 200 10401 "http://www.dobney.com/segmentation.htm" "Mozilla/4.7 [en] (Win95; I)"
216.72.94.70 - - [06/Sep/2000:20:24:12 +0100] "GET /_borders/_derived/left.htm_txt_Water_side.gif HTTP/1.0" 200 10874 "http://www.dobney.com/segmentation.htm" "Mozilla/4.7 [en] (Win95; I)"
216.72.94.70 - - [06/Sep/2000:20:24:12 +0100] "GET /logo.gif HTTP/1.0" 200 1387 "http://www.dobney.com/segmentation.htm" "Mozilla/4.7 [en] (Win95; I)"
216.72.94.70 - - [06/Sep/2000:20:24:13 +0100] "GET /images/Water_rule.gif HTTP/1.0" 200 1747 "http://www.dobney.com/segmentation.htm" "Mozilla/4.7 [en] (Win95; I)"

The other alternative to tracking just via your website, is to use Google Analytics (Urchin), or one of the targeted ad networks. If you sign up to some of the ad networks, a cookie is placed on a page on your site, and on the site of the advertiser. It is then possible for the ad network to monitor who visits each site and in which order.

Another alternative is to include Google ads on your site. From these details, Google can provide information about the number of page views and click behaviour. Similarly Google ads (or any other ad network) can be used to bring traffic to the site. By using things like reference IDs and landing pages, it is possible to target different customers with different campaigns and to understand which adverts are generating the best return on investment.