A rant about User-Agent strings

Web browsers tell a web server their name with every request they make. This is done by the User-Agent header which clients should (according to RFC2068, section 14.42) be included with all requests.

Web server log file analysis tools will often process these User-Agent strings to see the types of browsers which are accessing a site, however since there is no real standard for the format of the user agent strings, a lot of guess work has to go into processing the values passed by the clients into something useful. Whilst the major browsers like Netscape and IE are supported there are many browsers in use out there and there is no way to support all of them.

This situation is in no way helped by a large number of browsers returning the string

Mozilla/[0-9.]+

at the front of their user agent strings. Even IE does this ie:

Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)

The above being a copy of MSIE 5.0 running on Windows NT. This state of affairs is thanks to histerical reasons. In the dim and distant past of the world wide web a company called Netscape was formed, which took the NCSA Mosaic browser and created Mozilla - better known as Netscape Navigator. Now Netscape started to do lots of work on their browser, adding such fun tags as the annoying <blink>...</blink>, <center>...</center>, the often misused <frame>...</frame> and the vaguely useful <table>...</table>. At the time, other browsers didn't support these non-standard extentsions to the HTML language. So people started to change web pages to say the ever-so-annoying "This page best viewed on Netscape foo.bar or above". Other people altered their web servers or wrote CGI scripts to return back different HTML depending on your web browser. So if you were using Netscape you'd get the enhanced version. If you were using another browser you got the un-enhanced version.

As a result of this, new browsers which did support Netscape's non-standard tags had to put "Mozilla/[0-9.]+" at the beginning of their User-Agent string.

These days you also have people using browsers which allow them to alter the contents of the User-Agent string sent to the web servers they make requests to. Its not really possible to say if a given request really did from the browser mentioned in the User-Agent header.

So take the information returned in the User-Agent header with a hefty pinch of salt.