[WEB4LIB] HTTP User-Agent Variables

[XIST]

On Monday, March 22, 1999 at 5:39 PM, "Michael Sauers" (msauers at bcr.org) wrote:
> I'm putting together and advanced HTML class and will be giving a
> (Java)Script to the attendees to determine which browser a user has and
> redirect to the appropriate alternate page.  I need a list of valid options
> for the User-Agent variable in the HTTP headers.  Can anyone send me one or
> point me to a site that has it.  I've given up looking.

Unfortunately, there's no such list.  That header is optional(1), and those user 
agents that include it can fill it with any value they like(2).  Even were that 
not the case, there are literally thousands of "user-agent" strings currently 
being used:  not only by browsers (a very partial list of which can be found at 
Yahoo!(3)) but also by search engines, of which there are over 3000(4).

There are some lists, such as those at BrowserWatch(5) or UIUC(6), which show 
the values of "user-agent" sent to them by programs visiting their site:  as 
you'll note, such lists tend to be quite long.  Even so, they aren't complete, 
owing to the effects of caching:  those stats do not count browsers which 
retrieved the page from a cache(7) instead of fresh from the server.  In short, 
even to list every "user-agent" value being sent within a given period would be 
impossible; even more so would be to identify the capabilities of the program(s) 
sending that "user-agent" value.

The end result is that all scripts of the type referred to in the original 
message are unreliable:  they cannot help but misidentify at least some 
browsers.  Furthermore, as time passes they quickly become out of date -- 
facilities which one version of a program does not have, the next version does, 
so the script requires updating.

The good news is, there should be no reason to force the user to a 
browser-specific page in the first place.  The World-Wide Web was built on the 
principle of universality:  any browser, running on any computer, can display 
any page.  With only a small amount of care, it is easy to create Web pages 
which are useful to everyone without exception:  resources such as HTMLhelp(8) 
and Bobby(9) are very useful in providing guidance on such issues.  Often all 
that is needed is to provide sensible text for alt attributes and noframes 
regions, and one page can serve for all users.

Regards,

Kivi Shapiro
XIST Information Services & Technology


(1) see 
<URL:http://www.w3.org/Protocols/HTTP/1.1/draft-ietf-http-v11-spec-rev-06.txt>, 
section 4.43.

(2) Generally, this is done by the authors of the software itself, leading to a 
certain consistency among users of that program, but it is often possible to 
change the value (see <URL:http://www.ryan.net/fakeMSIE.html> for some ideas on 
how to do this).  Other programs, such as Lynx, make it very easy for their 
users to change that value to whatever they choose.

(3) 
<URL:http://www.yahoo.com/Computers_and_Internet/Software/Internet/World_Wide_We
b/Browsers/>.

(4) <URL:http://www.isleuth.com/about.html> -- also, see 
<URL:http://info.webcrawler.com/mak/projects/robots/active/all.txt> for a list 
of some common robots and their "user-agent" values.

(5) <URL:http://browserwatch.internet.com/stats.html>.

(6) <URL:http://www.cen.uiuc.edu/bstats/months/>

(7) such as the one held (at least at one time) by America OnLine, see for 
example <URL:http://xent.ics.uci.edu/FoRK-archive/summer96/0079.html>

(8) <URL:http://www.htmlhelp.org/>

(9) <URL:http://www.cast.org/bobby/>