Hiding draft pages from browsers, search engines

[morganj at iupui.edu]

I sometimes have draft pages in a web directory.  I have assumed that if
there are no links to these pages, others would have to know the name of
the file to get to them.  One exception seems to be if there is no index
or home file in the directory, the browser returns a list of files in the
directory.  However, if there is an index or home file is there any way to
force a browser to bypass it and list the files in the directory?

Secondly, can these "hidden" files be indexed by search engines?  I know
that same-site search engines, like Swish, index all files in the
directory that are not specially marked, and that a Harvest gatherer can
be pointed to index directories on the server as the gatherer (rather than
use html links).  However can search engines be set to ignore the index
and home files and index all files in a directory on a remote web server?

This came to mind when I contemplated having a local email directory, and
began to think about how to make it available to individuals within the
library but not to email spammers search engines.

Jim Morgan
morganj at iupui.edu