[WEB4LIB] robots.txt file: Unix linebreaks

[Eric Hellman]

You don't mention what the server is. Many web servers have an option 
that defines "TEXT" actions which automatically replace line endings 
with <cr><lf>.

Eric

At 9:43 AM -0800 12/2/99, SUSIE BREIER wrote:
>I have been working with our university's network administrator on getting
>our library site properly indexed in a newly installed  university-wide
>search engine (http://dig). Some of the folders that I disallowed in my
>robots.txt file were nevertheless included in the index. Apparently this
>is because the line endings are of the Mac OS variety <CR>, when the robot
>accepts only the Unix variety <LF>. I used my editor's (BBEdit lite) "Save
>as: Linebreaks: Unix" option, but according to the administrator this did
>not do the trick and the line endings are now of the <CR>+<LF>. Variety.
>
>So, finally my question, to anyone out there who might have dealt with
>this before: how can I ensure that the line endings are (and remain) Unix
><LF>? I have been sent a file with the correct line endings by the
>administrator, but am afraid that if I save the file with BBedit again (or
>any other Mac or windows wordprocessor for that matter) in order to put
>it on our server, the linebreaks will revert back to MacOS.
>
>TIA for any hints out there!
>
>Susie Breier
>Assistant Systems Librarian
>Concordia University Libraries
>1400 de Maisonneuve W, LB 303-3
>Montreal, Quebec

Eric Hellman
Openly Informatics, Inc.
http://www.openly.com/           21st Century Information Infrastructure