Command Line http request

[Thomas Dowling]

> > I would like to set up a small script that would automatically 
> > request each document in the directory through http.  My 
> > question is how do I communicate to a Web server from a
> > UNIX command line.  
> 
> If you're in the UNIX environment, then another approach to this
> (other than Expect) would be to call the lynx browser from your
script 
> with the -s option to get back the source of the page you've
requested. 
> That's probably the easiest thing to do...
> 

If you have a preset list of URLs, you could write them into a file and
do something like

foreach url (`cat list-of-urls.txt`)
  lynx -dump $url >> save-results-here.txt
end

Apologies if I've mangled the syntax here (after all these years, I
still ought to keep my Unix for Drooling Morons book by my side).

Thomas Dowling
Ohio Library and Information Network
tdowling at ohiolink.edu