Website crawlers?

HELLFIRE452

Solid State Member
Messages
12
Location
USA
what the best website crawler or spider to use to copy a hole site offline i what copy backup so if there shut down site i whatn't lose info and have to go find or pay for it most of the one i have found are trail verison no free one that are up to date help please
 
If it's your site simply doing a database back up and a home directory backup will be the solution.
As for sending a bot to someone else's site. Well let's just say that's not real bright. You can be back traced. It could get serious.
Just a word of advice. Web site owners can be a testy bunch. I'd think twice before you did that...
 
no not that i just what a copy of a hole website just so if they shut down in a year or
two i whatn't be lose any info i got off the site
 
just trying to find a program or way to copy all that on a website i found this how to site a couple years ago and it was all in txt format and had a zip or rar download or all the file on the site and i been try to see if there some way or program for that i wood love have a copy of fanfiction.com or some there site i go to just in case if they
shut down
 
I used to use a program called websnake.
by a company called annawave.

I don't know if it's still being made, but that does exactly what you wanted.
 
look and can't find it but thanks for help i been traying to find a freeware verison of this software or
program like and no one what to help or point me to info like this i wood like find a freeware but if
i have to buy one i what to buy one that will setup the same setup as it is online all link together you click on storie and can go chapter to chapter or go to the author page and read up on them and and click one of the storie and go and read them that what i wood like can any tell be witch one of the buy
programs are the best and fast at downloading?
 
Google

paid version is top link. (it's worth the $30 to do what you want?)
4th and 5th links are not paid for links.

not that I'm promoting 'that' kind of thing obviously.
 
than what pay for one wood be the best? becuase i what to save it like save a web page and than it save all linking pages to one i save so the hole web site will copyed i read fanfiction stories all the time i wood love for it to work so that i only have to download one time and the programs download all the chapter after that? and thanks for the help
 
that mirrors a whole site, it does it by saving the text and pictures of the site that you're on to your local computer.

also you set a link depth, so it follows links on the first page to pages inside the site.

the tool os meant for mirroring whole websites.

it is exactly what you want to copy a whole website off line. anything that is a link that you can click the program will follow and mirror the content at the end of that link.
 
When your bot starts sucking down massive amounts of bandwidth you're going to get either the host's attention or the webmaster's. Either way the bot's gonna get blocked.
 
Back
Top Bottom