Go Back   Computer Forums > General Computing > Server Administration
Click Here to Login
Join Computer forums Today


Reply
 
Thread Tools Search this Thread Display Modes
 
Old 11-20-2009, 03:56 AM   #1
Beta Member
 
Join Date: Nov 2009
Posts: 5
Default What will be better, please?

Good time of the day, everyone!
Not sure is this is the right plase to post this thread, but I'm kindly asking to be patient to me.
I've got such problem:
I'm serving network with 37 computers and shared storage for 30 TB for them. My object is to make data transmission optimized best and to all data was backup reliably. We're using Windows.
What is the best way to do this, please?
__________________

__________________
SvenKopitz is offline   Reply With Quote
Old 11-20-2009, 04:43 AM   #2
Site Team
 
root's Avatar
 
Join Date: Mar 2004
Posts: 8,107
Default Re: What will be better, please?

can you re-phrase the question.

I don't fully understand, are you trying to back up the machines? backup the shared storage? or optimise the traffic between the 37 machines and the shared storage?
__________________

__________________
I didn’t fight my way to the top of the food chain to be a vegetarian…
Im sick of people saying 'dont waste paper'. If trees wanted to live, they'd all carry guns.
"The inherent vice of capitalism is the unequal sharing of blessings; The inherent vice of socialism is the equal sharing of miseries."
root is offline   Reply With Quote
Old 11-20-2009, 04:51 AM   #3
Beta Member
 
Join Date: Nov 2009
Posts: 5
Default Re: What will be better, please?

Thanks for answer.
I want to backup everything - machines, shared storage. We are the new company, and I don't know what is the better way to do this.
__________________
SvenKopitz is offline   Reply With Quote
Old 11-23-2009, 11:25 AM   #4
Site Team
 
berry120's Avatar
 
Join Date: Jul 2009
Location: England, UK
Posts: 3,434
Default Re: What will be better, please?

There's lots of options available to you, but backing up 30TB + each individual machine's drive isn't going to be cheap. If I'm brutally honest, the fact you're implying you haven't got a reliable backup solution as a company with this much data at stake is pretty dire, you should've got this sorted as a priority before you started investing in lots of storage.

I'd personally look at first centralising all your data so everything worth keeping (i.e. all employee data and suchlike) all gets saved to a shared area, and keep that shared area as a robust RAID array (preferrably using RAID 6 here with this many drives.) This will at least give you protection if the individual drives start failing which is better than nothing - though you should then really look into off site backup solutions in case the worst happens.

I could go into lots of detail here, but if you're a registered company and you don't know how to backup things yourself reliably you really ought to be considering contracting someone to do this work for you, or at least looking at some formal training on the best approach. In this day and age, having over 30TB worth of data that's not backed up anywhere is a HUGE unnecessary risk.
__________________
berry120 is offline   Reply With Quote
Old 11-24-2009, 03:36 AM   #5
Site Team
 
root's Avatar
 
Join Date: Mar 2004
Posts: 8,107
Default Re: What will be better, please?

the best way to do this (in my opinion)...

firstly, back up all the 30 machines to the central server. have this run as a log off script in your domain so that the end users press shutdown and their work is automatically backed up to the central storage...
(you want your logoff script to do a differential copy so that it only copies things that are new, or the copy on the local machine is newer than the copy on the central server).

this will let them have a local copy of their work that they can access speedily, (assuming that a centrally stored network copy won't be sufficient).

after this you should backup your central storage to a tape drive...
(and this is where it get's really (really) expensive)...

LTO 5 tapes can store 1.6TB nativly, and almost double that when compressed
http://en.wikipedia.org/wiki/Linear_Tape-Open

if you can wait till LTO 6 then that's 3.2TB native capacity and you won't need to use hardware or software compression and your backups will happen faster... and you'll only need 10 tapes, instead of the 30 tapes that you'd currently need....

you'd also need a tape drive.
and a robotic loader...
and the backup software to control each of these...

also to back up 30 TB in an evening, you'd need something of a miracle...
a speed of LTO5 is 180MB/second...

180MB/second = 10.8GB/Minute = 648GB/hour = 15.5TB / day.

so it's going to take you 2 full days to backup your 30TB... (so that's nightly backups out the window!).

in my experience, it's difficult to get anywhere close to the max write speeds of the media...

so you end up with multiple tape drives inside the tape loaders...

you're talking thousands of dollars per LTO drive.
thousands of dollars for a tape loader robot.
the tapes themselves aren't cheap and you need at least 30 for a full backup, if not more to have a decent media rotation...
__________________
I didn’t fight my way to the top of the food chain to be a vegetarian…
Im sick of people saying 'dont waste paper'. If trees wanted to live, they'd all carry guns.
"The inherent vice of capitalism is the unequal sharing of blessings; The inherent vice of socialism is the equal sharing of miseries."
root is offline   Reply With Quote
Old 11-24-2009, 04:14 AM   #6
Wizard of Wires
 
setishock's Avatar
 
Join Date: Feb 2005
Location: Not sure
Posts: 10,030
Default Re: What will be better, please?

Could he make the first back up with all the time involved then do incremental backups from there?
__________________
setishock is offline   Reply With Quote
Old 11-24-2009, 05:10 AM   #7
Beta Member
 
Join Date: Nov 2009
Posts: 5
Default Re: What will be better, please?

Let me say with other words - we haven't got any network now - we're new company. I'm looking for solution to this situation for now.
root, I believe that tape devices are pretty expensive, but the idea is good.
setishock - yes, that is the best alternative for now.
BUT HOW?
__________________
SvenKopitz is offline   Reply With Quote
Old 11-27-2009, 04:48 PM   #8
Site Team
 
berry120's Avatar
 
Join Date: Jul 2009
Location: England, UK
Posts: 3,434
Default Re: What will be better, please?

Is the 30TB in a RAID configuration? If so what one? If you're not already I'd recommend that as a first step.
__________________
berry120 is offline   Reply With Quote
Old 12-01-2009, 10:48 AM   #9
Site Team
 
root's Avatar
 
Join Date: Mar 2004
Posts: 8,107
Default Re: What will be better, please?

for full backups then incremental,

I'm suggesting that for the workstation -> central storage that a full backup is taken first, then incremental backups from there on in.

So you have a network drive, which we'll call z: and all the work that needs to be backed up is on the local machines in a folder called d:\work

you run a logoff script that's a sime batch file that says

xcopy d:\work z:\backups\%USERNAME%\ /d /s

that makes an incremental backup, (only copies files/folders where the file is more recent on the machine than the server) to a folder that is on the Z drive, called backups, and stores the users data in a folder matching their user name...


the incremental backup to tape is a lot harder. you can't just take incremental backups forever.

basically you'll make 1 full backup so all your files will be stored on the tape.
we'll call these files 1, 2 and 3.

origin media has 1, 2 and 3 on it.

the next day file 2 changes, so file 2 is copied to the (new) backup tape,
now you have files 1, 2(original) and 3 on the origin media set and the most current version of file 2 on the tape for monday.

If throughout the week no files change the rest of the tapes for the week are blank.

the following monday file 3 changes, and this is written to the monday tape.

now you have file 1, 2 (original) and 3 (original) on the backup tape.
and you have the updated file 3 on mondays tape.

you don't have a current backup of file 2 any more because the tape is over written.


even imagining that you did have an endless supply of media, over a year you'll have 365 tapes. if you need to restore a lot of files that have only been periodically changed you'll be changing a lot of tapes...

so your only option for tape backup is the enormously expensive one!


what you really need to do is rationalise the data.
do you really need 30TB of data.... if you do need the 30TB of data does this all need to be backed up?

do you need to have your own backup at all? you could contact a company like Iron mountain who do offsite backups for you, (basically they connect to your network and backup the data remotely). -but this is an expensive option, and managed backup solutions get even more expensive when you need to restore.
__________________
I didn’t fight my way to the top of the food chain to be a vegetarian…
Im sick of people saying 'dont waste paper'. If trees wanted to live, they'd all carry guns.
"The inherent vice of capitalism is the unequal sharing of blessings; The inherent vice of socialism is the equal sharing of miseries."
root is offline   Reply With Quote
Old 12-11-2009, 06:21 AM   #10
Beta Member
 
Join Date: Nov 2009
Posts: 5
Default Re: What will be better, please?

Thank you. Please take a look at suggestion that I get on Technet:
http://social.technet.microsoft.com/...-3898b72797b0/
What do you think about that please?
__________________
SvenKopitz is offline   Reply With Quote
Old 12-11-2009, 08:55 AM   #11
Beta Member
 
Join Date: Oct 2009
Posts: 1
Default Re: What will be better, please?

I think Paul is right. StarWind seems to be pretty reliable. I have tested it once.
__________________
digitALL is offline   Reply With Quote
Old 12-14-2009, 05:27 PM   #12
Site Team
 
root's Avatar
 
Join Date: Mar 2004
Posts: 8,107
Default Re: What will be better, please?

digitALL...

can you explain how paul is right?

Starwind provides an ISCSI target...

he's got 37 PC's and want's 37 TB of centralised storage...
an ISCSI target would be great for providing A disk to A server, perhaps even starwind has multipath support so that it can be used on more than 1 machine, but windows doesn't have multipath support so all you're going to see when you hook 37 PCs up to the target at the same time is a whole load of corruption of files!

and lets face it, he wants to connect 37 PCs to a single storage device, so ISCSI is out of the window, he's already got to buy a decent server with several tons of disk space... then you're saying that he should spend a few thousand more dollars on the enterprise edition of starwind. an expensive solution that doesn't even fit the problem...

for the central storage...
there are essentially four options...

1, you buy a server with a lot of disk space

2, you buy a server and attach one of more SCSI Disk arrays to it and form the massive 37TB volume that you want.

3, you buy a server and connect to one or more ISCSI devices to make a massive disk. (but that's going to be a very flaky solution).

4, get a server, and connect it to a large fibre channel network. arguably the best (though also most expensive way) to get a 37TB singe volume for the backups (of workstations) that you want...


for backing up the central storage, as I said before lots and lots of tapes is probably your best solution. but to be honest I'd do as I suggested earlier in the thread...
rationalise the data... see what's most necessary to be backed up and start backing that up with the backup solution that you can afford now... in the future expand your backup solution as you need to...
__________________
I didn’t fight my way to the top of the food chain to be a vegetarian…
Im sick of people saying 'dont waste paper'. If trees wanted to live, they'd all carry guns.
"The inherent vice of capitalism is the unequal sharing of blessings; The inherent vice of socialism is the equal sharing of miseries."
root is offline   Reply With Quote
Old 01-20-2010, 02:32 AM   #13
Solid State Member
 
Join Date: Jan 2010
Posts: 15
Default Re: What will be better, please?

Quote:
Originally Posted by SvenKopitz View Post
Thanks for answer.
I want to backup everything - machines, shared storage. We are the new company, and I don't know what is the better way to do this.
You should have effective and reliable backup solution. As you told that you have 37 systems in your business network, so data protection is very necessary. Don't trust on various storage medias. As i got to know, that with 37 systems, your business data need to be secure.

I would recommend online data backup solution. Your whole business data get stored on a remote server and you can access your files or folders easily in case any disaster strike.

I think such data backup services would sure help you..
__________________
martin12 is offline   Reply With Quote
Old 01-20-2010, 12:45 PM   #14
Site Team
 
root's Avatar
 
Join Date: Mar 2004
Posts: 8,107
Default Re: What will be better, please?

Good luck backing up 47TB over a wan link

(practically this is an impossible option)
__________________
I didn’t fight my way to the top of the food chain to be a vegetarian…
Im sick of people saying 'dont waste paper'. If trees wanted to live, they'd all carry guns.
"The inherent vice of capitalism is the unequal sharing of blessings; The inherent vice of socialism is the equal sharing of miseries."
root is offline   Reply With Quote
Old 01-20-2010, 01:48 PM   #15
j03
~~~~~~~~
 
Join Date: Jan 2005
Location: Wales
Posts: 5,556
Send a message via MSN to j03
Default Re: What will be better, please?

This makes me wonder how huge datacenter's back-up their data... Do they do it off, or on site? Or a combination of both? It's a huge problem!
__________________
j03 is offline   Reply With Quote
Old 01-20-2010, 07:08 PM   #16
Fully Optimized
 
dude_56013's Avatar
 
Join Date: Jul 2007
Posts: 4,262
Default Re: What will be better, please?

It raises the question, how do places that back up data, back up THEIR data. Hehehehe .
__________________
dude_56013 is offline   Reply With Quote
Old 01-21-2010, 04:19 AM   #17
Site Team
 
root's Avatar
 
Join Date: Mar 2004
Posts: 8,107
Default Re: What will be better, please?

Quote:
Originally Posted by j03 View Post
This makes me wonder how huge datacenter's back-up their data... Do they do it off, or on site? Or a combination of both? It's a huge problem!
one of the companies that we support have roughly 4TB of data. that is backed up in the following way.

differential backups to tap (2 lto4 tapes) on a nightly basis (kept for one week)
full backups made on fridays, (4 LTO4 tapes) (job runs till sunday) to backup all data (Kept for 1 month)
full backups made on the last friday of the month -kept for a year
Full backups made on the last friday of the year -kept forever.

the tapes are all loaded into a robotic loader, so there is 1 tape drive, the tapes are put into a magazine and inserted into the robot and the robot picker picks up the tapes and loads it into the drive, (it decides the tape to load either by it's position in the magazine or by a bar code printed onto labels on each tape.

for another customer where we backup a lot more data we have a robotic loader that has two tape drives in it, another tape loader has four tape drives in it (that is a very large tape library that can have 24 tapes loaded at once.

but as I said before these tapes are expensive, the tape loaders and libraries are even more expensive and there is a finite throughput that can be written to the drive, (limited by the bus speed of the tape library connection bus, and the network that's going to the server that is controlling the jobs etc.

(if you allow the hours of between 6pm and 8am) you can practically backup on a single tape drive around 2TB. -that also allows time for the software or hardware to compress the data so that it actually fits onto a single drive.
(assuming that your media server is one device and that the tape drive is attached by SCSI to the one server).

to backup 37TB you can do a few things, but these are going to be limited.
step 1 would be to get an intelligent storage device that can control the tape library, that would take out a bottle neck at the network stage.

after this you're going to need to start looking at multiple tape drives.

So far as who backs up the data, the daily, weekly, monthly yearly tapes etc are collected from our data centres on a daily basis by a third party called Iron Mountain, they take the tapes away and store them in secure facilities. and deliver the next days tapes.

Practically, if you can throw enough resource at a backup problem then you can solve most through intelligent design and using the right equipment...



of course, the OPs backup requirements are 37TB.
there are 37 machines with 1TB drives,
the drives aren't going to be full,
there will be an amount, (probably at least 10GB) on each machine that is used by windows installation and program files.

there will undoubtedly be personal data that doesn't need to be backed up.

and even if the drives were absolutely full you need to figure out how much of the data is duplicated.
how many common project files are there that every one of the 37 computers each have, but you only need 1 backup copy of?

(this is why I said that they need to rationalise and actually think about what needs to be backed up).
__________________
I didn’t fight my way to the top of the food chain to be a vegetarian…
Im sick of people saying 'dont waste paper'. If trees wanted to live, they'd all carry guns.
"The inherent vice of capitalism is the unequal sharing of blessings; The inherent vice of socialism is the equal sharing of miseries."
root is offline   Reply With Quote
Old 03-13-2010, 09:38 PM   #18
In Runtime
 
Join Date: Mar 2010
Posts: 156
Default Re: What will be better, please?

what kind of biz are you in that you need to backup all the data on every machine this is nutty. You should simply have shares on the server where these people can save the stuff they need. Being in the biz the way i feel is if they really need it they will save it to mydocs (which we have redirected) or on the shares we haave out there that just a little nuts to backup all there junk gl going to cost alot
__________________

__________________
frank2085 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off



All times are GMT -5. The time now is 07:04 AM.


Powered by vBulletin® Version 3.8.8 Beta 4
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Search Engine Friendly URLs by vBSEO 3.6.0
×