Re: Using another server to perform jobs?
is https an option?
honestly, if you share the files via HTTP or FTP it's going to be resource intensive.
if you share the files using shared folders, or samba it's not going to be as intensive.
you're using Cron, which makes me assume that you're running linux on both servers?
have you considered using ssl to share the files?
it'll take a bit more work to setup, but, it'll be more secure than all your other options, and realistically it's still not going to add a massive overhead to your core hosting server.
alternatively, another thing you could look at doing is putting the data on an external disk, and using a cluster aware file system.
in this way your core hosting server can access the files, and your processing server can also access the files picking them straight off the disk without taking any processing resource from the hosting server.
warning: if you go that way it's going to be long, complicated and fraught with problems.
you can do it, it's possible with USB or Firewire disks. -but you're going to have to research exactly how and what equipment you need.
if you want a slightly more high grade version, you could always use iSCSI you could host the disks on your processing server which could serve as an ISCSI target that your hosting server connects to, and that your processing server connects to, you just need to allow multiple initiators and use a cluster aware file system if both machines want to access the file system at once.
or you could use an external disk enclosure and use SCSI (or iscsi), or fibre channel.
again you'll need to be able to support multiple initiators, and you'll need to make the servers cluster aware so that the servers know that there will bo other machines connecting to the disks, you'll need to use a cluster aware file system, and you'll need to set up quorum between the servers so that each server shares access to the disk sensibly...
-and if you don't do all those things, expect corruption of your data.
I didn’t fight my way to the top of the food chain to be a vegetarian…
Im sick of people saying 'dont waste paper'. If trees wanted to live, they'd all carry guns.
"The inherent vice of capitalism is the unequal sharing of blessings; The inherent vice of socialism is the equal sharing of miseries."