Using another server to perform jobs?

[jordanmoore]

I'm testing a few ideas at the moment, using Apache to create a home server on the Local host.

I'm stuck at one stage, and can't figure out what I should do.

Long Explanation:
I have the main server, which has all the files on, and processes the incoming request, etc.
I wish to keep as much of the processor and ram free as possible for use by Apache to give users a quicker experience (especially with PHP reading large text files).
So that's all fine, the problem arises when I wish to use some "cron jobs", I use that term loosely as what I really wish to do is have a task that is on an endless loop, as soon as it completes it does it again.
Now, this task will be fairly demanding on system resources, and consequently lower the servers performance.
With this I thought the best idea would be to use a second computer to perform the task, so the server still has as many resources available as it usually would, and the cron jobs are being performed.

This is where the problem is.
I can't find any way that the computer can get access to the hard drive other than sharing it over the network (which could be insecure and use up resources right?).

Do you guys know any ways of doing this, which is fast and uses minimal resources?


Short Explanation:
How can I let a second computer access the files on the server to perform tasks, without reducing any performance on the server? (HTTP not an option).

 

[root]

is https an option?

honestly, if you share the files via HTTP or FTP it's going to be resource intensive.

if you share the files using shared folders, or samba it's not going to be as intensive.

you're using Cron, which makes me assume that you're running linux on both servers?

have you considered using ssl to share the files?

it'll take a bit more work to setup, but, it'll be more secure than all your other options, and realistically it's still not going to add a massive overhead to your core hosting server.

alternatively, another thing you could look at doing is putting the data on an external disk, and using a cluster aware file system.

in this way your core hosting server can access the files, and your processing server can also access the files picking them straight off the disk without taking any processing resource from the hosting server.

warning: if you go that way it's going to be long, complicated and fraught with problems.

you can do it, it's possible with USB or Firewire disks. -but you're going to have to research exactly how and what equipment you need.

if you want a slightly more high grade version, you could always use iSCSI you could host the disks on your processing server which could serve as an ISCSI target that your hosting server connects to, and that your processing server connects to, you just need to allow multiple initiators and use a cluster aware file system if both machines want to access the file system at once.

or you could use an external disk enclosure and use SCSI (or iscsi), or fibre channel.

again you'll need to be able to support multiple initiators, and you'll need to make the servers cluster aware so that the servers know that there will bo other machines connecting to the disks, you'll need to use a cluster aware file system, and you'll need to set up quorum between the servers so that each server shares access to the disk sensibly...

-and if you don't do all those things, expect corruption of your data.