Re: Performance & Scalability Considerations
sounds like you are giong to have to shell out quite a lot of money regardless of your final solution...
At the last place I worked we setup a system simillar to this for the purpose of an online point of sale system to server the entire of europe, (many thousands of outlets).
so I'll give you a couple of suggestions...
Data minning is always going to be resource intensive.
this is my suggestion.
first use linux.
get two duel processor servers each with stacks of Ram, each with onboard SCSI drives.
then get an external SCSI drive with two SCSI connections, (something like a Dell Power Vault).
then get a really fast database server (something like Oracle). and set up a cluster of the database servers with shared harddisk storage (you'll need at least two network cards in each Database server, 1 in each for outside connectiovity, on to connect to the other (with a crossover) for the heartbeat to keep the database syncronised.
congratulations you just spent,
£1000 per server
£1000+ on a power vault
£9000 on a database program,
BUT... you'll have a premium setup that will easily serve several hundred concurrent connections over a massive database, (which could be thousands of concurrent website connections).
Now for the webservers...
it's difficult to say what you'll need here, it really depends on the machines that you have, and howmany connection that you want to serve...
look in the default httpd.conf file as to the active threads allowed, now you can increase this (to as much as even four fold the default). but performance will suffer, it's probably best to thikn about how many concurrent connections that you'll have and got from there with a new server.
e.g. say you want to have 50 active threads.
but expect 130 connections at any one time, get three servers.
Load balancing is something that I'm not too hot on so I'll refrain from suggesting anything.
I didn’t fight my way to the top of the food chain to be a vegetarian…
Im sick of people saying 'dont waste paper'. If trees wanted to live, they'd all carry guns.
"The inherent vice of capitalism is the unequal sharing of blessings; The inherent vice of socialism is the equal sharing of miseries."