Sharing software and libraries in a cluster

Hey!

I'm planing on having several identical servers as PHP web application servers (among other things). Instead of having to install and compile software on each server individually, I've decided to compile and install everything on a deployment server and then use those files on all servers. To ease the deployment of all this files, I've decided to install everything shared into /cluster/sw directory. When compiling software I use --prefix=/cluster/sw and add /cluster/sw/lib into /etc/ld.so.config file. So far so good and most of the software I compile works. /cluster/sw is exported over NFS.

But just now, when trying to compile RabbitMQ client for PHP, I got an error, saying it cannot find some file. When I installed the depending library (Rabbit-C client) without the --prefix=/cluster/sw option, the PHP client compiled just nice.

What I'm asking is this: What else I need to do so I can compile and run everything from /cluster/sw shared directory?

From serverfault Matic

• I wouldn't let the nodes load their environment from network unless it's only loaded once and then used from the local memory.

  Instead you could create packages for the Linux distributions you are using and use a custom repository to serve them and install your on the nodes. If you have several identical nodes, this is far more economic than compiling the programs on each node individually.

  The configuration of each node could be automated using a program like Puppet, cfengine, bcfg2 or Chef which will also take care that your custom packages are installed on each machine.

  Matic : I've considered building .DEB packages (I use Debian), but it's too much work, especially when we will scale to 100s of servers in the future. It's not what we want.

  joschi : Especially when scaling in a high number of servers custom DEB packages start to pay out: build once, install everywhere. ;) Unless of course you *really* need to compile the programs for each node individually.

  Matic : The main problem I had with creating DEB packages was that when you do make install, files go into lots of different directories and manually finding/copying them into your DEB package environment takes a lot of effort.

  joschi : But consider this being a onetime effort compared to much more complicated and less performant/secure/stable solutions. I can assure you that acquiring knowledge about the package building process in Debian Linux pays out in the end - especially when managing a lot of similar systems.
  From joschi