OGSAConfig Demonstration

When applications collide...

Clusters have traditionally belonged to a small community of users. When groups begin to pool their resources to form grids, the size of the user community for each individual cluster grows. Each user community in turn brings with it new applications and requirements. In many cases, these new applications may have requirements that conflict, one typical example being applications that require different versions of the same package.

Diagram of conflicting requirements

In this example, application A requires version 1.7 of the foo package, but appplication B requires version 1.1. On a traditional fabric, these demands cannot both be satisfied at the same time.

However, a machine will only usually be running either application A or application B at any given moment, not both at once. This means that although each machine should be able to run both applications it doesn't need to provide all their dependencies all the time. In an ideal situation, the node might be able to decide which packages it should have installed at any one time, depending on the application the user wishes to run.

Diagram of
reconfiguration

In a system like this, the version of the foo package that is currently needed might be installed on-the-fly just before the application runs. But how can a node know what application is about to run? One easy way is to have the user tell it just before the application is executed. It can then perform any necessary changes before the run commences.

In our implementation, the nodeswitcher allows the user to inform the node of the requirements of the job that is about run. By including a call to nodeswitcher in their job script, the user can pick a predefined configuration that is suitable for their job to run in. The system is able to ensure that the user's job runs in the correct environment.

See nodeswitcher in action.
Resource Description Format Size Link
A user session showing job submission and its output AVI 1.7 Mb Download
A monitor window showing a node's RPMs change when its job runs.AVI 1.4 Mb Download
The monitor window in its final state, showing the changed RPMs. JPEG 129 Kb Download
The monitor window in its final state, showing the changed PATH. JPEG 139 Kb Download
An administrator session observing a node's configuration log as a job causes reconfiguration AVI 122 Kb Download
The configuration log in its final state JPEG 115 Kb Download

Automated Negotiation

These techniques enable administrators to support more applications on a cluster much more easily, but there is still a major source of overhead: for each different collection of packages we need, we must pre-arrange a name for them that the nodes can recognise. This might require a great deal of human input.

A diagram of the
OGSAConfig architecture

To avoid this problem, we developed a system whereby users can arrange for new groups of packages to be identified, without the involvement of the system administrators. Of course, adding new packages to the system remains within the domain of the system administrator, but for packages which are already available to the cluster, users may select new combinations to make use of.

We provide a simple GUI interface to allow users to list the packages they desire. From here, the packages are sent to a server which makes all nodes in the cluster aware of the new configuration. The GUI displays a reference which can be passed to nodeswitcher to notify a node that this list of packages is required. The references used in the previous section were obtained this way.

Watch negotiation in action.
Resource Description Format Size Link
The GUI main display JPEG 116 Kb Download
The package entry screen JPEG 62 Kb Download
A user session with the GUI setting up an environment for their application MPEG 430 Kb Download
The main display showing a successfully deployed configuration. JPEG 128 Kb Download