Cluster Support for Genetic Programming

is not all that hard, actually, so huzzah for that. I modified John’s script slightly to launch dummy worker processes (they just sleep for ten seconds) on the compute nodes, that then write out a file to my homedir on the NFS share. Looks like running one command via ssh on each desired node will more or less accomplish some of the sorts of things we intend to accomplish. Not that this is shocking, I just wanted to convince myself that I could do it.

I wrote a little script that uses ganglia (The cluster monitoring toolkit which was already installed on the cluster) to generate lists of nodes sorted in order of lowest cpu usage. It can also sort by memory free and several other metrics.

Example usage (Get 10 nodes, sorted by lowest cpu usage):

[jms07@fly ~]$ ./gquery -c 10
compute-0-2
compute-0-3
compute-0-4
compute-0-5
compute-0-6
compute-0-7
compute-0-8
compute-0-9
compute-1-5
compute-1-10


Changelog
Tarball
Git: git://anomos.info/~john/ClusterSupport

-John

I’ve made the source code for my Erlang implementation of Push available at anomos.info/~john/. I’m currently calling it PushAgner, this is a terrible name (Erlang is named after the inventor of queuing theory, Agner Krarup Erlang), please help me come up with a better one.
EDIT: I’ve renamed it to Erlush, which was another working title I had used, and was suggested by Lee.

If you have git installed, you can get the source by running:
$ git clone git://anomos.info/~john/erlush
Following which you can get any updates by running:
$ git pull

If you don’t have git, you can just grab a tarball of the latest source.

I haven’t written any documentation yet. If you really want to get this running it’s probably best to contact me directly.

– John

7:17PM: We are meeting.
7:18PM: We are talking about what we want to accomplish.
7:19PM: John is testing the use of CoDeploy on the cluster as a potential tool for deploying our code across the cluster.
7:27PM: We are discussing the feasibility of leaving Erlang interpreters idling on cluster nodes.
7:47PM: We are getting caught up doing weird things with ssh-agent.
7:51PM: We have resolved to meet with Josiah on Wednesday.