dsh - distributed shell on Mac OSX

We have several hundred computers at work. One of the tools we use to interact with these machines is dsh, also known as dancer's shell or distributed shell. You can use dsh to run a command on a group of machines. For example, you could run the following command to find out what operating systems are running on a group of machines:

dsh "uname -a"

I'm going to give a short tutorial on how to set this up on the Mac for a group of machines with ssh available. First of all, I'm using MacPorts, so you'll need to have that installed before proceeding. With MacPorts installed, you can run the following command:

sudo port install dsh

After ports has done it's thing, you will need to create the proper directories:

mkdir -p ~/.dsh/group

The '-p' tells mkdir to create both .dsh and group directories. Neither of these directories existed for me. Now we need to create a 'group' file with the names of the machines we want to access:

vim ~/.dsh/group/testMachines

Add machine names to the file with one machine per line. For example:

pluto
mars
saturn
earth

You can create as many files in the ~/.dsh/group directory with this format. We will use these files to organize what machines we're sending commands to.

Now we need to create a config file:

vim ~/.dsh/dsh.conf

with the following contents:

verbose=0
remoteshell=ssh

waitshell=1  # whether to wait for execution
# 1 = run commands serially
# 0 = run commands in parallel

remoteshellopt=-lmmasters # mmasters is the login id for the machines
verbose=0
showmachinenames=1

The last thing we need to do is set up passwordless ssh for all the machines in our group file. Once that's done, we can start using dsh. Go ahead and give it a test drive:

dsh -g testMachines 'uname -a'

The -g argument tells dsh which group file we want to use.

Load Balancing Software

I was talking to one of our system admins the other day about load balancing. He suggested I look into HAProxy. Apparently, gifts.com uses this for their load balancing rather than a hardware layer. One of the features I'm looking for is the ability to handle sticky sessions. I didn't see anything about it on the home page, but it's suppose to support it. This is something I'd like to look into at a later time:

http://haproxy.1wt.eu/

List of commands

I ran across a nice page of linux commands. Some of them I already knew, but there were several that were new to me.

http://www.pixelbeat.org/cmdline.html

spotlight on the command line

I've recently learned about mdfind on OSX. mdfind is essentially a commandline interface to Spotlight. One of the benefits of using mdfind over locate is that the indexing is done in real time. As soon as the file is added to the file system, it is added to the index.

You can read up on the command at:
http://macdevcenter.com/mac/2006/01/04/mdfind.html

Sending screen commands

You can start a detached screen session using the following command:


screen -dm -S myTest


This will create a screen session named 'myTest'. You can verify it was created with 'screen -list'. You should see something similar to the following when you type that command:


$ screen -list
There is a screen on:
31306.myTest (Detached)
1 Socket in /tmp/uscreens/S-mmasters.


Then you can send commands to the newly created screen session:


screen -S myTest -p 0 -X stuff 'top^M'


You have to type <control-v><control-m> to get the ^M character correctly. This command will essentially type 'top' and 'enter' in the screen session named 'myTest' for the 0th window. When you reconnect to the screen session, you should see top running. You can type 'q' to quit top.

The important part of this command is the -X. This switch can take several different commands. You can see all the commands available at

http://www.gnu.org/software/screen/manual/screen.html#Command-Summary

I found this link to be very helpful.

Tab Completion

Ever since I've used cvs and subversion I've wanted to exclude the CVS directories and .svn directories from the tab completion list. In the past, I've lackadaisically looked for a solution to this. This problem comes up quite a bit for me when I'm inside the directories of a java project. Most of the time people will break their classes into separate packages, which is exactly what they should do. Hence, you get packages like com.ibm.ltc.eclipse.foobar which correspond to a directory structure like com/ibm/ltc/eclipse/foobar. The problem is that all the classes might be in the foobar directory. Most of the time, in a situation like this, it's nice to start above the com directory and continuously hit tab until you get to the last directory. This assumes there's nothing in the other directories. But if you're using subversion, then there's a .svn directory in each of them. This prevents you from using the tab completion as I've described because it wants you to choose between com and .svn. NO MORE MY FRIEND! I finally got a little frustrated with this today.

I use bash, so I used the following in my .bash_profile to solve the problem:

export FIGNORE=CVS:.svn

This will ignore the CVS and .svn directories for tab completions.