finding ec2 nodes

Each and every day I find myself needing lists of host groups within EC2.  Lately it has been for building clusters of distributed Erlang and Riak, but also for adding dynamic or periodically updated lists for monitoring.

Normally I would just pipeline some shell together, but that is sub-optimal:

$ ec2-describe-instances -F tag:service=nat | grep ^INSTANCE | awk '{print $4;}'

Along the way I realized that I was rewriting similar fragments all too often, and though I usually wanted the private hostname, sometimes I needed IP address (riak – I’m looking at you!) or the public hostname.  Time to build a tool:

$ ./ec2nodefind -e test -s benchmark -i
$ ./ec2nodefind -e test -s benchmark -pF

Great!  So much easier, but I’m already on a host that is tagged, why does my config management system have to inject that info?  Lets make it autodiscover based on the instance metadata.  This requires an instance-profile role with permissions for “ec2:Describe*”.  Here we can be verbose and see the discovery values.

$ ./ec2nodefind -va
Autodiscovery: cluster benchmark
Autodiscovery: env test
Autodiscovery: service benchmark

Perfect!  Now we have automatically discovered peers without our (env,service,cluster) group.

Here’s the code:


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s