finding ec2 nodes

Each and every day I find myself needing lists of host groups within EC2.  Lately it has been for building clusters of distributed Erlang and Riak, but also for adding dynamic or periodically updated lists for monitoring.

Normally I would just pipeline some shell together, but that is sub-optimal:

$ ec2-describe-instances -F tag:service=nat | grep ^INSTANCE | awk '{print $4;}'
ec2-1-2-3-4.compute-1.amazonaws.com
ec2-2-3-4-5.compute-1.amazonaws.com

Along the way I realized that I was rewriting similar fragments all too often, and though I usually wanted the private hostname, sometimes I needed IP address (riak – I’m looking at you!) or the public hostname.  Time to build a tool:

$ ./ec2nodefind -e test -s benchmark -i
10.1.2.3
10.1.2.4
$ ./ec2nodefind -e test -s benchmark -pF
ec2-54-209-1-2.compute-1.amazonaws.com
ec2-54-209-1-3.compute-1.amazonaws.com

Great!  So much easier, but I’m already on a host that is tagged, why does my config management system have to inject that info?  Lets make it autodiscover based on the instance metadata.  This requires an instance-profile role with permissions for “ec2:Describe*”.  Here we can be verbose and see the discovery values.

$ ./ec2nodefind -va
Autodiscovery: cluster benchmark
Autodiscovery: env test
Autodiscovery: service benchmark
ip-10-1-2-3
ip-10-1-2-4

Perfect!  Now we have automatically discovered peers without our (env,service,cluster) group.

Here’s the code: https://github.com/WrathOfChris/ops/tree/master/ec2nodefind

Enjoy!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s