Each and every day I find myself needing lists of host groups within EC2. Lately it has been for building clusters of distributed Erlang and Riak, but also for adding dynamic or periodically updated lists for monitoring.
Normally I would just pipeline some shell together, but that is sub-optimal:
$ ec2-describe-instances -F tag:service=nat | grep ^INSTANCE | awk '{print $4;}'
ec2-1-2-3-4.compute-1.amazonaws.com
ec2-2-3-4-5.compute-1.amazonaws.com
Along the way I realized that I was rewriting similar fragments all too often, and though I usually wanted the private hostname, sometimes I needed IP address (riak – I’m looking at you!) or the public hostname. Time to build a tool:
$ ./ec2nodefind -e test -s benchmark -i
10.1.2.3
10.1.2.4
$ ./ec2nodefind -e test -s benchmark -pF
ec2-54-209-1-2.compute-1.amazonaws.com
ec2-54-209-1-3.compute-1.amazonaws.com
Great! So much easier, but I’m already on a host that is tagged, why does my config management system have to inject that info? Lets make it autodiscover based on the instance metadata. This requires an instance-profile role with permissions for “ec2:Describe*”. Here we can be verbose and see the discovery values.
$ ./ec2nodefind -va
Autodiscovery: cluster benchmark
Autodiscovery: env test
Autodiscovery: service benchmark
ip-10-1-2-3
ip-10-1-2-4
Perfect! Now we have automatically discovered peers without our (env,service,cluster) group.
Here’s the code: https://github.com/WrathOfChris/ops/tree/master/ec2nodefind
Enjoy!
WrathOfChris 