Blocking web traffic behind an Elastic Load Balancer (ELB)

Over the past few hours we’ve been on the receiving end of a fairly large scale set of web requests (read: attack) to a website we host over on Amazon EC2. Our setup is not really that complicated, however we encountered a problem that wasn’t that easy to solve.