North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: DNS - connection limit (without any extra hardware)

  • From: Fergie
  • Date: Fri Dec 08 14:52:33 2006

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sorry for the top-post, but wanted to retain context here.

Also, sorry for the specific product mention, but much of is
mentioned below is something that we are doing with ICSS/BASE:

 http://www.trendmicro.com/en/products/nss/icss/evaluate/overview.htm

$.02,

- - ferg

- -- Joe Abley <[email protected]> wrote:

On 8-Dec-2006, at 11:52, Geo. wrote:

>
>> Actually, reading your reply (which is the same as my own, pretty  
>> much), I
>> figure the guy asked a question and he has a real problem.  
>> Assuming he
>> doesn't want to clean them up is not nice of us.
>
> Infected machines (bots) will cause a lot more than just DNS  
> issues. Issues
> like this have a way of getting worse all by themselves if not  
> addressed.
>
> Anyway, to play nice.. how about using a router to dampen traffic  
> much like
> icmp dampening? Would it be possible to do DNS dampening?

I think the trouble comes when you want to limit the request rate  
*per client source address*, rather than limiting the request rate  
across the board. That implies the retention of state, and since DNS  
transactions are brief (and since the client population is often  
large) that can add up to a lot of state to keep at an aggregation  
point like a router.

There some appliances which are designed to hold large amounts of  
state (e.g. f5's big-ip) but you're talking non-trivial dollars for  
that. Beware enterprise-scale stateful firewall devices which might  
seem like sensible solutions to this problem. They are often not  
suitable for use in front of busy DNS servers (even a few hundred new  
flows per second is a lot for some vendors, despite the apparent  
marketing headroom based on the number of kbps you need to handle).

You may find that you can install ipfw (or similar) rules on your  
nameservers themselves to do this kind of thing. Take careful note of  
what happens when the client population becomes large, though -- the  
garbage collection ought to be smooth and painless, or you'll just  
wind up swapping one worm proliferation failure mode for another.

Host-based per-client rate limits scale better if there are many  
hosts providing service, e.g. behind a load balancer or using  
something like <http://www.isc.org/pubs/tn/isc-tn-2004-1.html>.

As to the wider question, cleaning up the infected hosts is an  
excellent goal, but it'd certainly be nice if your DNS servers  
continued to function while you were doing so. Having every non- 
infected customer phone up screaming at once can be an unwelcome  
distraction when you already have more man hours of work to do per  
day than you have (staff * 24).


Joe

-----BEGIN PGP SIGNATURE-----
Version: PGP Desktop 9.5.1 (Build 1557)

wj8DBQFFebFQq1pz9mNUZTMRAk+xAKCg1dPMivTo6ee5Nj1I4yjVXQzvCQCgnBSI
NV3RnsEijPJcHNawWS4uWog=
=pawb
-----END PGP SIGNATURE-----

--
"Fergie", a.k.a. Paul Ferguson
 Engineering Architecture for the Internet
 fergdawg(at)netzero.net
 ferg's tech blog: http://fergdawg.blogspot.com/