North American Network Operators Group|
Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical
Broken domain statistics...
Hi all, I'd finally had it up to here with people coming from misconfigured domains trying to connect to my servers and filling up the logs with 'Host name mismatch..' errors, so I decided to put together a bunch of scripts to try and see exactly how widespread the problem of bogus DNS info is. What I found was kind of surprising. Here're the raw stats for my test (done on the entire com zone): Total domains checked: 1401150 Domains with NO good nameservers (all responded non-auth): 236008 Domains with NO good nameservers (some timed out): 107482 Domains with at least one bad non-auth (but most/all answering):99211 Bringing that into percentages, about 17% of all domains in COM have NO good nameservers listed. If one adds in the nameservers that timed out, that number goes up to 25%, and adding in domains with at least one bad nameserver brings the number up to 32% of the domains in .com that have bad nameserver info registered. Please note, it appears that it's not entirely accurate to view the nameservers that timed out as necessarily 'bad' in my test - several known good nameservers timed out during the runs, and I only had a retry of 1 (so the nameservers got 1 chance to give the correct data within 4 seconds). It's very interesting, however, that the number of domains that had all listed nameservers respond, all of which responded non-authoritatively (i.e. 'I don't know about this domain') is so high. Here is my testing methodology: I read in the entire com zone, and when I found a line containing dom IN NS server I would spawn off a 'dig ANY dom @server +retry=1', and parse the output to see if it contained 'aa', the authoritative flag. If it did, it was a good domain. If it didn't, it was a bad domain. Timeouts and non-authoritative responses were counted separately. I then had three variables for each domain - goodResponses, badResponses, and timeouts. Domains where badResponses and timeouts were both 0 were considered 'good'. Domains where goodResponses and timeouts were both 0 were considered to have 'no good nameservers (but all responded)'. Else, domains where goodResponses was 0 were considered bad (noting that some queries timed out). Beyond that, if there was 1 or more badResponses, it was listed in the 'at least one bad non-auth NS' list. The processes that did this would fork out about 80 processes per host to run the digs; I was on a reasonably fast connection, so bandwidth shouldn't have become a problem as far as increasing the timeouts I got. I split the com zone file into 200,000 line sections and ran one section per host. I then stopped the stats collection server after every few runs to gather statistics. The queries occurred over a period of 12 hours between 3pm and 3am Pacific Time, Tuesday 2/9 - Wednesday 2/10. I've put up the code, results, and the logs of non-auth queries and timed-out queries at ftp1.dal.net:/pub/misc/domain-test/. The files haven't (and won't) propogate to the mirror sites. (Note, this machine will be switching IPs sometime this week, so there may be a period of an hour or two when the machine will be unreachable.) If I've got some sort of flaw in my logic, please let me know; I'm willing to correct it and run the test again. But it looks right. I haven't tested the net/edu/org domains, but I suspect that since folks using those are slightly more clued than the folks using .com, the numbers of bogus domains will be lower. And if it is, it means that 17% of the folks on the internet are paying for domains that don't work. Either that, or something else is broken. I'm posting here because I feel it is an operational issue; that, plus I feel there're more folks here who can and will hammer at InterNIC to start doing something to enforce their policies that require real, authoritative nameservers. One last request - if you plan to use this data somewhere, *please* listen to any responses that may show up here explaining how it might be wrong, and *please* go through my methodology and find out for yourself if it looks right. I don't want to be responsible for any false/overinflated claims out there }:P . And please provide context, too, especially where the 'nameserver timed out' statistics are concerned. Anyhow, there it is. -dalvenjah -- Dalvenjah FoxFire (aka Sven Nielsen) Stupid people shouldn't breed. Founder, the DALnet IRC Network e-mail: [email protected] WWW: http://www.dal.net/~dalvenjah/ whois: SN90 Try DALnet! http://www.dal.net/