Negative Caching in DNS

by
, posted

Resolving Domain Name

When we visit a domain from our browser, the first thing browser do is to ask the DNS resolver for the IP address. If the IP is found in the DNS cache the resolver hands over the IP to browser and the request is made to the server & the website is served in return. If the IP is not found in the cache, the resolver asks the authoritive nameservers for the IP which is eventually sent to the resolver if it is a valid domain name.

Subdomain Not Loading

Here’s is what happened today. I own this domain name cosmicqbit.dev & I thought to host my knowledge wiki on subdomain wiki.cosmicqbit.dev. Before setting the CNAME records for the subdomain I visited it in my browser and it wasn’t working which was expected. I logged into the DNS console of my domain on Cloudflare and added the CNAME record for wiki.cosmicqbit.dev.

After integrating my wiki which is hosted on cf-pages with subdomain I opened the subdomain in browser & it wasn’t working. The site was accessible on pages.dev subdomain which Cloudflare generates by default. The situaton instantly reminded me of a concept called negative caching that I had learned from an amazing zine How DNS Works by Julia Evans.

Resolvers Cache Negative Results

Here is what was happening here actually! So when I visited the subdomain for first time there existed no records in the local DNS cache so the request was sent to authoritative nameservers. The DNS records for the subdomain weren’t set so the authoritative nameservers returned with no A records for subdomain, the DNS cached ‘does not exist’ in the cache. The good part is the records saved in DNS cache have a TTL (Time to Live) which comes from the SOA record.

SOA records control the negative caching TTL. We can’t change ‘em unless we run our own authoritative nameserer. Running dig SOA wiki.cosmicqbit.dev

cosmicqbit.dev. 3600 IN	SOA	addilyn.ns.cloudflare.com. dns.cloudflare.com. 2309632975 10000 2400 604800 3600

The TTL is the smaller of two numbers: the numbers in second & last column of output. In this case it is 3600 is both cases, whcih is equivalent to 1 hour. Implying my DNS cached record of subdomain will expire after an hour and then the resolver with make a new request to the authoritative nameservers. So I waited for 1 hour and it worked.

How to Avoid this

Simply don’t visit the site before creating its DNS record and if you have accidently done wait for the DNS cache to expire.

Hope this was helpful. You can reply via mail.

Your Signature