An upgrade to China’s Great Firewall is having knock-on effects all over the internet, with seemingly random sites experiencing massive traffic spikes.
One site owner in North Carolina, Craig Hockenberry, has written up how, after he looked into why his mail server was down, he found 52Mbps of search traffic piling into his system: 13,000 requests per second, or roughly a third of Google’s search traffic.
The post goes into some detail over howHockenberry managed to deal with the firehose-blast of requests, all of it coming from China and much of it trying to find Bittorrents or reach Facebook. Short version: he blocked all of China’s IP blocks.
Hockenberry is not the only one dealing with a sudden flood of requests, though. There are numerous reports of sysadmins finding that their IP address has appeared in front of the headlights of the Chinese government’s censorship juggernaut, causing them to fall over and forcing them to introduce blocking measures to get back online.
After a number of different theories about what was happening, including focussed DDoS attacks and “foreign hackers” – that suggestion courtesy of the Chinese government itself – the overall conclusion of the technical community is that bugs have been introduced into China’s firewall. Particularly, something seems to have gone wrong in how it uses DNS cache poisoning to redirect users away from sites the government doesn’t want them to see.
China uses a weak spot of the DNS system to intercept requests coming into and going out of the country. If it spots something it doesn’t like – such as a request for “facebook.com” or “twitter.com” – it redirects that request to a different IP address.
For a long while, China simply sent these requests into the ether – i.e. to IP addresses that don’t exist, which has the effect of causing the requests to time out. However, possibly in order to analyze the traffic more, the country has started sending requests to IP addresses used by real servers.
Unfortunately, it seems that there have been some configuration mishaps and the wrong IP addresses have been entered. When one wrong number means that a server on the other side of the world suddenly gets hits with the full stream of millions of Chinese users requesting information, well then … that server falls over.
The situation has had a broader impact within China. Tens of millions of users weren’t able to access the Web while the government scrambled to fix the problem. According to one Chinese anti-virus vendor, Qihoo 360, two-thirds of Chinese websites were caught up in the mess.
China’s DNS infrastructure experts started pointing the finger at unknown assailants outside its system. “The industry needs to give more attention to prevent stronger DNS-related attacks,” said Li Xiaodong, executive director of China’s Internet Network Information Center (CNNIC).
Your own medicine
The reality, however, is that China has seen the downside to its efforts to reconfigure the basic underpinnings of the domain name system to meet political ends. The network is designed to be widely distributed and route around anything that prevents effective communication.
By setting itself up as a bottleneck – and an increasingly huge bottleneck as more and more Chinese users get online – the Chinese government is making itself a single point of failure. The slightest error in its configurations will blast traffic in uncertain directions as well as cut off its own users from the internet.
For years, experts have been warning about the “balkanization” of the internet, where governments impose greater and greater constraints within their borders and end up effectively breaking up the global internet. What has not been covered in much detail is the downside to the countries themselves if they try to control their users’ requests, yet make mistakes.