“No Filters” or An Easy Way to Shoot In the Foot

Several times in our posts we discussed consequences of lack of ingress filtering. Such mistake configuration can work fine most of the time, but one day may result in an outage at regional or even global scale. And yesterday, 25.11.2018, it happened again, this time in Russia.

At 13:00 UTC a small Russian network Krek Ltd (AS57494) started leaking prefixes between its providers, thus sending routes learned from three regional providers to national career Rostelecom (AS12389) - the biggest IP-transit and broadband provider in Russia. The incident affected more than 40 thousand prefixes, that belong to networks all over the world, including Russian, American, Australian, and many others, thus disrupting connectivity for the 10-20% of Russian users. It also hit popular services such as Amazon, Youtube, and Russian social network Vkontakte making them partially unreachable.

For more than 5000 prefixes this incident propagated beyond Russian Internet segment, pulling traffic from other regions to this black hole.

This route leak was a result of two interconnected mistakes - the Krek network that broke egress BGP filters and Rostelecom that failed to configure ingress filters. Both networks have paid price for this mistakes, but it can only partially comfort other players that lost a significant fraction of their users.

Breaking Leaks

It’s common to think that a network has no way to break a leak if it has happened. You can’t configure proper filters at the router that is out of your control. Of course, you can start writing emails to responsible parties, and this may speed up the remediation process, but it will not instantly return your traffic.

But there is another way to restore proper connectivity, we can exploit the BGP loop detection mechanism: a network MUST automatically drop route if AS Path includes its own AS number. So, if you know the source of the problem, you can add “leaker” AS in your prepend policy, thus forcing it to reject this route.

Such AS Path manipulation complements monitoring service and provides a guaranteed method to actively fight route leak incidents. But you should be careful to keep 'valid' result of ROA validation procedure.