Bird Unreachable Routes on multihop BGP Sessions

In the past few months, quite a few people have asked me this question: “I bought a VPS from provider X, turned up a BGP session with them and my routes are all marked unreachable”. Today’s post will hopefully shed some light on what causes this phenomenon and how to deal with it.

The Problem

You’re setting up a shiny new BGP router at a cloud provider (like VMHaus or Vultr)

protocol bgp Provider {
        local as 204543;
        source address 2402:28c0:100:fa10::1;

        import filter {
                accept;
        };
        export filter {
                if source = RTS_STATIC then { # Only announce our routes
                        accept;
                }
                reject;
        };
        export limit 50; # Just in case
        graceful restart on;
        multihop 3;
        neighbor 2402:28c0:3::179 as 136620;
}

getting ready to accept the full table and propagate it to the kernel and … uh oh. You see the following sight:

unreachable 2c0f:f140:c000::/36 dev lo proto bird metric 1024 error -113 pref medium
unreachable 2c0f:f140:d000::/36 dev lo proto bird metric 1024 error -113 pref medium
unreachable 2c0f:f5f0:c000::/36 dev lo proto bird metric 1024 error -113 pref medium
unreachable 2c0f:f5f0:d000::/36 dev lo proto bird metric 1024 error -113 pref medium
unreachable 2c0f:f7b8:c000::/36 dev lo proto bird metric 1024 error -113 pref medium
unreachable 2c0f:f7b8:d000::/36 dev lo proto bird metric 1024 error -113 pref medium
unreachable 2c0f:f850:dc01::/48 dev lo proto bird metric 1024 error -113 pref medium
unreachable 2c0f:f850:dc03::/48 dev lo proto bird metric 1024 error -113 pref medium
unreachable 2c0f:f850:dc05::/48 dev lo proto bird metric 1024 error -113 pref medium
unreachable 2c0f:fb20:d000::/36 dev lo proto bird metric 1024 error -113 pref medium

All Bird supplied routes in the routing table are marked as “unreachable”. But why’s that?

 

Explanation

Bird exporting unreachable routes to the kernel is a sign that Bird can’t reach the next-hop of these routes. But wait, how is that possible, when they are supplied using BGP by our provider, who surely would not feed us bad routes?

The answer is actually pretty simple. This problem occurs as the BGP session we have with the provider is “multihop” – in other words, there are multiple network hops between the BGP router and our VM. Bird can only route to the first one, and not the other ones.

 

Solution

As Bird can’t automagically resolve past the first hop, we have to do it manually.

The first thing we need to do is to see which hops are in the way. The way we do this is by doing a traceroute to our BGP neighbour:

# traceroute6 -n 2402:28c0:3::179
traceroute to 2402:28c0:3::179 (2402:28c0:3::179), 30 hops max, 80 byte packets
 1  2402:28c0:100:fe00::1  0.374 ms  0.275 ms  0.200 ms
 2  2402:28c0:100:ffff:ffff:ffff:ffff:fffd  0.280 ms  0.460 ms  0.420 ms
 3  2402:28c0:3::179  0.375 ms  0.322 ms  0.335 ms

2402:28c0:100:fe00::1 is the server’s default gateway, so we needn’t worry about it. The other two hops have to be addressed though.

One of the easiest ways is to just manually define these routes in Bird:

protocol static noAnnounce {
        import all;

        route 2402:28c0:100:ffff:ffff:ffff:ffff:fffd/128 via 2402:28c0:100:fe00::1;
        route 2402:28c0:3::179/128 via 2402:28c0:100:fe00::1;
}

We’re now telling Bird that these hops are reachable via 2402:28c0:100:fe00::1, the server’s default gateway.

 

Last but not least, we need to make sure we don’t export these routes to the outside world, as that could cause some problems. We can achieve this by modifying the BGP session configuration like so:

protocol bgp Provider {
    local as 204543;
    source address 2402:28c0:100:fa10::1;

    import filter {
        accept;
    };
    export filter {
        if proto = "noAnnounce" then reject; # Do not announce local routes
        if source = RTS_STATIC then { # Only announce our routes
            accept;
        }
        reject;
    };
    export limit 50; # Just in case
    graceful restart on;
    multihop 3;
    neighbor 2402:28c0:3::179 as 136620;
}

That is hopefully all that needs to be done.

 

Conclusion

With a simple birdc6 configure, everything should be working correctly now.

2c0f:ff98::/32 via 2402:28c0:100:fe00::1 dev eth0 proto bird src 2a0d:1a40:6300::1 metric 1024 pref medium
2c0f:ffa0::/32 via 2402:28c0:100:fe00::1 dev eth0 proto bird src 2a0d:1a40:6300::1 metric 1024 pref medium
2c0f:ffa8::/32 via 2402:28c0:100:fe00::1 dev eth0 proto bird src 2a0d:1a40:6300::1 metric 1024 pref medium
2c0f:ffb8::/33 via 2402:28c0:100:fe00::1 dev eth0 proto bird src 2a0d:1a40:6300::1 metric 1024 pref medium
2c0f:ffc8::/32 via 2402:28c0:100:fe00::1 dev eth0 proto bird src 2a0d:1a40:6300::1 metric 1024 pref medium
2c0f:ffd0::/32 via 2402:28c0:100:fe00::1 dev eth0 proto bird src 2a0d:1a40:6300::1 metric 1024 pref medium
2c0f:ffd8::/32 via 2402:28c0:100:fe00::1 dev eth0 proto bird src 2a0d:1a40:6300::1 metric 1024 pref medium
2c0f:fff0::/32 via 2402:28c0:100:fe00::1 dev eth0 proto bird src 2a0d:1a40:6300::1 metric 1024 pref medium

Leave a Reply...