GoBGP – ThingNetwork.io

This post is about how you can use BGP to augment your network edge protection. Specifically BGP Flow spec can be used to dynamically add entries to an access control list (or ip-filter in Nokia SROS speak). Previous posts have been using ExaBGP as the method to inject traffic, and while it’s good at what it does, I’ve moved on to use GoBGP not because it seems to be quite a high performance BGP implementation (I’m just doing proof of concept work) but because GoBGP’s command line interface is more friendly and better documented than ExaBGP, making it the more versatile tool to generate route advertisements in a lab on the fly (at least for me).

The lab topology is designed to demonstrate filtering with flowspec – two routers (SR1 is under my control, while the External router is a gateway to unscrupulous entities that I cannot control) and GoBGP running on a computer in my Network Operations centre.

Before getting into the flowspec piece, I’ll go through the installation process for GoBGP and the Go Language for Ubuntu 16.04 (as the routers I’m using as running on eve-ng) and the computer in the example is bridged to SR1) and then how to develop the initial GoBGP configuration.

The Go Programming language is available via the standard Ubuntu repository
[php]adam@m4600:~$ sudo apt-get install golang-go[/php]

GoBGP is pulled down as source code and compiled as part of the download process. An environmental variable needs to be set so Go knows where to pull down code, and where to build the binaries.
[php]adam@m4600:~$ export GOPATH=$HOME/go[/php]
Tell go to pull down and build gobgpd (this takes awhile at least on my machine and you don’t get an indication on what it’s doing)
[php]adam@m4600:~$ go get github.com/osrg/gobgp/gobgpd[/php]
The gobgp cli is a separate program that talks to gobgpd using the GoBGP API – downloading and building it is pretty quick in comparison to gobgpd:
[php]adam@m4600:~$ go get github.com/osrg/gobgp/gobgp[/php]
Once built, we shall copy the generated binaries to the an executable path
[php]adam@m4600:~$ sudo cp $GOPATH/bin/* /usr/local/sbin/[/php]
For those of us using the default bash shell, we can enable tab completion for the cli
[php]adam@m4600:~$ sudo cp $GOPATH/src/github.com/osrg/gobgp/tools/completion/*.bash /etc/bash_completion.d/
[/php]

Once GoBGP is installed, we can develop a configuration file that will be used to set up the GoBGP instance – in this case we’re defining the router itself and SR1 as a neighbor for the IPv4 FlowSpec address family:
A configuration file that will be used by the GoBGP Daemon enabling FlowSpec for IPv4 where we peer against SR1:[php]
adam@m4600:~$ cat gobgp-flowspec.conf
[global.config]
as = 64512
router-id = “1.2.3.4”

[[neighbors]]
[neighbors.config]
neighbor-address = “192.168.1.123”
peer-as = 64512
[[neighbors.afi-safis]]
[neighbors.afi-safis.config]
afi-safi-name = “ipv4-flowspec”[/php]
The GoBGP documentation is also a bit nicer than ExaBGP too – additional information on setting up a configuration (including different formats is described in the GoBGP getting started document)

Start the GoBGP Daemon and read our new config:
[php]adam@m4600:~$ sudo gobgpd -f gobgp-flowspec.conf
{“level”:”info”,”msg”:”gobgpd started”,”time”:”2017-05-18T22:57:22+10:00″}
{“Topic”:”Config”,”level”:”info”,”msg”:”Finished reading the config file”,”time”:”2017-05-18T22:57:22+10:00″}
{“level”:”info”,”msg”:”Peer 192.168.1.123 is added”,”time”:”2017-05-18T22:57:22+10:00″}
{“Topic”:”Peer”,”level”:”info”,”msg”:”Add a peer configuration for:192.168.1.123″,”time”:”2017-05-18T22:57:22+10:00″}
{“Key”:”192.168.1.123″,”State”:”BGP_FSM_OPENCONFIRM”,”Topic”:”Peer”,”level”:”info”,”msg”:”Peer Up”,”time”:”2017-05-18T22:57:40+10:00″}[/php]We can see shortly after gobgpd started, it has already established a peering session with SR1 (192.168.1.123)

Opening up a new shell, we can use the gobgp client to verify that:
[php]adam@m4600:~$ gobgp neighbor
Peer AS Up/Down State |#Received Accepted
192.168.1.123 64512 00:00:22 Establ | 0 0
adam@m4600:~$ gobgp neighbor 192.168.1.123
BGP neighbor is 192.168.1.123, remote AS 64512
BGP version 4, remote router ID 123.123.123.123
BGP state = established, up for 00:01:53
BGP OutQ = 0, Flops = 0
Hold time is 90, keepalive interval is 30 seconds
Configured hold time is 90, keepalive interval is 30 seconds
%s
Neighbor capabilities:
multiprotocol:
ipv4-flowspec: advertised and received
route-refresh: advertised and received
4-octet-as: advertised and received
Message statistics:
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 0 0
Keepalives: 4 5
Route Refresh: 0 0
Discarded: 0 0
Total: 5 6
Route statistics:
Advertised: 0
Received: 0
Accepted: 0[/php]
We can also get SR1 to confirm it too:
[php]*A:SR1# show router bgp summary | match Summ post-lines 100
BGP Summary
===============================================================================
Legend : D – Dynamic Neighbor
===============================================================================
Neighbor
Description
AS PktRcvd InQ Up/Down State|Rcv/Act/Sent (Addr Family)
PktSent OutQ
——————————————————————————-
192.168.1.46
64512 9 0 00h00m54s 0/0/0 (FlowIPv4)
13 0
——————————————————————————-[/php]

Now that the flowspec session is up and running but no one has exchanged any routes yet, I’ll park this for a moment to create a simple ip-filter (or ACL)[php linenumbers=”True” highlight=”5,13,14,22″]*A:SR1# /configure filter
*A:SR1>config>filter# info
———————————————-
match-list
ip-prefix-list “FPL_RFC1918” create
prefix 10.0.0.0/8
prefix 172.16.0.0/12
prefix 192.168.0.0/16
exit
exit
ip-filter 100 create
default-action forward
embed-filter flowspec router “Base” offset 1000
entry 10 create
match
src-ip ip-prefix-list “FPL_RFC1918”
exit
action
drop
exit
exit
entry 20 create
match
dst-ip ip-prefix-list “FPL_RFC1918”
exit
action
drop
exit
exit
exit
———————————————-[/php]

The match-list ip-prefix-list “FPL_RFC1918” is a way to consolidate a number of network prefixes into a single named list enabling filter configurations to be smaller and more manageable (and easier to debug)

ip-filter 100 is the thing of interest. The default-action is what happens if none of the previous entries are matched – in this case it is a black-list ip-filter, since anything we don’t specifically drop will get forwarded
Each filter entry has a number which is used for the ascending evaluation order (entry 10 before entry 20 here) and usually has a match entry and an action to perform when the match was successful. This filter is relatively simple since all it is doing is blocking IP packets that either have source or destination IP addresses from RFC1918 space.

The interesting thing is the embed-filter flowspec entry. Router “Base” is the base routing instance (as opposed to a VPRN) and offset 1000 means that any flowspec entries will be dynamically added to ip-filter 100 starting at entry 1000 (and with an additional offset determined by flowspec)

This filter will be applied to the IP interface facing the external router – This interface is associated with an Internet Enhanced Service (IES) associated with the SR1’s Global Routing Table (Router “Base”)
[php highlight=”21,22″]*A:SR1>config>service>ies# /configure service ies 1
*A:SR1>config>service>ies# info
———————————————-
interface “External” create
address 200.200.200.1/24
sap 1/1/1 create
exit
exit
no shutdown
———————————————-
*A:SR1>config>service>ies# show router route-table

===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
——————————————————————————-
1.1.1.1/32 Local Local 00h20m17s 0
Loop_1 0
0.0.0.0/0 Remote Static 00h20m02s 5
200.200.200.2 1
123.123.123.123/32 Local Local 00h20m17s 0
system 0
192.168.1.0/24 Local Local 00h20m02s 0
NetOps 0
200.200.200.0/24 Local Local 00h20m02s 0
External 0
——————————————————————————-
No. of Routes: 5
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available[/php]
In this contrived example SR1 just has a default route point to the External interface in IES 1.
Before we apply the IP filter – check to see we can reach an RFC1918 IP Address:
[php]A:External# ping 1.1.1.1 source 172.16.0.1 count 2
PING 1.1.1.1 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=1.40ms.
64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=1.91ms.

—- 1.1.1.1 PING Statistics —-
2 packets transmitted, 2 packets received, 0.00% packet loss
round-trip min = 1.40ms, avg = 1.66ms, max = 1.91ms, stddev = 0.253ms[/php]
Apply the IP Filter in the ingress direction of interface External on SR1:
[php]*A:SR1>config>service>ies# interface “External” sap 1/1/1 ingress filter ip 100
*A:SR1>config>service>ies# info
———————————————-
interface “External” create
address 200.200.200.1/24
sap 1/1/1 create
ingress
filter ip 100
exit
exit
exit
no shutdown
[/php]
And repeat the test:
[php]*A:External# ping 1.1.1.1 source 172.16.0.1 count 2
PING 1.1.1.1 56 data bytes
Request timed out. icmp_seq=1.
Request timed out. icmp_seq=2.

—- 1.1.1.1 PING Statistics —-
2 packets transmitted, 0 packets received, 100% packet loss[/php]
We can see the ping failed and if we look at the counters
[php highlight=”20″]*A:SR1>config>service>ies# show filter ip 100 counters

Entry : 20
Ing. Matches : 0 pkts
Egr. Matches : 0 pkts

===============================================================================[/php]We can see the ping was unsuccessful because of entry 10 stopping traffic coming back to SR1 (because we weren’t filtering traffic leaving SR1)

Now all of a sudden we find a ping flood (well okay, calling two packets a flood is a rather long bow to draw but anyway) is coming from 100.100.100.100 and hitting 1.1.1.1[php]A:External# ping 1.1.1.1 source 100.100.100.100 count 2
PING 1.1.1.1 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=1.16ms.
64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=1.26ms.[/php]
These ICMP packets are causing havok to 1.1.1.1, so it would be nice to add some precision filtering to stop that from occuring, while allowing other traffic through – this is where our GoBGP instance and flowspec comes back into the story. We will ask gobgp to announce a flowspec “route” that specifies a discard action (actually a rate-limit of 0 kbps) for traffic from 100.100.100.100 to 1.1.1.1 using ICMP:
[python]adam@m4600:~$ gobgp global rib add -a ipv4-flowspec match source 100.100.100.100/32 destination 1.1.1.1/32 protocol icmp then discard[/python]
SR1 can confirm it recieved this route:
[php]*A:SR1>config>service>ies# show router bgp routes flow-ipv4
===============================================================================
BGP Router ID:123.123.123.123 AS:64512 Local AS:64512
===============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete

===============================================================================
BGP FLOW IPV4 Routes
===============================================================================
Flag Network Nexthop LocalPref MED
As-Path
——————————————————————————-
u*>? — 0.0.0.0 100 None
No As-Path

Community Action: rate-limit: 0 kbps
NLRI Subcomponents:
Dest Pref : 1.1.1.1/32
Src Pref : 100.100.100.100/32
Ip Proto : [ == 1 ]
——————————————————————————-
Routes : 1
===============================================================================[/php]
Flowspec can do quite interesting things, besides rate-limiting traffic to 0kbps (dropping it), it’s possible to use flowspec to tell the router to rate limit flows rather than completely blocking, remark traffic to a different level of priority or even redirect traffic to a VRF by setting the appropriate route targets (this could for example allow the forwarding of traffic into a VPRN or VRF to reach a scrubbing network prior to re-injection of clean traffic back into the network)

Lets see if ip filter has taken this flowspec entry:[php highlight=”57,58″]*A:SR1>config>service>ies# show filter ip 100

===============================================================================
IP Filter
===============================================================================
Filter Id : 100 Applied : Yes
Scope : Template Def. Action : Forward
System filter : Unchained
Radius Ins Pt : n/a
CrCtl. Ins Pt : n/a
RadSh. Ins Pt : n/a
PccRl. Ins Pt : n/a
Entries : 2/0/0/1 (Fixed/Radius/Cc/Embedded)
Sub-Entries : 6/0/0/1
Description : (Not Specified)
——————————————————————————-
Filter Match Criteria : IP
——————————————————————————-
Entry : 10
Description : (Not Specified)
Log Id : n/a
Src. IP : ip-prefix-list “FPL_RFC1918”
Src. Port : n/a
Dest. IP : 0.0.0.0/0
Dest. Port : n/a
Protocol : Undefined Dscp : Undefined
ICMP Type : Undefined ICMP Code : Undefined
Fragment : Off Src Route Opt : Off
Sampling : Off Int. Sampling : On
IP-Option : 0/0 Multiple Option: Off
TCP-syn : Off TCP-ack : Off
Option-pres : Off
Egress PBR : Disabled
Primary Action : Drop
Ing. Matches : 2 pkts (204 bytes)
Egr. Matches : 0 pkts

Entry : 20
Description : (Not Specified)
Log Id : n/a
Src. IP : 0.0.0.0/0
Src. Port : n/a
Dest. IP : ip-prefix-list “FPL_RFC1918”
Dest. Port : n/a
Protocol : Undefined Dscp : Undefined
ICMP Type : Undefined ICMP Code : Undefined
Fragment : Off Src Route Opt : Off
Sampling : Off Int. Sampling : On
IP-Option : 0/0 Multiple Option: Off
TCP-syn : Off TCP-ack : Off
Option-pres : Off
Egress PBR : Disabled
Primary Action : Drop
Ing. Matches : 0 pkts
Egr. Matches : 0 pkts

Entry : 1256
Origin : Inserted by embedded filter fSpec-0 entry 256
Description : (Not Specified)
Log Id : n/a
Src. IP : 100.100.100.100/32
Src. Port : n/a
Dest. IP : 1.1.1.1/32
Dest. Port : n/a
Protocol : 1 Dscp : Undefined
ICMP Type : Undefined ICMP Code : Undefined
Fragment : Off Src Route Opt : Off
Sampling : Off Int. Sampling : On
IP-Option : 0/0 Multiple Option: Off
TCP-syn : Off TCP-ack : Off
Option-pres : Off
Egress PBR : Disabled
Primary Action : Drop
Ing. Matches : 0 pkts
Egr. Matches : 0 pkts

===============================================================================[/php]We had specified that the offset for flowspec was 1000, this flowspec entry (number 256) became ip filter 100 entry 256. Time to verify that the ping now should fail:
[php]*A:External# ping 1.1.1.1 source 100.100.100.100 count 2
PING 1.1.1.1 56 data bytes
Request timed out. icmp_seq=1.
Request timed out. icmp_seq=2.

—- 1.1.1.1 PING Statistics —-
2 packets transmitted, 0 packets received, 100% packet loss[/php]
Yes, and we can verify that it’s only blocking icmp between those endpoints by sending IP packets from another protocol say UDP by doing a traceroute:
[php]*A:External# traceroute 1.1.1.1 source 100.100.100.100
traceroute to 1.1.1.1 from 100.100.100.100, 30 hops max, 40 byte packets
1 1.1.1.1 (1.1.1.1) 1.74 ms 1.51 ms 1.52 ms[/php]Yes, this is pretty good as far as network based filters (that aren’t performing payload based inspection) are doing, there’s quite a few match conditions that can be used to narrow things down even further. In this contrived example it may be similar effort to configure a filter on the fly but in a production environment, you could hook GoBGP to your route reflector and through a single procedure push updates to all your edge routers in one go. Another good thing about this method, is if the filtering is only temporary, it’s just as easy to remove as it was to put in.
To see what’s currently in the rib for ipv4-flowspec:
[php]adam@m4600:~$ gobgp global rib -a ipv4-flowspec
Network Next Hop AS_PATH Age Attrs
*> [destination:1.1.1.1/32][source:100.100.100.100/32][protocol:==icmp ]fictitious 00:12:06 [{Origin: ?} {Extcomms: [discard]}][/php]Withdraw the announcement:
[php]adam@m4600:~$ gobgp global rib del -a ipv4-flowspec match source 100.100.100.100/32 destination 1.1.1.1/32 protocol icmp then discard
adam@m4600:~$ gobgp global rib -a ipv4-flowspec Network not in table[/php]
Verify SR1 ip filter 100 no longer has the entry:
[php]*A:SR1>config>service>ies# show filter ip 100

===============================================================================
IP Filter
===============================================================================
Filter Id : 100 Applied : Yes
Scope : Template Def. Action : Forward
System filter : Unchained
Radius Ins Pt : n/a
CrCtl. Ins Pt : n/a
RadSh. Ins Pt : n/a
PccRl. Ins Pt : n/a
Entries : 2
Sub-Entries : 6
Description : (Not Specified)
——————————————————————————-
Filter Match Criteria : IP
——————————————————————————-
Entry : 10
Description : (Not Specified)
Log Id : n/a
Src. IP : ip-prefix-list “FPL_RFC1918”
Src. Port : n/a
Dest. IP : 0.0.0.0/0
Dest. Port : n/a
Protocol : Undefined Dscp : Undefined
ICMP Type : Undefined ICMP Code : Undefined
Fragment : Off Src Route Opt : Off
Sampling : Off Int. Sampling : On
IP-Option : 0/0 Multiple Option: Off
TCP-syn : Off TCP-ack : Off
Option-pres : Off
Egress PBR : Disabled
Primary Action : Drop
Ing. Matches : 2 pkts (204 bytes)
Egr. Matches : 0 pkts

===============================================================================[/php]Entry 1256 has left the building.

Hopefully that was a reasonable introduction to using BGP Flowspec with filtering and GoBGP.

Flowspec is quite often a piece of the puzzle in DDoS mitigation tools, usually there is some kind of automation attached which makes decisions using inputs such a Netflow/IPFIX data, SNMP polling of interfaces to track anomalous loading and in some cases threat intelligence feeds from security vendors.

Category: GoBGP

Using BGP Flowspec with Nokia SROS to protect the network edge