This post is about how you can use BGP to augment your network edge protection. Specifically BGP Flow spec can be used to dynamically add entries to an access control list (or ip-filter in Nokia SROS speak). Previous posts have been using ExaBGP as the method to inject traffic, and while it’s good at what it does, I’ve moved on to use GoBGP not because it seems to be quite a high performance BGP implementation (I’m just doing proof of concept work) but because GoBGP’s command line interface is more friendly and better documented than ExaBGP, making it the more versatile tool to generate route advertisements in a lab on the fly (at least for me).
The lab topology is designed to demonstrate filtering with flowspec – two routers (SR1 is under my control, while the External router is a gateway to unscrupulous entities that I cannot control) and GoBGP running on a computer in my Network Operations centre.
Before getting into the flowspec piece, I’ll go through the installation process for GoBGP and the Go Language for Ubuntu 16.04 (as the routers I’m using as running on eve-ng) and the computer in the example is bridged to SR1) and then how to develop the initial GoBGP configuration.
The Go Programming language is available via the standard Ubuntu repository
adam@m4600:~$ sudo apt-get install golang-go
GoBGP is pulled down as source code and compiled as part of the download process. An environmental variable needs to be set so Go knows where to pull down code, and where to build the binaries.
adam@m4600:~$ export GOPATH=$HOME/go
Tell go to pull down and build gobgpd (this takes awhile at least on my machine and you don’t get an indication on what it’s doing)
adam@m4600:~$ go get github.com/osrg/gobgp/gobgpd
The gobgp cli is a separate program that talks to gobgpd using the GoBGP API – downloading and building it is pretty quick in comparison to gobgpd:
adam@m4600:~$ go get github.com/osrg/gobgp/gobgp
Once built, we shall copy the generated binaries to the an executable path
adam@m4600:~$ sudo cp $GOPATH/bin/* /usr/local/sbin/
For those of us using the default bash shell, we can enable tab completion for the cli
adam@m4600:~$ sudo cp $GOPATH/src/github.com/osrg/gobgp/tools/completion/*.bash /etc/bash_completion.d/
Once GoBGP is installed, we can develop a configuration file that will be used to set up the GoBGP instance – in this case we’re defining the router itself and SR1 as a neighbor for the IPv4 FlowSpec address family:
A configuration file that will be used by the GoBGP Daemon enabling FlowSpec for IPv4 where we peer against SR1:
adam@m4600:~$ cat gobgp-flowspec.conf [global.config] as = 64512 router-id = "1.2.3.4" [[neighbors]] [neighbors.config] neighbor-address = "192.168.1.123" peer-as = 64512 [[neighbors.afi-safis]] [neighbors.afi-safis.config] afi-safi-name = "ipv4-flowspec"
The GoBGP documentation is also a bit nicer than ExaBGP too – additional information on setting up a configuration (including different formats is described in the GoBGP getting started document)
Start the GoBGP Daemon and read our new config:
adam@m4600:~$ sudo gobgpd -f gobgp-flowspec.conf
{"level":"info","msg":"gobgpd started","time":"2017-05-18T22:57:22+10:00"}
{"Topic":"Config","level":"info","msg":"Finished reading the config file","time":"2017-05-18T22:57:22+10:00"}
{"level":"info","msg":"Peer 192.168.1.123 is added","time":"2017-05-18T22:57:22+10:00"}
{"Topic":"Peer","level":"info","msg":"Add a peer configuration for:192.168.1.123","time":"2017-05-18T22:57:22+10:00"}
{"Key":"192.168.1.123","State":"BGP_FSM_OPENCONFIRM","Topic":"Peer","level":"info","msg":"Peer Up","time":"2017-05-18T22:57:40+10:00"}
We can see shortly after gobgpd started, it has already established a peering session with SR1 (192.168.1.123)
Opening up a new shell, we can use the gobgp client to verify that:
adam@m4600:~$ gobgp neighbor Peer AS Up/Down State |#Received Accepted 192.168.1.123 64512 00:00:22 Establ | 0 0 adam@m4600:~$ gobgp neighbor 192.168.1.123 BGP neighbor is 192.168.1.123, remote AS 64512 BGP version 4, remote router ID 123.123.123.123 BGP state = established, up for 00:01:53 BGP OutQ = 0, Flops = 0 Hold time is 90, keepalive interval is 30 seconds Configured hold time is 90, keepalive interval is 30 seconds %s Neighbor capabilities: multiprotocol: ipv4-flowspec: advertised and received route-refresh: advertised and received 4-octet-as: advertised and received Message statistics: Sent Rcvd Opens: 1 1 Notifications: 0 0 Updates: 0 0 Keepalives: 4 5 Route Refresh: 0 0 Discarded: 0 0 Total: 5 6 Route statistics: Advertised: 0 Received: 0 Accepted: 0
We can also get SR1 to confirm it too:
*A:SR1# show router bgp summary | match Summ post-lines 100
BGP Summary
===============================================================================
Legend : D - Dynamic Neighbor
===============================================================================
Neighbor
Description
AS PktRcvd InQ Up/Down State|Rcv/Act/Sent (Addr Family)
PktSent OutQ
-------------------------------------------------------------------------------
192.168.1.46
64512 9 0 00h00m54s 0/0/0 (FlowIPv4)
13 0
-------------------------------------------------------------------------------
Now that the flowspec session is up and running but no one has exchanged any routes yet, I’ll park this for a moment to create a simple ip-filter (or ACL)
*A:SR1# /configure filter
*A:SR1>config>filter# info
----------------------------------------------
match-list
ip-prefix-list "FPL_RFC1918" create
prefix 10.0.0.0/8
prefix 172.16.0.0/12
prefix 192.168.0.0/16
exit
exit
ip-filter 100 create
default-action forward
embed-filter flowspec router "Base" offset 1000
entry 10 create
match
src-ip ip-prefix-list "FPL_RFC1918"
exit
action
drop
exit
exit
entry 20 create
match
dst-ip ip-prefix-list "FPL_RFC1918"
exit
action
drop
exit
exit
exit
----------------------------------------------
The match-list ip-prefix-list “FPL_RFC1918” is a way to consolidate a number of network prefixes into a single named list enabling filter configurations to be smaller and more manageable (and easier to debug)
ip-filter 100 is the thing of interest. The default-action is what happens if none of the previous entries are matched – in this case it is a black-list ip-filter, since anything we don’t specifically drop will get forwarded
Each filter entry has a number which is used for the ascending evaluation order (entry 10 before entry 20 here) and usually has a match entry and an action to perform when the match was successful. This filter is relatively simple since all it is doing is blocking IP packets that either have source or destination IP addresses from RFC1918 space.
The interesting thing is the embed-filter flowspec entry. Router “Base” is the base routing instance (as opposed to a VPRN) and offset 1000 means that any flowspec entries will be dynamically added to ip-filter 100 starting at entry 1000 (and with an additional offset determined by flowspec)
This filter will be applied to the IP interface facing the external router – This interface is associated with an Internet Enhanced Service (IES) associated with the SR1’s Global Routing Table (Router “Base”)
*A:SR1>config>service>ies# /configure service ies 1 *A:SR1>config>service>ies# info ---------------------------------------------- interface "External" create address 200.200.200.1/24 sap 1/1/1 create exit exit no shutdown ---------------------------------------------- *A:SR1>config>service>ies# show router route-table =============================================================================== Route Table (Router: Base) =============================================================================== Dest Prefix[Flags] Type Proto Age Pref Next Hop[Interface Name] Metric ------------------------------------------------------------------------------- 1.1.1.1/32 Local Local 00h20m17s 0 Loop_1 0 0.0.0.0/0 Remote Static 00h20m02s 5 200.200.200.2 1 123.123.123.123/32 Local Local 00h20m17s 0 system 0 192.168.1.0/24 Local Local 00h20m02s 0 NetOps 0 200.200.200.0/24 Local Local 00h20m02s 0 External 0 ------------------------------------------------------------------------------- No. of Routes: 5 Flags: n = Number of times nexthop is repeated B = BGP backup route available L = LFA nexthop available
In this contrived example SR1 just has a default route point to the External interface in IES 1.
Before we apply the IP filter – check to see we can reach an RFC1918 IP Address:
A:External# ping 1.1.1.1 source 172.16.0.1 count 2 PING 1.1.1.1 56 data bytes 64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=1.40ms. 64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=1.91ms. ---- 1.1.1.1 PING Statistics ---- 2 packets transmitted, 2 packets received, 0.00% packet loss round-trip min = 1.40ms, avg = 1.66ms, max = 1.91ms, stddev = 0.253ms
Apply the IP Filter in the ingress direction of interface External on SR1:
*A:SR1>config>service>ies# interface "External" sap 1/1/1 ingress filter ip 100
*A:SR1>config>service>ies# info
----------------------------------------------
interface "External" create
address 200.200.200.1/24
sap 1/1/1 create
ingress
filter ip 100
exit
exit
exit
no shutdown
And repeat the test:
A:External# ping 1.1.1.1 PING 1.1.1.1 56 data bytes Request timed out. icmp_seq=1. Request timed out. icmp_seq=2. ---- 1.1.1.1 PING Statistics ---- 2 packets transmitted, 0 packets received, 100% packet loss
We can see the ping failed and if we look at the counters
*A:SR1>config>service>ies# show filter ip 100 counters =============================================================================== IP Filter =============================================================================== Filter Id : 100 Applied : Yes Scope : Template Def. Action : Forward System filter : Unchained Radius Ins Pt : n/a CrCtl. Ins Pt : n/a RadSh. Ins Pt : n/a PccRl. Ins Pt : n/a Entries : 2 Sub-Entries : 6 Description : (Not Specified) ------------------------------------------------------------------------------- Filter Match Criteria : IP ------------------------------------------------------------------------------- Entry : 10 Ing. Matches : 2 pkts (204 bytes) Egr. Matches : 0 pkts Entry : 20 Ing. Matches : 0 pkts Egr. Matches : 0 pkts ===============================================================================
We can see the ping was unsuccessful because of entry 10 stopping traffic coming back to SR1 (because we weren’t filtering traffic leaving SR1)
Now all of a sudden we find a ping flood (well okay, calling two packets a flood is a rather long bow to draw but anyway) is coming from 100.100.100.100 and hitting 1.1.1.1
A:External# ping 1.1.1.1 source 100.100.100.100 count 2 PING 1.1.1.1 56 data bytes 64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=1.16ms. 64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=1.26ms.
Supposedly these ICMP packets are causing havok to 1.1.1.1, so it would be nice to add some precision filtering to stop that from concurring, while allowing other traffic through – this is where our GoBGP instance and flowspec comes back into the story. We will ask gobgp to announce a flowspec “route” that specifies a discard action (actually a rate-limit of 0 kbps) for traffic from 100.100.100.100 to 1.1.1.1 using ICMP:
adam@m4600:~$ gobgp global rib add -a ipv4-flowspec match source 100.100.100.100/32 destination 1.1.1.1/32 protocol icmp then discard
SR1 can confirm it recieved this route:
*A:SR1>config>service>ies# show router bgp routes flow-ipv4
===============================================================================
BGP Router ID:123.123.123.123 AS:64512 Local AS:64512
===============================================================================
Legend -
Status codes : u - used, s - suppressed, h - history, d - decayed, * - valid
l - leaked, x - stale, > - best, b - backup, p - purge
Origin codes : i - IGP, e - EGP, ? - incomplete
===============================================================================
BGP FLOW IPV4 Routes
===============================================================================
Flag Network Nexthop LocalPref MED
As-Path
-------------------------------------------------------------------------------
u*>? -- 0.0.0.0 100 None
No As-Path
Community Action: rate-limit: 0 kbps
NLRI Subcomponents:
Dest Pref : 1.1.1.1/32
Src Pref : 100.100.100.100/32
Ip Proto : [ == 1 ]
-------------------------------------------------------------------------------
Routes : 1
===============================================================================
Flowspec can do quite interesting things, besides rate-limiting traffic to 0kbps (dropping it), it’s possible to use flowspec to tell the router to rate limit flows rather than completely blocking, remark traffic to a different level of priority or even redirect traffic to a VRF by setting the appropriate route targets (this could for example allow the forwarding of traffic into a VPRN or VRF to reach a scrubbing network prior to re-injection of clean traffic back into the network)
Lets see if ip filter has taken this flowspec entry:
*A:SR1>config>service>ies# show filter ip 100 =============================================================================== IP Filter =============================================================================== Filter Id : 100 Applied : Yes Scope : Template Def. Action : Forward System filter : Unchained Radius Ins Pt : n/a CrCtl. Ins Pt : n/a RadSh. Ins Pt : n/a PccRl. Ins Pt : n/a Entries : 2/0/0/1 (Fixed/Radius/Cc/Embedded) Sub-Entries : 6/0/0/1 Description : (Not Specified) ------------------------------------------------------------------------------- Filter Match Criteria : IP ------------------------------------------------------------------------------- Entry : 10 Description : (Not Specified) Log Id : n/a Src. IP : ip-prefix-list "FPL_RFC1918" Src. Port : n/a Dest. IP : 0.0.0.0/0 Dest. Port : n/a Protocol : Undefined Dscp : Undefined ICMP Type : Undefined ICMP Code : Undefined Fragment : Off Src Route Opt : Off Sampling : Off Int. Sampling : On IP-Option : 0/0 Multiple Option: Off TCP-syn : Off TCP-ack : Off Option-pres : Off Egress PBR : Disabled Primary Action : Drop Ing. Matches : 2 pkts (204 bytes) Egr. Matches : 0 pkts Entry : 20 Description : (Not Specified) Log Id : n/a Src. IP : 0.0.0.0/0 Src. Port : n/a Dest. IP : ip-prefix-list "FPL_RFC1918" Dest. Port : n/a Protocol : Undefined Dscp : Undefined ICMP Type : Undefined ICMP Code : Undefined Fragment : Off Src Route Opt : Off Sampling : Off Int. Sampling : On IP-Option : 0/0 Multiple Option: Off TCP-syn : Off TCP-ack : Off Option-pres : Off Egress PBR : Disabled Primary Action : Drop Ing. Matches : 0 pkts Egr. Matches : 0 pkts Entry : 1256 Origin : Inserted by embedded filter fSpec-0 entry 256 Description : (Not Specified) Log Id : n/a Src. IP : 100.100.100.100/32 Src. Port : n/a Dest. IP : 1.1.1.1/32 Dest. Port : n/a Protocol : 1 Dscp : Undefined ICMP Type : Undefined ICMP Code : Undefined Fragment : Off Src Route Opt : Off Sampling : Off Int. Sampling : On IP-Option : 0/0 Multiple Option: Off TCP-syn : Off TCP-ack : Off Option-pres : Off Egress PBR : Disabled Primary Action : Drop Ing. Matches : 0 pkts Egr. Matches : 0 pkts ===============================================================================
We had specified that the offset for flowspec was 1000, this flowspec entry (number 256) became ip filter 100 entry 256. Time to verify that the ping now should fail:
*A:External# ping 1.1.1.1 source 100.100.100.100 count 2 PING 1.1.1.1 56 data bytes Request timed out. icmp_seq=1. Request timed out. icmp_seq=2. ---- 1.1.1.1 PING Statistics ---- 2 packets transmitted, 0 packets received, 100% packet loss
Yes, and we can verify that it’s only blocking icmp between those endpoints by sending IP packets from another protocol say UDP by doing a traceroute:
*A:External# traceroute 1.1.1.1 source 100.100.100.100 traceroute to 1.1.1.1 from 100.100.100.100, 30 hops max, 40 byte packets 1 1.1.1.1 (1.1.1.1) 1.74 ms 1.51 ms 1.52 ms
Yes, this is pretty good as far as network based filters (that aren’t performing payload based inspection) are doing, there’s quite a few match conditions that can be used to narrow things down even further. In this contrived example it may be similar effort to configure a filter on the fly but in a production environment, you could hook GoBGP to your route reflector and through a single procedure push updates to all your edge routers in one go. Another good thing about this method, is if the filtering is only temporary, it’s just as easy to remove as it was to put in.
To see what’s currently in the rib for ipv4-flowspec:
adam@m4600:~$ gobgp global rib -a ipv4-flowspec
Network Next Hop AS_PATH Age Attrs
*> [destination:1.1.1.1/32][source:100.100.100.100/32][protocol:==icmp ]fictitious 00:12:06 [{Origin: ?} {Extcomms: [discard]}]
Withdraw the announcement:
adam@m4600:~$ gobgp global rib del -a ipv4-flowspec match source 100.100.100.100/32 destination 1.1.1.1/32 protocol icmp then discard adam@m4600:~$ gobgp global rib -a ipv4-flowspec Network not in table
Verify SR1 ip filter 100 no longer has the entry:
*A:SR1>config>service>ies# show filter ip 100 =============================================================================== IP Filter =============================================================================== Filter Id : 100 Applied : Yes Scope : Template Def. Action : Forward System filter : Unchained Radius Ins Pt : n/a CrCtl. Ins Pt : n/a RadSh. Ins Pt : n/a PccRl. Ins Pt : n/a Entries : 2 Sub-Entries : 6 Description : (Not Specified) ------------------------------------------------------------------------------- Filter Match Criteria : IP ------------------------------------------------------------------------------- Entry : 10 Description : (Not Specified) Log Id : n/a Src. IP : ip-prefix-list "FPL_RFC1918" Src. Port : n/a Dest. IP : 0.0.0.0/0 Dest. Port : n/a Protocol : Undefined Dscp : Undefined ICMP Type : Undefined ICMP Code : Undefined Fragment : Off Src Route Opt : Off Sampling : Off Int. Sampling : On IP-Option : 0/0 Multiple Option: Off TCP-syn : Off TCP-ack : Off Option-pres : Off Egress PBR : Disabled Primary Action : Drop Ing. Matches : 2 pkts (204 bytes) Egr. Matches : 0 pkts Entry : 20 Description : (Not Specified) Log Id : n/a Src. IP : 0.0.0.0/0 Src. Port : n/a Dest. IP : ip-prefix-list "FPL_RFC1918" Dest. Port : n/a Protocol : Undefined Dscp : Undefined ICMP Type : Undefined ICMP Code : Undefined Fragment : Off Src Route Opt : Off Sampling : Off Int. Sampling : On IP-Option : 0/0 Multiple Option: Off TCP-syn : Off TCP-ack : Off Option-pres : Off Egress PBR : Disabled Primary Action : Drop Ing. Matches : 0 pkts Egr. Matches : 0 pkts ===============================================================================
Entry 1256 has left the building.
Hopefully that was a reasonable introduction to using BGP Flowspec with filtering and GoBGP.
Flowspec is quite often a piece of the puzzle in DDoS mitigation tools, usually there is some kind of automation attached which makes decisions using inputs such a Netflow/IPFIX data, SNMP polling of interfaces to track anomalous loading and in some cases threat intelligence feeds from security vendors.
Recent Comments