BGP – ThingNetwork.io

Carrier Supporting Carrier with SROS

Carrier of Carriers or Carrier supporting Carrier is a hierarchical construct to allow a network provider to provide MPLS connectivity with relatively low complexity. While it is even simpler to build layer 2 circuits over a backhaul network, in some cases routed connectivity may be preferred as you have a common connection point to the far end regions without needing to concern yourself with the provisioning scale and bandwidth management of each path. While this is quite a large posting it is mainly to show the configuration and verification steps. The summary of actions to create a CsC configuration is at the end of the post if you run out of patience.

This Example will have a three router Carrier of Carriers Network Operator (SC-1, SC-2 and SC-3) providing connectivity between a customer (VPRN 200) with two regions – Region A (RA-1, RA-2 and RA-3) and Region B (RB-1, RB-2 and RB-3)

9 Router Carrier Supporting Carrier Topology

Prior to SROS version 14.0R4, a single RIB was used for both labeled and unlabeled prefixes. With 14.0R4, two RIBs were established (actually, two RIBs for IPv4 and another two for IPv6) – this demonstration will have half the routers using 12.0R6 and half running 14.0R8 to highlight the configuration differences. The routers that are running SROS 14.0R8 in this example are configured using the VSR-D model which has the Control Plane Module operating in a VM (VM-CP) independent of the Input/Output Module (VM-DP) each VM-CP and VM-DP is logically connected via Switch Fabric ports – this is transparent to the configuration and operation of the simulated routers.

The initial configurations for Region A without the CSC Uplink on RA-1 as as below. OSPF as the IGP, LDP signalled LSPs are used, RA-3 is the local VPN Route Reflector for AS65000, VPRN 123 is created on each router.
[codegroup]
[php tab=”RA-1 Initial”]configure
system
name “RA-1”
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m5-1gb-sfp-b
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
port 1/1/2
ethernet
exit
no shutdown
exit
router
interface “RA-2”
address 1.1.2.1/29
port 1/1/1
no shutdown
exit
interface “RA-3”
address 1.1.3.1/29
port 1/1/2
no shutdown
exit
interface “system”
address 1.1.1.1/32
no shutdown
exit
autonomous-system 65000
ospf
area 0.0.0.0
interface “system”
no shutdown
exit
interface “RA-2”
no shutdown
exit
interface “RA-3”
no shutdown
exit
exit
exit
ldp
interface-parameters
interface “RA-2”
exit
interface “RA-3”
exit
exit
no shutdown
exit
bgp
split-horizon
group “VPN-RR”
family vpn-ipv4
peer-as 65000
neighbor 1.1.1.3
description “RA-3”
exit
exit
no shutdown
exit
exit
service
vprn 123 customer 1 create
route-distinguisher 1.1.1.1:123
auto-bind ldp
vrf-target target:65000:123
interface “Loop” create
address 123.1.1.1/32
loopback
exit
no shutdown
exit
exit
exit all[/php]
[php tab=”RA-2 Initial”]configure
system
name “RA-2”
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m5-1gb-sfp-b
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
port 1/1/2
ethernet
exit
no shutdown
exit
router
interface “RA-1”
address 1.1.2.2/29
port 1/1/1
no shutdown
exit
interface “RA-3”
address 1.2.3.2/29
port 1/1/2
no shutdown
exit
interface “system”
address 1.1.1.2/32
no shutdown
exit
autonomous-system 65000
ospf
area 0.0.0.0
interface “system”
no shutdown
exit
interface “RA-1”
no shutdown
exit
interface “RA-3”
no shutdown
exit
exit
exit
ldp
interface-parameters
interface “RA-1”
exit
interface “RA-3”
exit
exit
no shutdown
exit
bgp
group “VPN-RR”
family vpn-ipv4
peer-as 65000
neighbor 1.1.1.3
description “RA-3”
exit
exit
no shutdown
exit
exit
service
vprn 123 customer 1 create
route-distinguisher 1.1.1.2:123
auto-bind ldp
vrf-target target:65000:123
interface “Loop” create
address 123.1.1.2/32
loopback
exit
no shutdown
exit
exit
exit all[/php]
[php tab=”RA-3 (RR) Initial”]configure
system
name “RA-3”
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m5-1gb-sfp-b
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
port 1/1/2
ethernet
exit
no shutdown
exit
router
interface “RA-1”
address 1.1.3.3/29
port 1/1/2
no shutdown
exit
interface “RA-2”
address 1.2.3.3/29
port 1/1/1
no shutdown
exit
interface “system”
address 1.1.1.3/32
no shutdown
exit
autonomous-system 65000
ospf
area 0.0.0.0
interface “system”
no shutdown
exit
interface “RA-1”
no shutdown
exit
interface “RA-2”
no shutdown
exit
exit
exit
ldp
interface-parameters
interface “RA-1”
exit
interface “RA-2”
exit
exit
exit
bgp
group “VPN-RRC”
family vpn-ipv4
cluster 1.1.1.3
peer-as 65000
neighbor 1.1.1.1
description “RA-1”
exit
neighbor 1.1.1.2
description “RA-2”
exit
exit
exit
exit
service
vprn 123 customer 1 create
route-distinguisher 1.1.1.3:123
auto-bind ldp
vrf-target target:65000:123
interface “Loop” create
address 123.1.1.3/32
loopback
exit
no shutdown
exit
exit
exit all[/php]
[/codegroup]
Similarly for Region B, the initial configurations without the CSC Uplink on RB-1 as as below. OSPF as the IGP, LDP signalled LSPs, RB-3 is the local VPN Route Reflector for AS65000, VPRN 123 is created on each router. Besides the card/mda differences on the emulated hardware, the primary difference here compared to Region A which is using an older SROS version is the LSP binding (auto-bind) syntax changes to give more control as to the LSP types that may be desired.
[codegroup]
[php tab=”RB-1 Initial”]configure
system
name “RB-1”
exit
sfm 1
sfm-type m-sfm5-12
no shutdown
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m10-1gb-xp-sfp
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
port 1/1/2
ethernet
exit
no shutdown
exit
router
interface “RB-2”
address 2.1.2.1/29
port 1/1/1
no shutdown
exit
interface “RB-3”
address 2.1.3.1/29
port 1/1/2
no shutdown
exit
interface “system”
address 2.2.2.1/32
no shutdown
exit
autonomous-system 65000
ospf 0
area 0.0.0.0
interface “system”
no shutdown
exit
interface “RB-2”
no shutdown
exit
interface “RB-3”
no shutdown
exit
exit
no shutdown
exit
ldp
interface-parameters
interface “RB-2” dual-stack
ipv4
fec-type-capability
prefix-ipv6 disable
p2mp-ipv6 disable
exit
no shutdown
exit
no shutdown
exit
interface “RB-3” dual-stack
ipv4
fec-type-capability
prefix-ipv6 disable
p2mp-ipv6 disable
exit
no shutdown
exit
no shutdown
exit
exit
no shutdown
exit
bgp
split-horizon
group “VPN-RR”
family vpn-ipv4
peer-as 65000
neighbor 2.2.2.3
description “RB-3”
exit
exit
no shutdown
exit
exit
service
vprn 123 customer 1 create
route-distinguisher 2.2.2.1:123
auto-bind-tunnel
resolution-filter
ldp
exit
resolution filter
exit
vrf-target target:65000:123
interface “Loop” create
address 123.2.2.1/32
loopback
exit
no shutdown
exit
exit
exit all[/php]
[php tab=”RB-2 Initial”]configure
system
name “RB-2”
exit
sfm 1
sfm-type m-sfm5-12
no shutdown
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m10-1gb-xp-sfp
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
port 1/1/2
ethernet
exit
no shutdown
exit
router
interface “RB-1”
address 2.1.2.2/29
port 1/1/1
no shutdown
exit
interface “RB-3”
address 2.2.3.2/29
port 1/1/2
no shutdown
exit
interface “system”
address 2.2.2.2/32
no shutdown
exit
autonomous-system 65000
ospf 0
area 0.0.0.0
interface “system”
no shutdown
exit
interface “RB-1”
no shutdown
exit
interface “RB-3”
no shutdown
exit
exit
no shutdown
exit
ldp
interface-parameters
interface “RB-1” dual-stack
ipv4
fec-type-capability
prefix-ipv6 disable
p2mp-ipv6 disable
exit
no shutdown
exit
no shutdown
exit
interface “RB-3” dual-stack
ipv4
fec-type-capability
prefix-ipv6 disable
p2mp-ipv6 disable
exit
no shutdown
exit
no shutdown
exit
exit
no shutdown
exit
bgp
group “VPN-RR”
family vpn-ipv4
peer-as 65000
neighbor 2.2.2.3
description “RB-3”
exit
exit
no shutdown
exit
exit
service
vprn 123 customer 1 create
route-distinguisher 2.2.2.2:123
auto-bind-tunnel
resolution-filter
ldp
exit
resolution filter
exit
vrf-target target:65000:123
interface “Loop” create
address 123.2.2.2/32
loopback
exit
no shutdown
exit
exit
exit all[/php]
[php tab=”RB-3 (RR) Initial”]configure
system
name “RB-3”
exit
sfm 1
sfm-type m-sfm5-12
no shutdown
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m10-1gb-xp-sfp
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
port 1/1/2
ethernet
exit
no shutdown
exit
router
interface “RB-1”
address 2.1.3.3/29
port 1/1/2
no shutdown
exit
interface “RB-2”
address 2.2.3.3/29
port 1/1/1
no shutdown
exit
interface “system”
address 2.2.2.3/32
no shutdown
exit
autonomous-system 65000
ospf 0
area 0.0.0.0
interface “system”
no shutdown
exit
interface “RB-1”
no shutdown
exit
interface “RB-2”
no shutdown
exit
exit
no shutdown
exit
ldp
interface-parameters
interface “RB-1” dual-stack
ipv4
fec-type-capability
prefix-ipv6 disable
p2mp-ipv6 disable
exit
no shutdown
exit
no shutdown
exit
interface “RB-2” dual-stack
ipv4
fec-type-capability
prefix-ipv6 disable
p2mp-ipv6 disable
exit
no shutdown
exit
no shutdown
exit
exit
no shutdown
exit
bgp
group “VPN-RRC”
family vpn-ipv4
cluster 2.2.2.3
peer-as 65000
neighbor 2.2.2.1
description “RB-1”
exit
neighbor 2.2.2.2
description “RB-2”
exit
exit
no shutdown
exit
exit
service
vprn 123 customer 1 create
route-distinguisher 2.2.2.3:123
auto-bind-tunnel
resolution-filter
ldp
exit
resolution filter
exit
vrf-target target:65000:123
interface “Loop” create
address 123.2.2.3/32
loopback
exit
no shutdown
exit
exit
exit all[/php]
[/codegroup]
The CSC Operators Network uses IS-IS as the IGP, RSVP-TE signalled LSPs, SC-R3 is the local VPN Route Reflector for AS64512.
[codegroup]
[php tab=”SC-R1 Initial”]configure
system
name “SC-R1”
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m5-1gb-sfp-b
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
port 1/1/2
ethernet
exit
no shutdown
exit
router
interface “SC-R2”
address 10.1.2.1/29
port 1/1/1
no shutdown
exit
interface “SC-R3”
address 10.1.3.1/29
port 1/1/2
no shutdown
exit
interface “system”
address 10.10.10.1/32
no shutdown
exit
autonomous-system 64512
isis
level-capability level-2
area-id 49.0001
traffic-engineering
interface “system”
no shutdown
exit
interface “SC-R2”
no shutdown
exit
interface “SC-R3”
no shutdown
exit
no shutdown
exit
mpls
interface “system”
no shutdown
exit
interface “SC-R2”
no shutdown
exit
interface “SC-R3”
no shutdown
exit
path “LOOSE_HOPS”
no shutdown
exit
lsp “SC-R2”
to 10.10.10.2
cspf
adspec
fast-reroute facility
exit
primary “LOOSE_HOPS”
exit
no shutdown
exit
lsp “SC-R3”
to 10.10.10.3
cspf
adspec
fast-reroute facility
exit
primary “LOOSE_HOPS”
exit
no shutdown
exit
no shutdown
exit
rsvp
interface “system”
no shutdown
exit
interface “SC-R2”
no shutdown
exit
interface “SC-R3”
no shutdown
exit
no shutdown
exit
bgp
group “IBGP”
family vpn-ipv4
peer-as 64512
neighbor 10.10.10.3
description “SC-R3″
exit
exit
no shutdown
exit
exit
exit all[/php]
[php tab=”SC-R2 Initial”]configure
system
name “SC-R2”
exit
sfm 1
sfm-type m-sfm5-12
no shutdown
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m10-1gb-xp-sfp
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
port 1/1/2
ethernet
exit
no shutdown
exit
router
interface “SC-R1”
address 10.1.2.2/29
port 1/1/1
no shutdown
exit
interface “SC-R3”
address 10.2.3.2/29
port 1/1/2
no shutdown
exit
interface “system”
address 10.10.10.2/32
no shutdown
exit
autonomous-system 64512
isis 0
level-capability level-2
area-id 49.0001
traffic-engineering
interface “system”
no shutdown
exit
interface “SC-R1”
no shutdown
exit
interface “SC-R3”
no shutdown
exit
no shutdown
exit
mpls
interface “system”
no shutdown
exit
interface “SC-R1”
no shutdown
exit
interface “SC-R3”
no shutdown
exit
path “LOOSE_HOPS”
no shutdown
exit
lsp “SC-R3”
to 10.10.10.3
cspf
adspec
fast-reroute facility
exit
primary “LOOSE_HOPS”
exit
no shutdown
exit
lsp “SC-R1”
to 10.10.10.1
cspf
adspec
fast-reroute facility
exit
primary “LOOSE_HOPS”
exit
no shutdown
exit
no shutdown
exit
rsvp
interface “system”
no shutdown
exit
interface “SC-R1”
no shutdown
exit
interface “SC-R3”
no shutdown
exit
no shutdown
exit
bgp
group “IBGP”
family vpn-ipv4
peer-as 64512
neighbor 10.10.10.3
description “SC-R3″
exit
exit
no shutdown
exit
exit
exit all[/php]
[php tab=”SC-R3 (RR) Initial”]configure
system
name “SC-R3”
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m5-1gb-sfp-b
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
port 1/1/2
ethernet
exit
no shutdown
exit
router
interface “SC-R1”
address 10.1.3.3/29
port 1/1/1
no shutdown
exit
interface “SC-R2”
address 10.2.3.3/29
port 1/1/2
no shutdown
exit
interface “system”
address 10.10.10.3/32
no shutdown
exit
autonomous-system 64512
isis
level-capability level-2
area-id 49.0001
traffic-engineering
interface “system”
no shutdown
exit
interface “SC-R1”
no shutdown
exit
interface “SC-R2”
no shutdown
exit
no shutdown
exit
mpls
interface “system”
no shutdown
exit
interface “SC-R1”
no shutdown
exit
interface “SC-R2”
no shutdown
exit
path “LOOSE_HOPS”
no shutdown
exit
lsp “SC-R1”
to 10.10.10.1
cspf
adspec
fast-reroute facility
exit
primary “LOOSE_HOPS”
exit
no shutdown
exit
lsp “SC-R2”
to 10.10.10.2
cspf
adspec
fast-reroute facility
exit
primary “LOOSE_HOPS”
exit
no shutdown
exit
no shutdown
exit
rsvp
interface “system”
no shutdown
exit
interface “SC-R1”
no shutdown
exit
interface “SC-R2”
no shutdown
exit
no shutdown
exit
bgp
group “IBGP”
family vpn-ipv4
cluster 10.10.10.3
peer-as 64512
neighbor 10.10.10.1
description “SC-R1”
exit
neighbor 10.10.10.2
description “SC-R2”
exit
exit
no shutdown
exit
exit
exit all[/php][/codegroup]

At this stage, we have three Islands – Region A, Region B and the Carrier of Carriers Network (without the CSC configuration)

RA-1 is the Region A CSC-CE Router (which is also operating as a regular PE Router) which peers with the CSC Network SC-R1 to exchange labeled routes using BGP. Note: Both of these routers are running SROS 12.0R6 and are configured in the “old way”

Create VPRN 200 on SC-R1 and BGP Peering for RA-1:
[php]configure
router
policy-options
begin
policy-statement “PS_MPBGP_VPN_TO_BGP”
entry 10
from
protocol bgp-vpn
exit
action accept
exit
exit
default-action reject
exit
commit
exit
exit
port 1/1/3
ethernet
mode network
exit
no shutdown
exit
service
vprn 200 customer 1 create
carrier-carrier-vpn
autonomous-system 64512
route-distinguisher 10.10.10.1:200
auto-bind rsvp-te
vrf-target target:64512:200
network-interface “RA-1” create
address 10.1.1.11/24
port 1/1/3
no shutdown
exit
bgp
split-horizon
group “RegionA”
family ipv4
as-override
export “PS_MPBGP_VPN_TO_BGP”
peer-as 65000
neighbor 10.1.1.1
advertise-label ipv4
exit
exit
no shutdown
exit
no shutdown
exit
exit
exit all[/php]
Of note is the definition of the VPRN type: carrier-carrier-vpn, once enabled regular IP interfaces cannot be created, only “network-interfaces” which must be on Ethernet ports that are of mode network. As both Region A and Region B both use AS 65000, as-override has been used to stop AS-Path loops occuring when advertising prefixes from other regions with the same BGP AS. We are using the IPv4 Address family but are specifically advertising associated MPLS Labels (This particular configuration component is different after SROS Release 14.0R4) Routing policy is defined and used to export VPN routes to RA-1.

Create the BGP Peering for RA-1 with SC-R1:
[php]configure
port 1/1/3
ethernet
mode network
exit
no shutdown
exit
router
interface “SC-R1”
address 10.1.1.1/24
port 1/1/3
no shutdown
exit
policy-options
begin
prefix-list “RegionA”
prefix 1.1.1.0/24 prefix-length-range 32-32
exit
policy-statement “PS_BGP_EXP_CSC”
entry 10
from
prefix-list “RegionA”
exit
action accept
exit
exit
default-action reject
exit
commit
exit
bgp
group “CSC”
export “PS_BGP_EXP_CSC”
neighbor 10.1.1.11
description “SC-R1”
peer-as 64512
advertise-label ipv4 include-ldp-prefix
exit
exit
exit
exit all[/php]
Routing policy is used to advertise any system addresses from Region A (1.1.1.x/32) with MPLS labels to the CSC

RB-1 is the Region B CSC-CE Router (which is also operating as a regular PE Router) which peers with the CSC Network SC-R2 to exchange labeled routes using BGP. Note: Both of these routers are running SROS 14.0R8 and are configured in the “new way”

Create VPRN 200 on SC-R2 and BGP Peering for RB-1:
[php]configure
router
policy-options
begin
policy-statement “PS_MPBGP_VPN_TO_BGP”
entry 10
from
protocol bgp-vpn
exit
action accept
exit
exit
default-action reject
exit
commit
exit
exit
port 1/1/3
ethernet
mode network
exit
no shutdown
exit
service
vprn 200 customer 1 create
carrier-carrier-vpn
autonomous-system 64512
route-distinguisher 10.10.10.2:200
auto-bind-tunnel
resolution-filter
rsvp
exit
resolution filter
exit
vrf-target target:64512:200
network-interface “RB-1” create
address 10.2.2.11/24
port 1/1/3
no shutdown
exit
bgp
split-horizon
group “RB”
family label-ipv4
as-override
export “PS_MPBGP_VPN_TO_BGP”
peer-as 65000
neighbor 10.2.2.1
exit
exit
no shutdown
exit
no shutdown
exit
exit
exit all[/php]
The main difference compared with the configuration on SC-1 besides the auto-bind-tunnel changes, are that we are specifically peering using the label-ipv4 address family instead of the ipv4 address family and advertising labels.

Create the BGP Peering for RB-1 with SC-R2:
[php]configure
port 1/1/3
ethernet
mode network
exit
no shutdown
exit
router
interface “SC-R2”
address 10.2.2.1/24
port 1/1/3
no shutdown
exit
policy-options
begin
prefix-list “RegionB”
prefix 2.2.2.0/24 prefix-length-range 32-32
exit
policy-statement “PS_BGP_EXP_CSC”
entry 10
from
prefix-list “RegionB”
exit
action accept
exit
exit
default-action reject
exit
commit
exit
bgp
group “CSC”
export “PS_BGP_EXP_CSC”
neighbor 10.2.2.11
description “SC-R2″
family label-ipv4
peer-as 64512
advertise-ldp-prefix
exit
exit
exit
exit all[/php]
Routing policy is used to advertise any system addresses from Region B (2.2.2.x/32) with MPLS labels to the CSC

Once all peers are configured, examine the CSC VPRN 200 Routing Tables:
[codegroup][php tab=”SC-R1 VPRN 200″]A:SC-R1# show router 200 route-table

===============================================================================
Route Table (Service: 200)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
——————————————————————————-
1.1.1.1/32 Remote BGP 00h07m55s 170
10.1.1.1 0
1.1.1.2/32 Remote BGP 00h07m55s 170
10.1.1.1 0
1.1.1.3/32 Remote BGP 00h07m55s 170
10.1.1.1 0
2.2.2.1/32 Remote BGP VPN 00h07m28s 170
10.10.10.2 (tunneled:RSVP:1) 0
2.2.2.2/32 Remote BGP VPN 00h07m28s 170
10.10.10.2 (tunneled:RSVP:1) 0
2.2.2.3/32 Remote BGP VPN 00h07m28s 170
10.10.10.2 (tunneled:RSVP:1) 0
10.1.1.0/24 Local Local 00h08m29s 0
RA-1 0
10.2.2.0/24 Remote BGP VPN 00h07m28s 170
10.10.10.2 (tunneled:RSVP:1) 0
——————————————————————————-
No. of Routes: 8
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================[/php]
[php tab=”SC-R2 VPRN 200″]A:SC-R2# show router 200 route-table

===============================================================================
Route Table (Service: 200)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
——————————————————————————-
1.1.1.1/32 Remote BGP VPN 00h07m20s 170
10.10.10.1 (tunneled:RSVP:2) 0
1.1.1.2/32 Remote BGP VPN 00h07m20s 170
10.10.10.1 (tunneled:RSVP:2) 0
1.1.1.3/32 Remote BGP VPN 00h07m20s 170
10.10.10.1 (tunneled:RSVP:2) 0
2.2.2.1/32 Remote BGP_LABEL 02h02m23s 170
10.2.2.1 0
2.2.2.2/32 Remote BGP_LABEL 02h02m23s 170
10.2.2.1 0
2.2.2.3/32 Remote BGP_LABEL 02h02m23s 170
10.2.2.1 0
10.1.1.0/24 Remote BGP VPN 00h07m20s 170
10.10.10.1 (tunneled:RSVP:2) 0
10.2.2.0/24 Local Local 02h11m42s 0
RB-1 0
——————————————————————————-
No. of Routes: 8
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================[/php][/codegroup]

With Region A and Region B connectivity via the CSC we should be able to see the initial list of tunnels on the CSC-CE routers (RA-1 and RA-2):[codegroup]
[php tab=”RA-1 (CSC-CE) Tunnel Table”]A:RA-1# show router tunnel-table

===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination Owner Encap TunnelId Pref Nexthop Metric
——————————————————————————-
1.1.1.2/32 ldp MPLS – 9 1.1.2.2 100
1.1.1.3/32 ldp MPLS – 9 1.1.3.3 100
2.2.2.1/32 bgp MPLS – 10 10.1.1.11 1000
2.2.2.2/32 bgp MPLS – 10 10.1.1.11 1000
2.2.2.3/32 bgp MPLS – 10 10.1.1.11 1000
——————————————————————————-
Flags: B = BGP backup route available
E = inactive best-external BGP route
===============================================================================[/php]

[php tab=”RB-1 (CSC-CE) Tunnel Table”]A:RB-1# show router tunnel-table

===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination Owner Encap TunnelId Pref Nexthop Metric
——————————————————————————-
1.1.1.1/32 ldp MPLS – 9 1.1.2.1 100
1.1.1.3/32 ldp MPLS – 9 1.2.3.3 100
——————————————————————————-
Flags: B = BGP backup route available
E = inactive best-external BGP route
===============================================================================[/php]
[php tab=”RA-3 Tunnel Table”]A:RA-3# show router tunnel-table

===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination Owner Encap TunnelId Pref Nexthop Metric
——————————————————————————-
1.1.1.1/32 ldp MPLS – 9 1.1.3.1 100
1.1.1.2/32 ldp MPLS – 9 1.2.3.2 100
——————————————————————————-
Flags: B = BGP backup route available
E = inactive best-external BGP route
===============================================================================[/php]
[php tab=”RB-2 Tunnel Table”]A:RB-2# show router tunnel-table

===============================================================================
IPv4 Tunnel Table (Router: Base)
===============================================================================
Destination Owner Encap TunnelId Pref Nexthop Metric
——————————————————————————-
2.2.2.1/32 ldp MPLS 65597 9 2.1.3.1 100
2.2.2.2/32 ldp MPLS 65537 9 2.2.3.2 100
——————————————————————————-
Flags: B = BGP backup route available
E = inactive best-external BGP route
===============================================================================[/php][/codegroup]
RA-2, RA-3, RB-2 and RB-3 need the CSC-CEs to to redistribute the prefixes we learnt from the CSC into OSPF and LDP.[codegroup]
[php tab=”RA-1 (CSC-CE) Redistribution”]configure
router
policy-options
begin
prefix-list “RegionB”
prefix 2.2.2.0/24 prefix-length-range 32-32
exit
policy-statement “PS_OSPF_LDP_EXP_RB”
entry 10
from
prefix-list “RegionB”
exit
action accept
exit
exit
default-action reject
exit
commit
exit
ospf
asbr
export “PS_OSPF_LDP_EXP_RB”
exit
ldp
export-tunnel-table “PS_OSPF_LDP_EXP_RB”
exit
exit
exit all[/php]
[php tab=”RB-1 (CSC-CE) Redistribution”]configure
router
policy-options
begin
prefix-list “RegionA”
prefix 1.1.1.0/24 prefix-length-range 32-32
exit
policy-statement “PS_OSPF_LDP_EXP_RA”
entry 10
from
prefix-list “RegionA”
exit
action accept
exit
exit
default-action drop
exit
exit
commit
exit
ospf
asbr
export “PS_OSPF_LDP_EXP_RA”
exit
ldp
export-tunnel-table “PS_OSPF_LDP_EXP_RA”
exit[/php][/codegroup]Remember: SROS specifically requires OSPF in the GRT to be defined as an ASBR if you intend to export routes.
[codegroup][php tab=”RA-2 Tunnel Table”]A:RA-2# show router tunnel-table

===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination Owner Encap TunnelId Pref Nexthop Metric
——————————————————————————-
1.1.1.1/32 ldp MPLS – 9 1.1.2.1 100
1.1.1.3/32 ldp MPLS – 9 1.2.3.3 100
2.2.2.1/32 ldp MPLS – 9 1.1.2.1 1
2.2.2.2/32 ldp MPLS – 9 1.1.2.1 1
2.2.2.3/32 ldp MPLS – 9 1.1.2.1 1
——————————————————————————-
Flags: B = BGP backup route available
E = inactive best-external BGP route
===============================================================================[/php]
[php tab=”RA-3 Tunnel Table”]A:RA-3# show router tunnel-table

===============================================================================
Tunnel Table (Router: Base)
===============================================================================
Destination Owner Encap TunnelId Pref Nexthop Metric
——————————————————————————-
1.1.1.1/32 ldp MPLS – 9 1.1.3.1 100
1.1.1.2/32 ldp MPLS – 9 1.2.3.2 100
2.2.2.1/32 ldp MPLS – 9 1.1.3.1 1
2.2.2.2/32 ldp MPLS – 9 1.1.3.1 1
2.2.2.3/32 ldp MPLS – 9 1.1.3.1 1
——————————————————————————-
Flags: B = BGP backup route available
E = inactive best-external BGP route
===============================================================================[/php]
[php tab=”RB-2 Tunnel Table”]A:RB-2# show router tunnel-table

===============================================================================
IPv4 Tunnel Table (Router: Base)
===============================================================================
Destination Owner Encap TunnelId Pref Nexthop Metric
——————————————————————————-
1.1.1.1/32 ldp MPLS 65620 9 2.1.3.1 1
1.1.1.2/32 ldp MPLS 65621 9 2.1.3.1 1
1.1.1.3/32 ldp MPLS 65622 9 2.1.3.1 1
2.2.2.1/32 ldp MPLS 65597 9 2.1.3.1 100
2.2.2.2/32 ldp MPLS 65537 9 2.2.3.2 100
——————————————————————————-
Flags: B = BGP backup route available
E = inactive best-external BGP route
===============================================================================[/php][/codegroup]Okay, we have tunnels between all routers in both Regions A and B.
The final step to get VPRN connectivity between Region A and Region B is to peer the VPN Route Reflectors (RA-3 and RB-3)[codegroup][php tab=”RA-3 RR Peering”]configure
bgp
group “VPN-RR-RegionB”
family vpn-ipv4
peer-as 65000
neighbor 2.2.2.3
description “RB-3″
exit
exit
exit
exit all[/php]
[php tab=”RB-3 RR Peering”]configure
bgp
group “VPN-RR-RegionA”
family vpn-ipv4
peer-as 65000
neighbor 1.1.1.3
description “RA-3″
exit
exit
exit
exit all[/php][/codegroup]
We should be able to now verify this with VPRN 123
[codegroup][php tab=”RA-1 Verify”]
A:RA-1# show router 123 route-table

===============================================================================
Route Table (Service: 123)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
——————————————————————————-
123.1.1.1/32 Local Local 00h50m13s 0
Loop 0
123.1.1.2/32 Remote BGP VPN 00h44m50s 170
1.1.1.2 (tunneled) 0
123.1.1.3/32 Remote BGP VPN 00h45m49s 170
1.1.1.3 (tunneled) 0
123.2.2.1/32 Remote BGP VPN 00h00m26s 170
2.2.2.1 (tunneled:BGP) 0
123.2.2.2/32 Remote BGP VPN 00h00m26s 170
2.2.2.2 (tunneled:BGP) 0
123.2.2.3/32 Remote BGP VPN 00h00m26s 170
2.2.2.3 (tunneled:BGP) 0
——————————————————————————-
No. of Routes: 6
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
A:RA-1# ping router 123 123.1.1.1 count 1
PING 123.1.1.1 56 data bytes
64 bytes from 123.1.1.1: icmp_seq=1 ttl=64 time=0.245ms.

—- 123.1.1.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 0.245ms, avg = 0.245ms, max = 0.245ms, stddev = 0.000ms
A:RA-1# ping router 123 123.1.1.2 count 1
PING 123.1.1.2 56 data bytes
64 bytes from 123.1.1.2: icmp_seq=1 ttl=64 time=1.26ms.

—- 123.1.1.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.26ms, avg = 1.26ms, max = 1.26ms, stddev = 0.000ms
A:RA-1# ping router 123 123.1.1.3 count 1
PING 123.1.1.3 56 data bytes
64 bytes from 123.1.1.3: icmp_seq=1 ttl=64 time=1.37ms.

—- 123.1.1.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.37ms, avg = 1.37ms, max = 1.37ms, stddev = 0.000ms
A:RA-1# ping router 123 123.2.2.1 count 1
PING 123.2.2.1 56 data bytes
64 bytes from 123.2.2.1: icmp_seq=1 ttl=64 time=3.46ms.

—- 123.2.2.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 3.46ms, avg = 3.46ms, max = 3.46ms, stddev = 0.000ms
A:RA-1# ping router 123 123.2.2.2 count 1
PING 123.2.2.2 56 data bytes
64 bytes from 123.2.2.2: icmp_seq=1 ttl=64 time=3.92ms.

—- 123.2.2.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 3.92ms, avg = 3.92ms, max = 3.92ms, stddev = 0.000ms
A:RA-1# ping router 123 123.2.2.3 count 1
PING 123.2.2.3 56 data bytes
64 bytes from 123.2.2.3: icmp_seq=1 ttl=64 time=3.83ms.

—- 123.2.2.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 3.83ms, avg = 3.83ms, max = 3.83ms, stddev = 0.000ms[/php]
[php tab=”RA-2 Verify”]A:RA-2# show router 123 route-table

===============================================================================
Route Table (Service: 123)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
——————————————————————————-
123.1.1.1/32 Remote BGP VPN 00h46m12s 170
1.1.1.1 (tunneled) 0
123.1.1.2/32 Local Local 00h48m34s 0
Loop 0
123.1.1.3/32 Remote BGP VPN 00h46m12s 170
1.1.1.3 (tunneled) 0
123.2.2.1/32 Remote BGP VPN 00h01m31s 170
2.2.2.1 (tunneled) 0
123.2.2.2/32 Remote BGP VPN 00h01m31s 170
2.2.2.2 (tunneled) 0
123.2.2.3/32 Remote BGP VPN 00h01m31s 170
2.2.2.3 (tunneled) 0
——————————————————————————-
No. of Routes: 6
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
A:RA-2# ping router 123 123.1.1.1 count 1
PING 123.1.1.1 56 data bytes
64 bytes from 123.1.1.1: icmp_seq=1 ttl=64 time=1.12ms.

—- 123.1.1.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.12ms, avg = 1.12ms, max = 1.12ms, stddev = 0.000ms
A:RA-2# ping router 123 123.1.1.2 count 1
PING 123.1.1.2 56 data bytes
64 bytes from 123.1.1.2: icmp_seq=1 ttl=64 time=0.171ms.

—- 123.1.1.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 0.171ms, avg = 0.171ms, max = 0.171ms, stddev = 0.000ms
A:RA-2# ping router 123 123.1.1.3 count 1
PING 123.1.1.3 56 data bytes
64 bytes from 123.1.1.3: icmp_seq=1 ttl=64 time=1.31ms.

—- 123.1.1.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.31ms, avg = 1.31ms, max = 1.31ms, stddev = 0.000ms
A:RA-2# ping router 123 123.2.2.1 count 1
PING 123.2.2.1 56 data bytes
64 bytes from 123.2.2.1: icmp_seq=1 ttl=64 time=4.12ms.

—- 123.2.2.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 4.12ms, avg = 4.12ms, max = 4.12ms, stddev = 0.000ms
A:RA-2# ping router 123 123.2.2.2 count 1
PING 123.2.2.2 56 data bytes
64 bytes from 123.2.2.2: icmp_seq=1 ttl=64 time=7.76ms.

—- 123.2.2.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 7.76ms, avg = 7.76ms, max = 7.76ms, stddev = 0.000ms
A:RA-2# ping router 123 123.2.2.3 count 1
PING 123.2.2.3 56 data bytes
64 bytes from 123.2.2.3: icmp_seq=1 ttl=64 time=4.53ms.

—- 123.2.2.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 4.53ms, avg = 4.53ms, max = 4.53ms, stddev = 0.000ms[/php]
[php tab=”RA-3 Verify”]A:RA-3# show router 123 route-table

===============================================================================
Route Table (Service: 123)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
——————————————————————————-
123.1.1.1/32 Remote BGP VPN 00h47m48s 170
1.1.1.1 (tunneled) 0
123.1.1.2/32 Remote BGP VPN 00h46m56s 170
1.1.1.2 (tunneled) 0
123.1.1.3/32 Local Local 00h49m05s 0
Loop 0
123.2.2.1/32 Remote BGP VPN 00h02m37s 170
2.2.2.1 (tunneled) 0
123.2.2.2/32 Remote BGP VPN 00h02m37s 170
2.2.2.2 (tunneled) 0
123.2.2.3/32 Remote BGP VPN 00h02m37s 170
2.2.2.3 (tunneled) 0
——————————————————————————-
No. of Routes: 6
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
A:RA-3>config>router# ping router 123 123.1.1.1 count 1
PING 123.1.1.1 56 data bytes
64 bytes from 123.1.1.1: icmp_seq=1 ttl=64 time=1.17ms.

—- 123.1.1.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.17ms, avg = 1.17ms, max = 1.17ms, stddev = 0.000ms
A:RA-3>config>router# ping router 123 123.1.1.2 count 1
PING 123.1.1.2 56 data bytes
64 bytes from 123.1.1.2: icmp_seq=1 ttl=64 time=1.07ms.

—- 123.1.1.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.07ms, avg = 1.07ms, max = 1.07ms, stddev = 0.000ms
A:RA-3>config>router# ping router 123 123.1.1.3 count 1
PING 123.1.1.3 56 data bytes
64 bytes from 123.1.1.3: icmp_seq=1 ttl=64 time=0.166ms.

—- 123.1.1.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 0.166ms, avg = 0.166ms, max = 0.166ms, stddev = 0.000ms
A:RA-3>config>router# ping router 123 123.2.2.1 count 1
PING 123.2.2.1 56 data bytes
64 bytes from 123.2.2.1: icmp_seq=1 ttl=64 time=6.13ms.

—- 123.2.2.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 6.13ms, avg = 6.13ms, max = 6.13ms, stddev = 0.000ms
A:RA-3>config>router# ping router 123 123.2.2.2 count 1
PING 123.2.2.2 56 data bytes
64 bytes from 123.2.2.2: icmp_seq=1 ttl=64 time=5.04ms.

—- 123.2.2.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 5.04ms, avg = 5.04ms, max = 5.04ms, stddev = 0.000ms
A:RA-3>config>router# ping router 123 123.2.2.3 count 1
PING 123.2.2.3 56 data bytes
64 bytes from 123.2.2.3: icmp_seq=1 ttl=64 time=4.37ms.

—- 123.2.2.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 4.37ms, avg = 4.37ms, max = 4.37ms, stddev = 0.000ms[/php]
[php tab=”RB-1 Verify”]A:RB-1# show router 123 route-table

===============================================================================
Route Table (Service: 123)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
——————————————————————————-
123.1.1.1/32 Remote BGP VPN 00h03m33s 170
1.1.1.1 (tunneled:BGP) 0
123.1.1.2/32 Remote BGP VPN 00h03m33s 170
1.1.1.2 (tunneled:BGP) 0
123.1.1.3/32 Remote BGP VPN 00h03m33s 170
1.1.1.3 (tunneled:BGP) 0
123.2.2.1/32 Local Local 02h35m54s 0
Loop 0
123.2.2.2/32 Remote BGP VPN 02h11m21s 170
2.2.2.2 (tunneled) 0
123.2.2.3/32 Remote BGP VPN 02h11m21s 170
2.2.2.3 (tunneled) 0
——————————————————————————-
No. of Routes: 6
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
A:RB-1# ping router 123 123.1.1.1 count 1
PING 123.1.1.1 56 data bytes
64 bytes from 123.1.1.1: icmp_seq=1 ttl=64 time=4.42ms.

—- 123.1.1.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 4.42ms, avg = 4.42ms, max = 4.42ms, stddev = 0.000ms
A:RB-1# ping router 123 123.1.1.2 count 1
PING 123.1.1.2 56 data bytes
64 bytes from 123.1.1.2: icmp_seq=1 ttl=64 time=3.89ms.

—- 123.1.1.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 3.89ms, avg = 3.89ms, max = 3.89ms, stddev = 0.000ms
A:RB-1# ping router 123 123.1.1.3 count 1
PING 123.1.1.3 56 data bytes
64 bytes from 123.1.1.3: icmp_seq=1 ttl=64 time=5.97ms.

—- 123.1.1.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 5.97ms, avg = 5.97ms, max = 5.97ms, stddev = 0.000ms
A:RB-1# ping router 123 123.2.2.1 count 1
PING 123.2.2.1 56 data bytes
64 bytes from 123.2.2.1: icmp_seq=1 ttl=64 time=0.118ms.

—- 123.2.2.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 0.118ms, avg = 0.118ms, max = 0.118ms, stddev = 0.000ms
A:RB-1# ping router 123 123.2.2.2 count 1
PING 123.2.2.2 56 data bytes
64 bytes from 123.2.2.2: icmp_seq=1 ttl=64 time=1.96ms.

—- 123.2.2.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.96ms, avg = 1.96ms, max = 1.96ms, stddev = 0.000ms
A:RB-1# ping router 123 123.2.2.3 count 1
PING 123.2.2.3 56 data bytes
64 bytes from 123.2.2.3: icmp_seq=1 ttl=64 time=1.80ms.

—- 123.2.2.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.80ms, avg = 1.80ms, max = 1.80ms, stddev = 0.000ms[/php]
[php tab=”RB-2 Verify”]A:RB-2# show router 123 route-table

===============================================================================
Route Table (Service: 123)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
——————————————————————————-
123.1.1.1/32 Remote BGP VPN 00h04m13s 170
1.1.1.1 (tunneled) 0
123.1.1.2/32 Remote BGP VPN 00h04m13s 170
1.1.1.2 (tunneled) 0
123.1.1.3/32 Remote BGP VPN 00h04m13s 170
1.1.1.3 (tunneled) 0
123.2.2.1/32 Remote BGP VPN 02h11m55s 170
2.2.2.1 (tunneled) 0
123.2.2.2/32 Local Local 03h04m35s 0
Loop 0
123.2.2.3/32 Remote BGP VPN 02h36m00s 170
2.2.2.3 (tunneled) 0
——————————————————————————-
No. of Routes: 6
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
A:RB-2# ping router 123 123.1.1.1 count 1
PING 123.1.1.1 56 data bytes
64 bytes from 123.1.1.1: icmp_seq=1 ttl=64 time=3.43ms.

—- 123.1.1.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 3.43ms, avg = 3.43ms, max = 3.43ms, stddev = 0.000ms
A:RB-2# ping router 123 123.1.1.2 count 1
PING 123.1.1.2 56 data bytes
64 bytes from 123.1.1.2: icmp_seq=1 ttl=64 time=5.63ms.

—- 123.1.1.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 5.63ms, avg = 5.63ms, max = 5.63ms, stddev = 0.000ms
A:RB-2# ping router 123 123.1.1.3 count 1
PING 123.1.1.3 56 data bytes
64 bytes from 123.1.1.3: icmp_seq=1 ttl=64 time=4.40ms.

—- 123.1.1.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 4.40ms, avg = 4.40ms, max = 4.40ms, stddev = 0.000ms
A:RB-2# ping router 123 123.2.2.1 count 1
PING 123.2.2.1 56 data bytes
64 bytes from 123.2.2.1: icmp_seq=1 ttl=64 time=1.85ms.

—- 123.2.2.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.85ms, avg = 1.85ms, max = 1.85ms, stddev = 0.000ms
A:RB-2# ping router 123 123.2.2.2 count 1
PING 123.2.2.2 56 data bytes
64 bytes from 123.2.2.2: icmp_seq=1 ttl=64 time=0.150ms.

—- 123.2.2.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 0.150ms, avg = 0.150ms, max = 0.150ms, stddev = 0.000ms
A:RB-2# ping router 123 123.2.2.3 count 1
PING 123.2.2.3 56 data bytes
64 bytes from 123.2.2.3: icmp_seq=1 ttl=64 time=1.90ms.

—- 123.2.2.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.90ms, avg = 1.90ms, max = 1.90ms, stddev = 0.000ms[/php]
[php tab=”RB-3 Verify”]A:RB-3# show router 123 route-table

===============================================================================
Route Table (Service: 123)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
——————————————————————————-
123.1.1.1/32 Remote BGP VPN 00h05m23s 170
1.1.1.1 (tunneled) 0
123.1.1.2/32 Remote BGP VPN 00h05m23s 170
1.1.1.2 (tunneled) 0
123.1.1.3/32 Remote BGP VPN 00h05m23s 170
1.1.1.3 (tunneled) 0
123.2.2.1/32 Remote BGP VPN 02h13m05s 170
2.2.2.1 (tunneled) 0
123.2.2.2/32 Remote BGP VPN 02h37m03s 170
2.2.2.2 (tunneled) 0
123.2.2.3/32 Local Local 02h37m07s 0
Loop 0
——————————————————————————-
No. of Routes: 6
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
A:RB-3# ping router 123 123.1.1.1 count 1
PING 123.1.1.1 56 data bytes
64 bytes from 123.1.1.1: icmp_seq=1 ttl=64 time=3.83ms.

—- 123.1.1.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 3.83ms, avg = 3.83ms, max = 3.83ms, stddev = 0.000ms
A:RB-3# ping router 123 123.1.1.2 count 1
PING 123.1.1.2 56 data bytes
64 bytes from 123.1.1.2: icmp_seq=1 ttl=64 time=4.38ms.

—- 123.1.1.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 4.38ms, avg = 4.38ms, max = 4.38ms, stddev = 0.000ms
A:RB-3# ping router 123 123.1.1.3 count 1
PING 123.1.1.3 56 data bytes
64 bytes from 123.1.1.3: icmp_seq=1 ttl=64 time=4.47ms.

—- 123.1.1.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 4.47ms, avg = 4.47ms, max = 4.47ms, stddev = 0.000ms
A:RB-3# ping router 123 123.2.2.1 count 1
PING 123.2.2.1 56 data bytes
64 bytes from 123.2.2.1: icmp_seq=1 ttl=64 time=1.97ms.

—- 123.2.2.1 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.97ms, avg = 1.97ms, max = 1.97ms, stddev = 0.000ms
A:RB-3# ping router 123 123.2.2.2 count 1
PING 123.2.2.2 56 data bytes
64 bytes from 123.2.2.2: icmp_seq=1 ttl=64 time=1.88ms.

—- 123.2.2.2 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 1.88ms, avg = 1.88ms, max = 1.88ms, stddev = 0.000ms
A:RB-3# ping router 123 123.2.2.3 count 1
PING 123.2.2.3 56 data bytes
64 bytes from 123.2.2.3: icmp_seq=1 ttl=64 time=0.269ms.

—- 123.2.2.3 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 0.269ms, avg = 0.269ms, max = 0.269ms, stddev = 0.000ms[/php][/codegroup]While this is quite a long post, the actual configuration itself for the CSC is straight forward:

The CSC VPRN must be set to carrier-carrier-vpn
The CSC VPRN uses Ethernet ports with mode network even though its for customers
The CSC-PE and CSC-CE exchange labeled routes
The CSC-CE redistributes between OSPF/LDP and BGP

Changes to the way the RIB works on SROS and how that impacts CsC configurations was also briefly discussed.

Using BGP Flowspec with Nokia SROS to protect the network edge

This post is about how you can use BGP to augment your network edge protection. Specifically BGP Flow spec can be used to dynamically add entries to an access control list (or ip-filter in Nokia SROS speak). Previous posts have been using ExaBGP as the method to inject traffic, and while it’s good at what it does, I’ve moved on to use GoBGP not because it seems to be quite a high performance BGP implementation (I’m just doing proof of concept work) but because GoBGP’s command line interface is more friendly and better documented than ExaBGP, making it the more versatile tool to generate route advertisements in a lab on the fly (at least for me).

The lab topology is designed to demonstrate filtering with flowspec – two routers (SR1 is under my control, while the External router is a gateway to unscrupulous entities that I cannot control) and GoBGP running on a computer in my Network Operations centre.

Before getting into the flowspec piece, I’ll go through the installation process for GoBGP and the Go Language for Ubuntu 16.04 (as the routers I’m using as running on eve-ng) and the computer in the example is bridged to SR1) and then how to develop the initial GoBGP configuration.

The Go Programming language is available via the standard Ubuntu repository
[php]adam@m4600:~$ sudo apt-get install golang-go[/php]

GoBGP is pulled down as source code and compiled as part of the download process. An environmental variable needs to be set so Go knows where to pull down code, and where to build the binaries.
[php]adam@m4600:~$ export GOPATH=$HOME/go[/php]
Tell go to pull down and build gobgpd (this takes awhile at least on my machine and you don’t get an indication on what it’s doing)
[php]adam@m4600:~$ go get github.com/osrg/gobgp/gobgpd[/php]
The gobgp cli is a separate program that talks to gobgpd using the GoBGP API – downloading and building it is pretty quick in comparison to gobgpd:
[php]adam@m4600:~$ go get github.com/osrg/gobgp/gobgp[/php]
Once built, we shall copy the generated binaries to the an executable path
[php]adam@m4600:~$ sudo cp $GOPATH/bin/* /usr/local/sbin/[/php]
For those of us using the default bash shell, we can enable tab completion for the cli
[php]adam@m4600:~$ sudo cp $GOPATH/src/github.com/osrg/gobgp/tools/completion/*.bash /etc/bash_completion.d/
[/php]

Once GoBGP is installed, we can develop a configuration file that will be used to set up the GoBGP instance – in this case we’re defining the router itself and SR1 as a neighbor for the IPv4 FlowSpec address family:
A configuration file that will be used by the GoBGP Daemon enabling FlowSpec for IPv4 where we peer against SR1:[php]
adam@m4600:~$ cat gobgp-flowspec.conf
[global.config]
as = 64512
router-id = “1.2.3.4”

[[neighbors]]
[neighbors.config]
neighbor-address = “192.168.1.123”
peer-as = 64512
[[neighbors.afi-safis]]
[neighbors.afi-safis.config]
afi-safi-name = “ipv4-flowspec”[/php]
The GoBGP documentation is also a bit nicer than ExaBGP too – additional information on setting up a configuration (including different formats is described in the GoBGP getting started document)

Start the GoBGP Daemon and read our new config:
[php]adam@m4600:~$ sudo gobgpd -f gobgp-flowspec.conf
{“level”:”info”,”msg”:”gobgpd started”,”time”:”2017-05-18T22:57:22+10:00″}
{“Topic”:”Config”,”level”:”info”,”msg”:”Finished reading the config file”,”time”:”2017-05-18T22:57:22+10:00″}
{“level”:”info”,”msg”:”Peer 192.168.1.123 is added”,”time”:”2017-05-18T22:57:22+10:00″}
{“Topic”:”Peer”,”level”:”info”,”msg”:”Add a peer configuration for:192.168.1.123″,”time”:”2017-05-18T22:57:22+10:00″}
{“Key”:”192.168.1.123″,”State”:”BGP_FSM_OPENCONFIRM”,”Topic”:”Peer”,”level”:”info”,”msg”:”Peer Up”,”time”:”2017-05-18T22:57:40+10:00″}[/php]We can see shortly after gobgpd started, it has already established a peering session with SR1 (192.168.1.123)

Opening up a new shell, we can use the gobgp client to verify that:
[php]adam@m4600:~$ gobgp neighbor
Peer AS Up/Down State |#Received Accepted
192.168.1.123 64512 00:00:22 Establ | 0 0
adam@m4600:~$ gobgp neighbor 192.168.1.123
BGP neighbor is 192.168.1.123, remote AS 64512
BGP version 4, remote router ID 123.123.123.123
BGP state = established, up for 00:01:53
BGP OutQ = 0, Flops = 0
Hold time is 90, keepalive interval is 30 seconds
Configured hold time is 90, keepalive interval is 30 seconds
%s
Neighbor capabilities:
multiprotocol:
ipv4-flowspec: advertised and received
route-refresh: advertised and received
4-octet-as: advertised and received
Message statistics:
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 0 0
Keepalives: 4 5
Route Refresh: 0 0
Discarded: 0 0
Total: 5 6
Route statistics:
Advertised: 0
Received: 0
Accepted: 0[/php]
We can also get SR1 to confirm it too:
[php]*A:SR1# show router bgp summary | match Summ post-lines 100
BGP Summary
===============================================================================
Legend : D – Dynamic Neighbor
===============================================================================
Neighbor
Description
AS PktRcvd InQ Up/Down State|Rcv/Act/Sent (Addr Family)
PktSent OutQ
——————————————————————————-
192.168.1.46
64512 9 0 00h00m54s 0/0/0 (FlowIPv4)
13 0
——————————————————————————-[/php]

Now that the flowspec session is up and running but no one has exchanged any routes yet, I’ll park this for a moment to create a simple ip-filter (or ACL)[php linenumbers=”True” highlight=”5,13,14,22″]*A:SR1# /configure filter
*A:SR1>config>filter# info
———————————————-
match-list
ip-prefix-list “FPL_RFC1918” create
prefix 10.0.0.0/8
prefix 172.16.0.0/12
prefix 192.168.0.0/16
exit
exit
ip-filter 100 create
default-action forward
embed-filter flowspec router “Base” offset 1000
entry 10 create
match
src-ip ip-prefix-list “FPL_RFC1918”
exit
action
drop
exit
exit
entry 20 create
match
dst-ip ip-prefix-list “FPL_RFC1918”
exit
action
drop
exit
exit
exit
———————————————-[/php]

The match-list ip-prefix-list “FPL_RFC1918” is a way to consolidate a number of network prefixes into a single named list enabling filter configurations to be smaller and more manageable (and easier to debug)

ip-filter 100 is the thing of interest. The default-action is what happens if none of the previous entries are matched – in this case it is a black-list ip-filter, since anything we don’t specifically drop will get forwarded
Each filter entry has a number which is used for the ascending evaluation order (entry 10 before entry 20 here) and usually has a match entry and an action to perform when the match was successful. This filter is relatively simple since all it is doing is blocking IP packets that either have source or destination IP addresses from RFC1918 space.

The interesting thing is the embed-filter flowspec entry. Router “Base” is the base routing instance (as opposed to a VPRN) and offset 1000 means that any flowspec entries will be dynamically added to ip-filter 100 starting at entry 1000 (and with an additional offset determined by flowspec)

This filter will be applied to the IP interface facing the external router – This interface is associated with an Internet Enhanced Service (IES) associated with the SR1’s Global Routing Table (Router “Base”)
[php highlight=”21,22″]*A:SR1>config>service>ies# /configure service ies 1
*A:SR1>config>service>ies# info
———————————————-
interface “External” create
address 200.200.200.1/24
sap 1/1/1 create
exit
exit
no shutdown
———————————————-
*A:SR1>config>service>ies# show router route-table

—- 1.1.1.1 PING Statistics —-
2 packets transmitted, 2 packets received, 0.00% packet loss
round-trip min = 1.40ms, avg = 1.66ms, max = 1.91ms, stddev = 0.253ms[/php]
Apply the IP Filter in the ingress direction of interface External on SR1:
[php]*A:SR1>config>service>ies# interface “External” sap 1/1/1 ingress filter ip 100
*A:SR1>config>service>ies# info
———————————————-
interface “External” create
address 200.200.200.1/24
sap 1/1/1 create
ingress
filter ip 100
exit
exit
exit
no shutdown
[/php]
And repeat the test:
[php]*A:External# ping 1.1.1.1 source 172.16.0.1 count 2
PING 1.1.1.1 56 data bytes
Request timed out. icmp_seq=1.
Request timed out. icmp_seq=2.

—- 1.1.1.1 PING Statistics —-
2 packets transmitted, 0 packets received, 100% packet loss[/php]
We can see the ping failed and if we look at the counters
[php highlight=”20″]*A:SR1>config>service>ies# show filter ip 100 counters

Entry : 20
Ing. Matches : 0 pkts
Egr. Matches : 0 pkts

===============================================================================[/php]We can see the ping was unsuccessful because of entry 10 stopping traffic coming back to SR1 (because we weren’t filtering traffic leaving SR1)

Now all of a sudden we find a ping flood (well okay, calling two packets a flood is a rather long bow to draw but anyway) is coming from 100.100.100.100 and hitting 1.1.1.1[php]A:External# ping 1.1.1.1 source 100.100.100.100 count 2
PING 1.1.1.1 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=1.16ms.
64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=1.26ms.[/php]
These ICMP packets are causing havok to 1.1.1.1, so it would be nice to add some precision filtering to stop that from occuring, while allowing other traffic through – this is where our GoBGP instance and flowspec comes back into the story. We will ask gobgp to announce a flowspec “route” that specifies a discard action (actually a rate-limit of 0 kbps) for traffic from 100.100.100.100 to 1.1.1.1 using ICMP:
[python]adam@m4600:~$ gobgp global rib add -a ipv4-flowspec match source 100.100.100.100/32 destination 1.1.1.1/32 protocol icmp then discard[/python]
SR1 can confirm it recieved this route:
[php]*A:SR1>config>service>ies# show router bgp routes flow-ipv4
===============================================================================
BGP Router ID:123.123.123.123 AS:64512 Local AS:64512
===============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete

===============================================================================
BGP FLOW IPV4 Routes
===============================================================================
Flag Network Nexthop LocalPref MED
As-Path
——————————————————————————-
u*>? — 0.0.0.0 100 None
No As-Path

Community Action: rate-limit: 0 kbps
NLRI Subcomponents:
Dest Pref : 1.1.1.1/32
Src Pref : 100.100.100.100/32
Ip Proto : [ == 1 ]
——————————————————————————-
Routes : 1
===============================================================================[/php]
Flowspec can do quite interesting things, besides rate-limiting traffic to 0kbps (dropping it), it’s possible to use flowspec to tell the router to rate limit flows rather than completely blocking, remark traffic to a different level of priority or even redirect traffic to a VRF by setting the appropriate route targets (this could for example allow the forwarding of traffic into a VPRN or VRF to reach a scrubbing network prior to re-injection of clean traffic back into the network)

Lets see if ip filter has taken this flowspec entry:[php highlight=”57,58″]*A:SR1>config>service>ies# show filter ip 100

===============================================================================
IP Filter
===============================================================================
Filter Id : 100 Applied : Yes
Scope : Template Def. Action : Forward
System filter : Unchained
Radius Ins Pt : n/a
CrCtl. Ins Pt : n/a
RadSh. Ins Pt : n/a
PccRl. Ins Pt : n/a
Entries : 2/0/0/1 (Fixed/Radius/Cc/Embedded)
Sub-Entries : 6/0/0/1
Description : (Not Specified)
——————————————————————————-
Filter Match Criteria : IP
——————————————————————————-
Entry : 10
Description : (Not Specified)
Log Id : n/a
Src. IP : ip-prefix-list “FPL_RFC1918”
Src. Port : n/a
Dest. IP : 0.0.0.0/0
Dest. Port : n/a
Protocol : Undefined Dscp : Undefined
ICMP Type : Undefined ICMP Code : Undefined
Fragment : Off Src Route Opt : Off
Sampling : Off Int. Sampling : On
IP-Option : 0/0 Multiple Option: Off
TCP-syn : Off TCP-ack : Off
Option-pres : Off
Egress PBR : Disabled
Primary Action : Drop
Ing. Matches : 2 pkts (204 bytes)
Egr. Matches : 0 pkts

Entry : 20
Description : (Not Specified)
Log Id : n/a
Src. IP : 0.0.0.0/0
Src. Port : n/a
Dest. IP : ip-prefix-list “FPL_RFC1918”
Dest. Port : n/a
Protocol : Undefined Dscp : Undefined
ICMP Type : Undefined ICMP Code : Undefined
Fragment : Off Src Route Opt : Off
Sampling : Off Int. Sampling : On
IP-Option : 0/0 Multiple Option: Off
TCP-syn : Off TCP-ack : Off
Option-pres : Off
Egress PBR : Disabled
Primary Action : Drop
Ing. Matches : 0 pkts
Egr. Matches : 0 pkts

Entry : 1256
Origin : Inserted by embedded filter fSpec-0 entry 256
Description : (Not Specified)
Log Id : n/a
Src. IP : 100.100.100.100/32
Src. Port : n/a
Dest. IP : 1.1.1.1/32
Dest. Port : n/a
Protocol : 1 Dscp : Undefined
ICMP Type : Undefined ICMP Code : Undefined
Fragment : Off Src Route Opt : Off
Sampling : Off Int. Sampling : On
IP-Option : 0/0 Multiple Option: Off
TCP-syn : Off TCP-ack : Off
Option-pres : Off
Egress PBR : Disabled
Primary Action : Drop
Ing. Matches : 0 pkts
Egr. Matches : 0 pkts

===============================================================================[/php]We had specified that the offset for flowspec was 1000, this flowspec entry (number 256) became ip filter 100 entry 256. Time to verify that the ping now should fail:
[php]*A:External# ping 1.1.1.1 source 100.100.100.100 count 2
PING 1.1.1.1 56 data bytes
Request timed out. icmp_seq=1.
Request timed out. icmp_seq=2.

—- 1.1.1.1 PING Statistics —-
2 packets transmitted, 0 packets received, 100% packet loss[/php]
Yes, and we can verify that it’s only blocking icmp between those endpoints by sending IP packets from another protocol say UDP by doing a traceroute:
[php]*A:External# traceroute 1.1.1.1 source 100.100.100.100
traceroute to 1.1.1.1 from 100.100.100.100, 30 hops max, 40 byte packets
1 1.1.1.1 (1.1.1.1) 1.74 ms 1.51 ms 1.52 ms[/php]Yes, this is pretty good as far as network based filters (that aren’t performing payload based inspection) are doing, there’s quite a few match conditions that can be used to narrow things down even further. In this contrived example it may be similar effort to configure a filter on the fly but in a production environment, you could hook GoBGP to your route reflector and through a single procedure push updates to all your edge routers in one go. Another good thing about this method, is if the filtering is only temporary, it’s just as easy to remove as it was to put in.
To see what’s currently in the rib for ipv4-flowspec:
[php]adam@m4600:~$ gobgp global rib -a ipv4-flowspec
Network Next Hop AS_PATH Age Attrs
*> [destination:1.1.1.1/32][source:100.100.100.100/32][protocol:==icmp ]fictitious 00:12:06 [{Origin: ?} {Extcomms: [discard]}][/php]Withdraw the announcement:
[php]adam@m4600:~$ gobgp global rib del -a ipv4-flowspec match source 100.100.100.100/32 destination 1.1.1.1/32 protocol icmp then discard
adam@m4600:~$ gobgp global rib -a ipv4-flowspec Network not in table[/php]
Verify SR1 ip filter 100 no longer has the entry:
[php]*A:SR1>config>service>ies# show filter ip 100

===============================================================================
IP Filter
===============================================================================
Filter Id : 100 Applied : Yes
Scope : Template Def. Action : Forward
System filter : Unchained
Radius Ins Pt : n/a
CrCtl. Ins Pt : n/a
RadSh. Ins Pt : n/a
PccRl. Ins Pt : n/a
Entries : 2
Sub-Entries : 6
Description : (Not Specified)
——————————————————————————-
Filter Match Criteria : IP
——————————————————————————-
Entry : 10
Description : (Not Specified)
Log Id : n/a
Src. IP : ip-prefix-list “FPL_RFC1918”
Src. Port : n/a
Dest. IP : 0.0.0.0/0
Dest. Port : n/a
Protocol : Undefined Dscp : Undefined
ICMP Type : Undefined ICMP Code : Undefined
Fragment : Off Src Route Opt : Off
Sampling : Off Int. Sampling : On
IP-Option : 0/0 Multiple Option: Off
TCP-syn : Off TCP-ack : Off
Option-pres : Off
Egress PBR : Disabled
Primary Action : Drop
Ing. Matches : 2 pkts (204 bytes)
Egr. Matches : 0 pkts

===============================================================================[/php]Entry 1256 has left the building.

Hopefully that was a reasonable introduction to using BGP Flowspec with filtering and GoBGP.

Flowspec is quite often a piece of the puzzle in DDoS mitigation tools, usually there is some kind of automation attached which makes decisions using inputs such a Netflow/IPFIX data, SNMP polling of interfaces to track anomalous loading and in some cases threat intelligence feeds from security vendors.

Route aggregation is not always straight forward

Today’s post is a simple 3 router topology based on a true story when route aggregation didn’t appear to working as expected at first glance and some additional thought was required as to why things were behaving that way and what was required to make it do what I wanted.

I’m using 3 x Nokia VSR-Sims running SROS 14.0R4, and while the concepts discussed here are definitely flavoured through a SROS lens, the concepts will be familiar to different router platforms and their associated operating systems.

R1 and R2 are in BGP AS 12 and use ospf to advertise their respective system addresses which are used for their IBGP peerings

[codegroup]
[php linenumbers=”false” tab=”R1 Config”]
configure
system
name “R1”
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m5-1gb-sfp-b
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
router
interface “Loop”
address 100.100.100.1/24
loopback
no shutdown
exit
interface “R2”
address 10.1.2.1/27
port 1/1/1
no shutdown
exit
interface “system”
address 1.1.1.1/32
no shutdown
exit
autonomous-system 12
ospf 0
area 0.0.0.0
interface “system”
no shutdown
exit
interface “R2”
no shutdown
exit
exit
no shutdown
exit
policy-options
begin
prefix-list “PL_LOOP”
prefix 100.100.100.0/24 exact
exit
policy-statement “PS_LOOP_EXP”
entry 10
from
protocol direct
prefix-list “PL_LOOP”
exit
action accept
exit
exit
exit
commit
exit
bgp
group “IBGP”
export “PS_LOOP_EXP”
peer-as 12
neighbor 2.2.2.2
exit
exit
no shutdown
exit
exit
exit all
[/php]
[php linenumbers=”false” tab=”R2 Initial Config”]
configure
system
name “R2”
exit
card 1
card-type iom3-xp-b
mda 1
mda-type m5-1gb-sfp-b
no shutdown
exit
no shutdown
exit
port 1/1/1
ethernet
exit
no shutdown
exit
port 1/1/2
ethernet
exit
no shutdown
exit
router
interface “R1”
address 10.1.2.2/27
port 1/1/1
no shutdown
exit
interface “R3”
address 10.2.3.2/27
port 1/1/2
no shutdown
exit
interface “system”
address 2.2.2.2/32
no shutdown
exit
autonomous-system 12
router-id 2.2.2.2
ospf 0
area 0.0.0.0
interface “system”
no shutdown
exit
interface “R1”
no shutdown
exit
exit
no shutdown
exit
exit
bgp
group “IBGP”
peer-as 12
neighbor 1.1.1.1
exit
exit
no shutdown
exit
exit
exit all
[/php]
[/codegroup]

R3 which is in BGP AS 3 will be peering with R2. While we can configure R2, as R3 is in a different AS, we cannot touch it nor modify its configuration. R3 is already configured to peer with R2 and is waiting for R2 to come online.

Our configuration for R2 to establish the BGP Session:

[php]A:R2# configure router bgp
A:R2>config>router>bgp# group EBGP
*A:R2>config>router>bgp>group$ neighbor 10.2.3.3 peer-as 3
*A:R2>config>router>bgp>group$ exit all [/php]

Assuming enough time has passed for BGP to come up, lets get a quick state of play with BGP on R2:

[php]*A:R2# show router bgp summary | match “BGP Sum” post-lines 100
BGP Summary
===============================================================================
Legend : D – Dynamic Neighbor
===============================================================================
Neighbor
Description
AS PktRcvd InQ Up/Down State|Rcv/Act/Sent (Addr Family)
PktSent OutQ
——————————————————————————-
1.1.1.1
12 11 0 00h03m30s 1/1/0 (IPv4)
10 0
10.2.3.3
3 6 0 00h00m34s 8/8/1 (IPv4)
5 0
——————————————————————————-[/php]

Right now R2 has active BGP sessions with R1 and R3 – we can see that R2 has received 8 routes from R3 and has sent 1 (from R1). R2 hasn’t yet sent any routes learnt from R3 to R1 however this should happen shortly.

These are the BGP routes that R2 knows of
[php]*A:R2# show router bgp routes
===============================================================================
BGP Router ID:2.2.2.2 AS:12 Local AS:12
===============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete

============================================================= BGP IPv4 Routes
============================================================= Flag Network Nexthop (Router) As-Path
————————̵ u*>i 3.0.0.0/24 None 10.2.3.3 None 3
u*>i 3.0.1.0/24 None 10.2.3.3 None 3
u*>i 3.0.2.0/24 None 10.2.3.3 None 3
u*>i 3.0.3.0/24 None 10.2.3.3 None 3
u*>i 3.0.4.0/24 None 10.2.3.3 None 3
u*>i 3.0.5.0/24 None 10.2.3.3 None 3
u*>i 3.0.6.0/24 None 10.2.3.3 None 3
u*>i 3.0.7.0/24 None 10.2.3.3 None 3
u*>i 100.100.100.0/24 100 1.1.1.1 None No As-Path
————————̵ Routes : 9
============================================================= As this post is about route aggregation, on R2 we want [php]*A:R2# configure router aggregate 3.0.0.0/21 summary-only *A:R2# show router aggregate detail ==================
==================
LocalPref MED
Path-Id Label
2;—————————————————-
None
–
None
–
None
–
None
–
None
–
None
–
None
–
None
–
None
–
2;—————————————————-
==================[/php]
to send a summary route through to R1 (3.0.0.0/21) instead of all the individual routes. To do this we will create the aggregate route and specify it to be a summary-only route and because we can, we will include the AS-Set in the aggregate so R1 knows these came from AS 3
as-set description “Aggregate from AS3”

===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
——————————————————————————-
1.1.1.1/32 Remote OSPF 00h10m16s 10
10.1.2.1 100
2.2.2.2/32 Local Local 00h11m14s 0
system 0
3.0.0.0/21 Blackh* Aggr 00h02m13s 130
Black Hole 0
3.0.0.0/24 Remote BGP 00h05m59s 170
10.2.3.3 0
3.0.1.0/24 Remote BGP 00h05m59s 170
10.2.3.3 0
3.0.2.0/24 Remote BGP 00h05m59s 170
10.2.3.3 0
3.0.3.0/24 Remote BGP 00h05m59s 170
10.2.3.3 0
3.0.4.0/24 Remote BGP 00h05m59s 170
10.2.3.3 0
3.0.5.0/24 Remote BGP 00h06m00s 170
10.2.3.3 0
3.0.6.0/24 Remote BGP 00h06m00s 170
10.2.3.3 0
3.0.7.0/24 Remote BGP 00h06m00s 170
10.2.3.3 0
10.1.2.0/27 Local Local 00h11m00s 0
R1 0
10.2.3.0/27 Local Local 00h11m00s 0
R3 0
100.100.100.0/24 Remote BGP 00h08m53s 170
10.1.2.1 0
——————————————————————————-
No. of Routes: 14
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
* indicates that the corresponding row element may have been truncated.[/php]
As we can see in the routing table, an aggregate route is treated as its own routing protocol, so we need to develop a routing policy to advertise the aggregate to R1
[php]*A:R2# configure router policy-options
*A:R2>config>router>policy-options# begin
*A:R2>config>router>policy-options# policy-statement PS_AGGREGATE
*A:R2>config>router>policy-options>policy-statement$ entry 10 from protocol aggregate
*A:R2>config>router>policy-options>policy-statement$ entry 10 action accept
*A:R2>config>router>policy-options>policy-statement>entry>action$ exit
*A:R2>config>router>policy-options>policy-statement$ info
———————————————-
entry 10
from
protocol aggregate
exit
action accept
exit
exit
———————————————-
*A:R2>config>router>policy-options>policy-statement$ exit
*A:R2>config>router>policy-options# commit[/php]
We then can use the policy to export to our neighbor (using group IBGP or on the neighbor directly)
[php]*A:R2>config>router>policy-options# /configure router bgp group “IBGP”
*A:R2>config>router>bgp>group# export “PS_AGGREGATE”[/php]
One thing we haven’t done yet is that the EBGP next-hop 10.2.3.3 will not be visible to R1, so we can either add that interface into OSPF (as a passive interface so we don’t attempt to peer with an external router at the IGP level) or have R2 set next-hop-self (I generally prefer this as it keeps the IGP just for internal core links)
[php]*A:R2>config>router>bgp>group# next-hop-self
*A:R2>config>router>bgp>group# info
———————————————-
next-hop-self
export “PS_AGGREGATE”
peer-as 12
neighbor 1.1.1.1
exit
———————————————-[/php]
Okay, so now R1 should have 3.0.0.0/21 and the job is done, so lets verify this is working on R1
[php]A:R1# show router route-table

===============================================================================
Legend: G – generate-icmp enabled
===============================================================================
Aggregates (Router: Base)
===============================================================================
Prefix Aggr IP-Address Aggr AS
Summary AS Set State
NextHop Community NextHopType
——————————————————————————-
3.0.0.0/21 0.0.0.0 0
True True Active
None
——————————————————————————-
No. of Aggregates: 1
===============================================================================[/php]
Well that looks okay but maybe the aggregate route is wrong
[php]*A:R2>config>router>bgp>group# /admin display-config | match expression “^\ +agg”
aggregate 3.0.0.0/21 summary-only as-set description “Aggregate from AS3″[/php]
Lets try it without including the summary-only option and see if the contributing routes will get advertised to R1.
[php]*A:R2>config>router>bgp>group# /configure router aggregate 3.0.0.0/21 as-set description “Agg AS3 no summary-only”
*A:R2>config>router>bgp>group# show router bgp neighbor 1.1.1.1 advertised-routes
===============================================================================
BGP Router ID:2.2.2.2 AS:12 Local AS:12
===============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete

===============================================================================
BGP IPv4 Routes
===============================================================================
Original Attributes

Network : 3.0.2.0/24
Nexthop : 10.2.3.3
Path Id : None
From : 10.2.3.3
Res. Nexthop : 10.2.3.3
Local Pref. : n/a Interface Name : R3
Aggregator AS : None Aggregator : None
Atomic Aggr. : Not Atomic MED : None
AIGP Metric : None
Connector : None
Community : no-advertise
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 3.3.3.3
Fwd Class : None Priority : None
Flags : Used Valid Best IGP
Route Source : External
AS-Path : 3
Route Tag : 0
Neighbor-AS : 3
Orig Validation: NotFound
Source Class : 0 Dest Class : 0
Add Paths Send : Default
Last Modified : 03h55m28s

Modified Attributes

Network : 3.0.2.0/24
Nexthop : 10.2.3.3
Path Id : None
From : 10.2.3.3
Res. Nexthop : 10.2.3.3
Local Pref. : None Interface Name : R3
Aggregator AS : None Aggregator : None
Atomic Aggr. : Not Atomic MED : None
AIGP Metric : None
Connector : None
Community : no-advertise
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 3.3.3.3
Fwd Class : None Priority : None
Flags : Used Valid Best IGP
Route Source : External
AS-Path : 3
Route Tag : 0
Neighbor-AS : 3
Orig Validation: NotFound
Source Class : 0 Dest Class : 0
Add Paths Send : Default
Last Modified : 03h55m30s

——————————————————————————-
——————————————————————————-
Routes : 1
===============================================================================[/php]
3.0.2.0/24 has no-advertise attached to it!
One of the things about aggregate routes is that they aggregate the associated communities as well, so the aggregate route will have no-advertise attached to it, so it will not be advertised to R1.
Unfortunately this doesn’t appear in the show “show router aggregate detail” – the community there is the one that is manually added during the creation of the aggregate.
So how can we fix this? Well there are two methods that spring to mind and I am sure that there are more.

Option 1 – Create an import policy on R2 that just drops the no-advertise community on imported routes.
I think this is the easiest option to implement because then the normal aggregate configuration will work.
[php]*A:R2>config>router>bgp>group# /configure router policy-options
*A:R2>config>router>policy-options# begin
*A:R2>config>router>policy-options# community NO_ADV members no-advertise
*A:R2>config>router>policy-options# policy-statement PS_IGNORE_NO_ADV
*A:R2>config>router>policy-options>policy-statement$ entry 10
*A:R2>config>router>policy-options>policy-statement>entry$ from community NO_ADV
*A:R2>config>router>policy-options>policy-statement>entry$ action accept
*A:R2>config>router>policy-options>policy-statement>entry>action$ community remove “NO_ADV”
*A:R2>config>router>policy-options>policy-statement>entry>action$ back
*A:R2>config>router>policy-options>policy-statement>entry$ back
*A:R2>config>router>policy-options>policy-statement$ info
———————————————-
entry 10
from
community “NO_ADV”
exit
action accept
community remove “NO_ADV”
exit
exit
———————————————-
*A:R2>config>router>policy-options>policy-statement$ back
*A:R2>config>router>policy-options# commit
*A:R2>config>router>policy-options# /configure router bgp group “EBGP”
*A:R2>config>router>bgp>group# neighbor 10.2.3.3 import “PS_IGNORE_NO_ADV”[/php]
Let’s see if that has resolved things:
[php]*A:R2>config>router>bgp>group# show router bgp neighbor 1.1.1.1 advertised-routes
===============================================================================
BGP Router ID:2.2.2.2 AS:12 Local AS:12
===============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete

===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id Label
As-Path
——————————————————————————-
i 3.0.0.0/21 100 None
2.2.2.2 None –
3
i 3.0.0.0/24 100 None
2.2.2.2 None –
3
i 3.0.1.0/24 100 None
2.2.2.2 None –
3
i 3.0.2.0/24 100 None
2.2.2.2 None –
3
i 3.0.3.0/24 100 None
2.2.2.2 None –
3
i 3.0.4.0/24 100 None
2.2.2.2 None –
3
i 3.0.5.0/24 100 None
2.2.2.2 None –
3
i 3.0.6.0/24 100 None
2.2.2.2 None –
3
i 3.0.7.0/24 100 None
2.2.2.2 None –
3
——————————————————————————-
Routes : 9
===============================================================================[/php]
Yes, 3.0.2.0/24 is present and because none of the routes that contribute to the aggregate have no-advertise attached, the aggregate is also advertised to R1. So time to change the Aggregate route so it’s back to summary only:
[php]*A:R2>config>router>bgp>group# /configure router aggregate 3.0.0.0/21 summary-only as-set description “Aggregate from AS3”
*A:R2>config>router>bgp>group# show router bgp neighbor 1.1.1.1 advertised-routes
===============================================================================
BGP Router ID:2.2.2.2 AS:12 Local AS:12
===============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete

Option 2 – Using Static Routes for Aggregation
If we wish to respect the no-advertise binding on 3.0.2.0/24, we can simulate some of the behavior of an aggregate route without caring about no-advertise (or no-export if we are concerned about advertisements outside of our AS).
First we need to remove the import policy on R2 facing R3.
[php]*A:R2>config>router>bgp>group# info
———————————————-
neighbor 10.2.3.3
import “PS_IGNORE_NO_ADV”
peer-as 3
exit
———————————————-
*A:R2>config>router>bgp>group# neighbor 10.2.3.3 no import[/php]
And remove the aggregate for 3.0.0.0/21
[php]*A:R2>config>router>bgp>group# /configure router no aggregate 3.0.0.0/21 [/php]
Now we create a static black-hole route with BGP community 12:12 attached to it. We’re attaching the community so we can distinguish between regular static routes and our “aggregate”
[php]A:R2>config>router# static-route-entry 3.0.0.0/21
*A:R2>config>router>static-route-entry$ black-hole
*A:R2>config>router>static-route-entry>black-hole$ community 12:12
*A:R2>config>router>static-route-entry>black-hole$ no shutdown[/php]
As a note, SROS Release 14 changed the specific syntax for creating static routes but the concepts generally remain the same for previous SROS versions.
Now we’ll work on the routing policy to advertise our static aggregate route.
First we’ll create a named community that was used for our aggregate:
[php]*A:R2>config>router>static-route-entry$ /configure router policy-options
*A:R2>config>router>policy-options# begin
*A:R2>config>router>policy-options# community STATIC_AGG members 12:12 [/php]
Now we create a prefix list to match the routes that contribute to our aggregate
[php]*A:R2>config>router>policy-options# prefix-list PL_R3_CONTRIB
*A:R2>config>router>policy-options>prefix-list$ prefix 3.0.0.0/21 longer
*A:R2>config>router>policy-options>prefix-list$ exit[/php]
Finally we take the existing PS_AGGREGATE and modify it to work with our static aggregate and drop the contributing routes:
[php]*A:R2>config>router>policy-options# policy-statement “PS_AGGREGATE”
*A:R2>config>router>policy-options>policy-statement# info
———————————————-
entry 10
from
protocol aggregate
exit
action accept
exit
exit
*A:R2>config>router>policy-options>policy-statement# entry 10
*A:R2>config>router>policy-options>policy-statement>entry# from protocol static
*A:R2>config>router>policy-options>policy-statement>entry# from community “STATIC_AGG”
*A:R2>config>router>policy-options>policy-statement>entry# back
*A:R2>config>router>policy-options>policy-statement# entry 20
*A:R2>config>router>policy-options>policy-statement>entry$ from prefix-list “PL_R3_CONTRIB”
*A:R2>config>router>policy-options>policy-statement>entry$ action drop
*A:R2>config>router>policy-options>policy-statement>entry>action$ exit
*A:R2>config>router>policy-options>policy-statement>entry$ exit
*A:R2>config>router>policy-options>policy-statement# info
———————————————-
entry 10
from
protocol static
community “STATIC_AGG”
exit
action accept
exit
exit
entry 20
from
prefix-list “PL_R3_CONTRIB”
exit
action drop
exit
exit
*A:R2>config>router>policy-options>policy-statement# back
*A:R2>config>router>policy-options# commit[/php]
Lets check what R2 is advertising to R1:
[php]*A:R2>config>router>bgp>group# show router bgp neighbor 1.1.1.1 advertised-routes
===============================================================================
BGP Router ID:2.2.2.2 AS:12 Local AS:12
===============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete

===============================================================================
BGP IPv4 Routes
===============================================================================
Flag Network LocalPref MED
Nexthop (Router) Path-Id Label
As-Path
——————————————————————————-
? 3.0.0.0/21 100 None
2.2.2.2 None –
No As-Path
——————————————————————————-
Routes : 1
===============================================================================[/php]
There are a couple of issues with this implementation the AS-Path information is lost (nothing we can do about that) but more importantly, this “aggregate” will stay up even if the contributing routes are not present. To overcome this issue, the static route can be associated with a prefix-list which will be used to determine if the static route can become active. It should be noted that although we already have PL_R3_CONTRIB, it cannot be used here as the prefix list in the static route requires specific prefixes to match against. While matching all possible prefixes could be problematic, in most instances simply matching against a few key prefixes will be sufficient:
[php]*A:R2>config>router>bgp>group# /configure router policy-options
*A:R2>config>router>policy-options# begin
*A:R2>config>router>policy-options# prefix-list PL_R3_STATIC_AGG_OK
*A:R2>config>router>policy-options>prefix-list$ prefix 3.0.0.0/24
*A:R2>config>router>policy-options>prefix-list$ prefix 3.0.7.0/24
*A:R2>config>router>policy-options>prefix-list$ exit
*A:R2>config>router>policy-options# commit [/php]
Modify the static route to be up when any of the prefixes in PL_R3_STATIC_AGG_OK are in the routing table:
[php]*A:R2>config>router>policy-options# /configure router static-route-entry 3.0.0.0/21 black-hole prefix-list “PL_R3_STATIC_AGG_OK” any
[/php]
We can see the route is active and the prefix-list being used to validate:
[php highlight=”9,10″]*A:R2>config>router>policy-options# show router static-route detail

===============================================================================
Static Route Table (Router: Base) Family: IPv4
===============================================================================
Prefix : 3.0.0.0/21
Nexthop : n/a
Type : Blackhole
Dynamic BGP : disabled Generate ICMP : disabled
Interface : n/a Active : Y
Prefix List : PL_R3_STATIC_AGG_OK Prefix List Type : Any
Metric : 1 Preference : 5
Source Class : 0 Dest Class : 0
Admin State : Up Tag : 0
Creation Origin : manual
BFD : disabled
Community : 12:12
CPE-check : disabled
——————————————————————————-
No. of Static Routes: 1

===============================================================================[/php]
If we shutdown our BGP session to R3, the routes in PL_R3_STATIC_AGG_OK will disappear from the routing table and the static route will be brought out of service
[php highlight=”12,13,21″]*A:R2>config>router>policy-options# /configure router bgp group “EBGP”
*A:R2>config>router>bgp>group# shutdown
*A:R2>config>router>bgp>group# show router static-route detail

===============================================================================
Static Route Table (Router: Base) Family: IPv4
===============================================================================
Prefix : 3.0.0.0/21
Nexthop : n/a
Type : Blackhole
Dynamic BGP : disabled Generate ICMP : disabled
Interface : n/a Active : N
Prefix List : PL_R3_STATIC_AGG_OK Prefix List Type : Any
Metric : 1 Preference : 5
Source Class : 0 Dest Class : 0
Admin State : Up Tag : 0
Creation Origin : manual
BFD : disabled
Community : 12:12
CPE-check : disabled
Inactive Reason : prefix-list match failed
——————————————————————————-
No. of Static Routes: 1

===============================================================================[/php]
We can see the static route is down because the prefix-list match has failed and we can confirm that we aren’t advertising this to R1:
[php]*A:R2>config>router>bgp>group# show router bgp neighbor 1.1.1.1 advertised-routes
===============================================================================
BGP Router ID:2.2.2.2 AS:12 Local AS:12
===============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete

===============================================================================[/php]
Yes, so we should be offering this to R3 again:
[php]*A:R2>config>router>bgp>group# show router bgp neighbor 1.1.1.1 advertised-routes
===============================================================================
BGP Router ID:2.2.2.2 AS:12 Local AS:12
===============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete

While this scenario isn’t likely to occur all the time, based on my experience it is something to consider if things are not working quite as expected.