1. Liu Johnson Page 1 6/15/2012
MC-LAG with BGP VPLS

2. Liu Johnson Page 2 6/15/2012
Topology:
PE1 – MX960 – themask,
- Loopback address – 10.255.49.164
PE2 – MX960 – gerbil
- Loopback address – 10.255.49.149
PE3 – MX960 – leopard
- Loopback address – 10.255.49.155
PE4 – MX960 – Auror
- Loopback address – 10.255.49.138
CE1 – M10i – kitten
- ae0 address – 10.49.1.1
CE1
PE2
PE1
CE2
PE4
PE3
LACP
LACP
ICC
P
ICCP

3. Liu Johnson Page 3 6/15/2012
CE2 – M10i – beaver
- ae0 address – 10.49.1.2
VPLS, MC-LAG and ICCP states:
Remote standby state is shown as RD instead of RS.MC-LAG states are not
correct.ICCP states are correct.
{master}[edit]
regress@gerbil# run show vpls connections
Layer-2 VPN connections:
Legend for connection status (St)
EI -- encapsulation invalid NC -- interface encapsulation not
CCC/TCC/VPLS
EM -- encapsulation mismatch WE -- interface and instance encaps
not same
VC-Dn -- Virtual circuit down NP -- interface hardware not present
CM -- control-word mismatch -> -- only outbound connection is up
CN -- circuit not provisioned <- -- only inbound connection is up
OR -- out of range Up -- operational
OL -- no outgoing label Dn -- down
LD -- local site signaled down CF -- call admission control failure
RD -- remote site signaled down SC -- local and remote site ID
collision
LN -- local site not designated LM -- local site ID not minimum
designated
RN -- remote site not designated RM -- remote site ID not minimum
designated
XX -- unknown connection status IL -- no incoming label
MM -- MTU mismatch MI -- Mesh-Group ID not availble
BK -- Backup connection ST -- Standby connection
PF -- Profile parse failure PB -- Profile busy
RS -- remote site standby SN -- Static Neighbor
Legend for interface status
Up -- operational
Dn -- down
Instance: vpls1
Local site: CE1 (2)
connection-site Type St Time last up # Up
trans
3 rmt RD
4 rmt Up Jul 8 11:59:35 2010
1
Remote PE: 10.255.49.155, Negotiated control-word: No
Incoming label: 800003, Outgoing label: 800001
Local interface: vt-4/3/10.1048577, Status: Up, Encapsulation:
VPLS
Description: Intf - vpls vpls1 local site 2 remote site 4

4. Liu Johnson Page 4 6/15/2012
{master}[edit]
regress@gerbil#
{master}[edit]
regress@gerbil# run show interfaces mc-ae id 1
Member Link : ae0
Current State Machine's State: mcae active state
Local Status : active
Local State : up
Peer Status : standby
Peer State : up
Logical Interface : ae0.1
Topology Type : NULL
Local State : up
Peer State : Unknown
Peer Ip/MCP/State : Unknown
Logical Interface : ae0.2
Topology Type : NULL
Local State : up
Peer State : Unknown
Peer Ip/MCP/State : Unknown
{master}[edit]
regress@gerbil#
{master}[edit]
regress@gerbil# run show iccp detail
Redundancy Group Information for peer 10.50.1.1
TCP Connection : Established
Liveliness Detection : Up
Redundancy Group ID Status
1 Up
2 Up
3 Up
Client Application: lacpd
Redundancy Group IDs Joined: 1
Client Application: MCSNOOPD
Redundancy Group IDs Joined: None
Client Application: l2ald_iccpd_client
Redundancy Group IDs Joined: None
{master}[edit]
regress@gerbil#

5. Liu Johnson Page 5 6/15/2012
Local standby state is shown as LD instead of ST.MC-AE states are not
correct.
{master}[edit]
regress@themask# run show vpls connections
Layer-2 VPN connections:
Legend for connection status (St)
EI -- encapsulation invalid NC -- interface encapsulation not
CCC/TCC/VPLS
EM -- encapsulation mismatch WE -- interface and instance encaps
not same
VC-Dn -- Virtual circuit down NP -- interface hardware not present
CM -- control-word mismatch -> -- only outbound connection is up
CN -- circuit not provisioned <- -- only inbound connection is up
OR -- out of range Up -- operational
OL -- no outgoing label Dn -- down
LD -- local site signaled down CF -- call admission control failure
RD -- remote site signaled down SC -- local and remote site ID
collision
LN -- local site not designated LM -- local site ID not minimum
designated
RN -- remote site not designated RM -- remote site ID not minimum
designated
XX -- unknown connection status IL -- no incoming label
MM -- MTU mismatch MI -- Mesh-Group ID not availble
BK -- Backup connection ST -- Standby connection
PF -- Profile parse failure PB -- Profile busy
RS -- remote site standby SN -- Static Neighbor
Legend for interface status
Up -- operational
Dn -- down
Instance: vpls1
Local site: CE1 (1)
connection-site Type St Time last up # Up
trans
3 rmt LD
4 rmt LD
{master}[edit]
regress@themask#
{master}[edit]
regress@themask# run show interfaces mc-ae id 1
Member Link : ae0
Current State Machine's State: mcae standby state
Local Status : standby
Local State : up
Peer Status : active
Peer State : up
Logical Interface : ae0.1
Topology Type : NULL
Local State : up
Peer State : Unknown

6. Liu Johnson Page 6 6/15/2012
Peer Ip/MCP/State : Unknown
Logical Interface : ae0.2
Topology Type : NULL
Local State : up
Peer State : Unknown
Peer Ip/MCP/State : Unknown
{master}[edit]
regress@themask#
{master}[edit]
regress@themask# run show iccp detail
Redundancy Group Information for peer 10.50.1.2
TCP Connection : Established
Liveliness Detection : Up
Redundancy Group ID Status
1 Up
2 Up
3 Up
Client Application: l2ald_iccpd_client
Redundancy Group IDs Joined: None
Client Application: lacpd
Redundancy Group IDs Joined: 1
Client Application: MCSNOOPD
Redundancy Group IDs Joined: None
{master}[edit]
regress@themask#
FAILOVER Scenarios:
CE1 to PE2 link failure:
Before the link flap:
Before the link flap, PE2 is MC-LAG active router.Ping traffic
forwarded by PE2 without any drop.
[edit]
regress@kitten# run ping 10.49.2.2 rapid count 10
PING 10.49.2.2 (10.49.2.2): 56 data bytes
!!!!!!!!!!
--- 10.49.2.2 ping statistics ---
10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.138/2.071/9.892/2.608 ms
[edit]

7. Liu Johnson Page 7 6/15/2012
regress@kitten#
PE2:
{master}[edit]
regress@gerbil# run show interfaces terse ae0
Interface Admin Link Proto Local
Remote
ae0 up up
ae0.1 up up vpls
ae0.2 up up vpls
ae0.32767 up up multiservice
{master}[edit]
regress@gerbil#
{master}[edit]
regress@gerbil# run show vpls connections
Layer-2 VPN connections:
Legend for connection status (St)
EI -- encapsulation invalid NC -- interface encapsulation not
CCC/TCC/VPLS
EM -- encapsulation mismatch WE -- interface and instance encaps
not same
VC-Dn -- Virtual circuit down NP -- interface hardware not present
CM -- control-word mismatch -> -- only outbound connection is up
CN -- circuit not provisioned <- -- only inbound connection is up
OR -- out of range Up -- operational
OL -- no outgoing label Dn -- down
LD -- local site signaled down CF -- call admission control failure
RD -- remote site signaled down SC -- local and remote site ID
collision
LN -- local site not designated LM -- local site ID not minimum
designated
RN -- remote site not designated RM -- remote site ID not minimum
designated
XX -- unknown connection status IL -- no incoming label
MM -- MTU mismatch MI -- Mesh-Group ID not availble
BK -- Backup connection ST -- Standby connection
PF -- Profile parse failure PB -- Profile busy
RS -- remote site standby SN -- Static Neighbor
Legend for interface status
Up -- operational
Dn -- down
Instance: vpls1
Local site: CE1 (2)
connection-site Type St Time last up # Up
trans
3 rmt RD

8. Liu Johnson Page 8 6/15/2012
4 rmt Up Jul 8 11:59:35 2010
1
Remote PE: 10.255.49.155, Negotiated control-word: No
Incoming label: 800003, Outgoing label: 800001
Local interface: vt-4/3/10.1048577, Status: Up, Encapsulation:
VPLS
Description: Intf - vpls vpls1 local site 2 remote site 4
{master}[edit]
regress@gerbil#
{master}[edit]
regress@gerbil# run show interfaces mc-ae id 1
Member Link : ae0
Current State Machine's State: mcae active state
Local Status : active
Local State : up
Peer Status : standby
Peer State : up
Logical Interface : ae0.1
Topology Type : NULL
Local State : up
Peer State : Unknown
Peer Ip/MCP/State : Unknown
Logical Interface : ae0.2
Topology Type : NULL
Local State : up
Peer State : Unknown
Peer Ip/MCP/State : Unknown
{master}[edit]
regress@gerbil#
After the link flap:
After the link between CE1 and PE2 flapped the traffic flows through
PE1.PE2 and PE1 MC-LAG and VPLS states changed.
edit]
regress@kitten# run ping 10.49.2.2 rapid count 10
PING 10.49.2.2 (10.49.2.2): 56 data bytes
!!!!!!!!!!
--- 10.49.2.2 ping statistics ---
10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.101/1.205/1.313/0.082 ms
[edit]
regress@kitten#
PE2:

9. Liu Johnson Page 9 6/15/2012
{master}[edit]
regress@gerbil# run test interface ge-4/0/1 restart-auto-negotiation
{master}[edit]
regress@gerbil# run show interfaces ae0 terse
Interface Admin Link Proto Local
Remote
ae0 up down
ae0.1 up down vpls
ae0.2 up down vpls
ae0.32767 up down multiservice
{master}[edit]
regress@gerbil#
master}[edit]
regress@gerbil# run show interfaces mc-ae id 1
Member Link : ae0
Current State Machine's State: mcae standby state
Local Status : standby
Local State : up
Peer Status : active
Peer State : up
Logical Interface : ae0.1
Topology Type : NULL
Local State : up
Peer State : Unknown
Peer Ip/MCP/State : Unknown
Logical Interface : ae0.2
Topology Type : NULL
Local State : up
Peer State : Unknown
Peer Ip/MCP/State : Unknown
{master}[edit]
regress@gerbil#
{master}[edit]
regress@gerbil# run show vpls connections
Layer-2 VPN connections:
Legend for connection status (St)
EI -- encapsulation invalid NC -- interface encapsulation not
CCC/TCC/VPLS
EM -- encapsulation mismatch WE -- interface and instance encaps
not same
VC-Dn -- Virtual circuit down NP -- interface hardware not present
CM -- control-word mismatch -> -- only outbound connection is up
CN -- circuit not provisioned <- -- only inbound connection is up
OR -- out of range Up -- operational
OL -- no outgoing label Dn -- down
LD -- local site signaled down CF -- call admission control failure
RD -- remote site signaled down SC -- local and remote site ID
collision
LN -- local site not designated LM -- local site ID not minimum
designated

10. Liu Johnson Page 10 6/15/2012
RN -- remote site not designated RM -- remote site ID not minimum
designated
XX -- unknown connection status IL -- no incoming label
MM -- MTU mismatch MI -- Mesh-Group ID not availble
BK -- Backup connection ST -- Standby connection
PF -- Profile parse failure PB -- Profile busy
RS -- remote site standby SN -- Static Neighbor
Legend for interface status
Up -- operational
Dn -- down
Instance: vpls1
Local site: CE1 (2)
connection-site Type St Time last up # Up
trans
3 rmt LD
4 rmt LD
{master}[edit]
regress@gerbil#
PE1:
{master}[edit]
regress@themask# run show interfaces mc-ae id 1
Member Link : ae0
Current State Machine's State: mcae active state
Local Status : active
Local State : up
Peer Status : standby
Peer State : up
Logical Interface : ae0.1
Topology Type : NULL
Local State : up
Peer State : Unknown
Peer Ip/MCP/State : Unknown
Logical Interface : ae0.2
Topology Type : NULL
Local State : up
Peer State : Unknown
Peer Ip/MCP/State : Unknown
{master}[edit]
regress@themask#
{master}[edit]
regress@themask# run show vpls connections
Layer-2 VPN connections:
Legend for connection status (St)
EI -- encapsulation invalid NC -- interface encapsulation not
CCC/TCC/VPLS

11. Liu Johnson Page 11 6/15/2012
EM -- encapsulation mismatch WE -- interface and instance encaps
not same
VC-Dn -- Virtual circuit down NP -- interface hardware not present
CM -- control-word mismatch -> -- only outbound connection is up
CN -- circuit not provisioned <- -- only inbound connection is up
OR -- out of range Up -- operational
OL -- no outgoing label Dn -- down
LD -- local site signaled down CF -- call admission control failure
RD -- remote site signaled down SC -- local and remote site ID
collision
LN -- local site not designated LM -- local site ID not minimum
designated
RN -- remote site not designated RM -- remote site ID not minimum
designated
XX -- unknown connection status IL -- no incoming label
MM -- MTU mismatch MI -- Mesh-Group ID not availble
BK -- Backup connection ST -- Standby connection
PF -- Profile parse failure PB -- Profile busy
RS -- remote site standby SN -- Static Neighbor
Legend for interface status
Up -- operational
Dn -- down
Instance: vpls1
Local site: CE1 (1)
connection-site Type St Time last up # Up
trans
3 rmt RD
4 rmt Up Jul 8 12:10:58 2010
1
Remote PE: 10.255.49.155, Negotiated control-word: No
Incoming label: 800003, Outgoing label: 800000
Local interface: vt-3/3/10.1049088, Status: Up, Encapsulation:
VPLS
Description: Intf - vpls vpls1 local site 1 remote site 4
{master}[edit]
regress@themask#

12. Liu Johnson Page 12 6/15/2012
Configuration:
PE1:
{master}[edit]
regress@themask# show interfaces ae0
vlan-tagging;
encapsulation vlan-vpls;
aggregated-ether-options {
lacp {
active;
system-id 00:00:00:00:00:01;
admin-key 10;
}
mc-ae {
mc-ae-id 1;
redundancy-group 1;
chassis-id 0;
mode active-standby;
status-control active;
}
}
unit 1 {
encapsulation vlan-vpls;
vlan-id 601;
}
unit 2 {
encapsulation vlan-vpls;
vlan-id 602;
}
{master}[edit]
regress@themask#
{master}[edit]
regress@themask# show routing-instances
vpls1 {
instance-type vpls;
interface ae0.1;
interface ae0.2;
route-distinguisher 100:1;
vrf-target target:100:1;
protocols {
vpls {
traceoptions {
file vpls_mclag.log size 20m world-readable;
flag all;
}
site-range 8;

13. Liu Johnson Page 13 6/15/2012
tunnel-services {
primary vt-3/3/10;
}
site CE1 {
site-identifier 1;
interface ae0.1;
interface ae0.2;
}
}
}
}
{master}[edit]
regress@themask#
{master}[edit]
regress@themask# show protocols
rsvp {
interface all;
}
mpls {
interface all;
}
bgp {
local-address 10.255.49.164;
family inet {
unicast;
}
family l2vpn {
signaling;
}
group ibgp {
type internal;
neighbor 10.255.49.138;
neighbor 10.255.49.155;
}
}
ospf {
traffic-engineering;
area 0.0.0.0 {
interface lo0.0 {
passive;
}
interface all;
interface fxp0.0 {
disable;
}
}
}
ldp {
traceoptions {
file ldp.log size 20m world-readable;
flag all;
}
interface all;
}

14. Liu Johnson Page 14 6/15/2012
iccp {
local-ip-addr 10.50.1.1;
peer 10.50.1.2 {
redundancy-group-id-list [ 1 2 3 ];
liveness-detection {
minimum-interval 1000;
detection-time {
threshold 2000000;
}
single-hop;
}
}
traceoptions {
file iccp.log size 20m world-readable;
flag all;
}
}
lacp {
traceoptions {
file lacp.log size 20m world-readable;
flag all;
}
}
{master}[edit]
regress@themask#
PE2:
{master}[edit]
regress@gerbil# show interfaces ae0
vlan-tagging;
encapsulation vlan-vpls;
aggregated-ether-options {
lacp {
active;
system-id 00:00:00:00:00:01;
admin-key 10;
}
mc-ae {
mc-ae-id 1;
redundancy-group 1;
chassis-id 1;
mode active-standby;
status-control standby;
}
}
unit 1 {
encapsulation vlan-vpls;
vlan-id 601;
}
unit 2 {
encapsulation vlan-vpls;
vlan-id 602;
}

15. Liu Johnson Page 15 6/15/2012
{master}[edit]
regress@gerbil#
{master}[edit]
regress@gerbil# show routing-instances
vpls1 {
instance-type vpls;
interface ae0.1;
interface ae0.2;
route-distinguisher 100:2;
vrf-target target:100:1;
protocols {
vpls {
traceoptions {
file vpls_mclag.log size 20m world-readable;
flag all;
}
site-range 8;
tunnel-services {
primary vt-4/3/10;
}
site CE1 {
site-identifier 2;
interface ae0.1;
interface ae0.2;
}
}
}
}
{master}[edit]
regress@gerbil#
{master}[edit]
regress@gerbil# show protocols
rsvp {
interface all;
}
mpls {
interface all;
}
bgp {
local-address 10.255.49.149;
family inet {
unicast;
}
family l2vpn {
signaling;
}
group ibgp {
type internal;
neighbor 10.255.49.138;
neighbor 10.255.49.155;
}
}

16. Liu Johnson Page 16 6/15/2012
ospf {
traffic-engineering;
area 0.0.0.0 {
interface lo0.0 {
passive;
}
interface all;
interface fxp0.0 {
disable;
}
}
}
ldp {
traceoptions {
file ldp.log size 20m world-readable;
flag all;
}
interface all;
}
iccp {
local-ip-addr 10.50.1.2;
peer 10.50.1.1 {
redundancy-group-id-list [ 1 2 3 ];
liveness-detection {
minimum-interval 1000;
detection-time {
threshold 2000000;
}
single-hop;
}
}
traceoptions {
file iccp.log size 20m world-readable;
flag all;
}
}
lacp {
traceoptions {
file lacp.log size 20m world-readable;
flag all;
}
}
{master}[edit]

17. Liu Johnson Page 17 6/15/2012
PE3:
master}[edit]
regress@leopard# show interfaces ae0
vlan-tagging;
encapsulation vlan-vpls;
aggregated-ether-options {
lacp {
active;
system-id 00:00:00:00:00:01;
admin-key 10;
}
mc-ae {
mc-ae-id 1;
redundancy-group 1;
chassis-id 0;
mode active-standby;
status-control active;
}
}
unit 1 {
encapsulation vlan-vpls;
vlan-id 601;
}
unit 2 {
encapsulation vlan-vpls;
vlan-id 602;
}
{master}[edit]
regress@leopard#
{master}[edit]
regress@leopard# show routing-instances
vpls1 {
instance-type vpls;
interface ae0.1;
interface ae0.2;
route-distinguisher 100:4;
vrf-target target:100:1;
protocols {
vpls {
traceoptions {
file vpls_mclag.log size 20m world-readable;
flag all;
}
site-range 8;
tunnel-services {
primary vt-5/3/10;
}
site CE2 {
site-identifier 4;
interface ae0.1;
interface ae0.2;
}
}

18. Liu Johnson Page 18 6/15/2012
}
}
{master}[edit]
regress@leopard#
{master}[edit]
regress@leopard# show protocols
rsvp {
interface all;
}
mpls {
interface all;
}
bgp {
local-address 10.255.49.155;
family inet {
unicast;
}
family l2vpn {
signaling;
}
group ibgp {
type internal;
neighbor 10.255.49.164;
neighbor 10.255.49.149;
}
}
ospf {
traffic-engineering;
area 0.0.0.0 {
interface lo0.0 {
passive;
}
interface all;
interface fxp0.0 {
disable;
}
}
}
ldp {
traceoptions {
file ldp.log size 20m world-readable;
flag all;
}
interface all;
}
iccp {
local-ip-addr 10.100.1.1;
peer 10.100.1.2 {
redundancy-group-id-list [ 1 2 3 ];
liveness-detection {
minimum-interval 1000;
detection-time {
threshold 2000000;
}
single-hop;

19. Liu Johnson Page 19 6/15/2012
}
}
traceoptions {
file iccp.log size 20m world-readable;
flag all;
}
}
lacp {
traceoptions {
file lacp.log size 20m world-readable;
flag all;
}
}
{master}[edit]
regress@leopard#
PE4:
regress@auror# show interfaces ae0
vlan-tagging;
encapsulation vlan-vpls;
aggregated-ether-options {
lacp {
active;
system-id 00:00:00:00:00:01;
admin-key 10;
}
mc-ae {
mc-ae-id 1;
redundancy-group 1;
chassis-id 1;
mode active-standby;
status-control standby;
}
}
unit 1 {
encapsulation vlan-vpls;
vlan-id 601;
}
unit 2 {
encapsulation vlan-vpls;
vlan-id 602;
}
{master}[edit]
regress@auror#
{master}[edit]
regress@auror# show routing-instances
vpls1 {
instance-type vpls;
interface ae0.1;
interface ae0.2;

20. Liu Johnson Page 20 6/15/2012
route-distinguisher 100:3;
vrf-target target:100:1;
protocols {
vpls {
traceoptions {
file vpls_mclag.log size 20m world-readable;
flag all;
}
site-range 8;
tunnel-services {
primary vt-5/3/10;
}
site CE2 {
site-identifier 3;
interface ae0.1;
interface ae0.2;
}
}
}
}
{master}[edit]
regress@auror#
{master}[edit]
regress@auror# show protocols
rsvp {
interface all;
}
mpls {
interface all;
}
bgp {
local-address 10.255.49.138;
family inet {
unicast;
}
family l2vpn {
signaling;
}
group ibgp {
type internal;
neighbor 10.255.49.164;
neighbor 10.255.49.149;
}
}
ospf {
traffic-engineering;
area 0.0.0.0 {
interface lo0.0 {
passive;
}
interface all;
interface fxp0.0 {
disable;
}

21. Liu Johnson Page 21 6/15/2012
}
}
ldp {
traceoptions {
file ldp.log size 20m world-readable;
flag all;
}
interface all;
}
iccp {
local-ip-addr 10.100.1.2;
peer 10.100.1.1 {
redundancy-group-id-list [ 1 2 3 ];
liveness-detection {
minimum-interval 1000;
detection-time {
threshold 2000000;
}
single-hop;
}
}
traceoptions {
file iccp.log size 20m world-readable;
flag all;
}
}
lacp {
traceoptions {
file lacp.log size 20m world-readable;
flag all;
}
}
{master}[edit]
regress@auror#
CE1:
[edit]
regress@kitten# show interfaces ae0
vlan-tagging;
aggregated-ether-options {
lacp {
passive;
}
}
unit 1 {
vlan-id 601;
family inet {
address 10.49.1.1/30;
}
}
unit 2 {
vlan-id 602;

22. Liu Johnson Page 22 6/15/2012
family inet {
address 10.49.2.1/30;
}
}
[edit]
regress@kitten#
CE2:
[edit]
regress@beaver# show interfaces ae0
vlan-tagging;
aggregated-ether-options {
lacp {
passive;
}
}
unit 1 {
vlan-id 601;
family inet {
address 10.49.1.2/30;
}
}
unit 2 {
vlan-id 602;
family inet {
address 10.49.2.2/30;
}
}
[edit]
regress@beaver#