2011-09-27 Wednesday: LACP and PAGP

I'm doing some research today on a batch performance issue that recently came up for a client engagement.


Initial research seems to indicate it may be something related to a recent network infrastructure change.  As part of that discussion, the configuration options for LACP and PAGP were mentioned.  Those configuration options were not previously familiar to me - so I spent a quick 15 minutes doing some research and found these resources to be of some interest:


Some background info on LACP and PAGP

http://en.wikipedia.org/wiki/EtherChannel

http://en.wikipedia.org/wiki/PAgP

http://en.wikipedia.org/wiki/Link_Aggregation_Control_Protocol

http://en.wikipedia.org/wiki/Port_Aggregation_Protocol

VMware Virtual Networking Concepts

http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf

Link Aggregation Control Protocol (LACP) (802.3ad) for Gigabit Interfaces

http://www.cisco.com/en/US/docs/ios/12_2sb/feature/guide/gigeth.html

For Catalyst 2950 and Catalyst 2955 Switch

Configuring EtherChannels

http://www.cisco.com/en/US/docs/switches/lan/catalyst2950/software/release/12.1_22_ea2/configuration/guide/swethchl.html

“Interfaces configured in the on mode do not exchange PAgP or LACP packets.”

Table 30-1 EtherChannel Modes
Mode	Description
active	Places an interface into an active negotiating state, in which the interface starts negotiations with other interfaces by sending LACP packets.
auto	Places an interface into a passive negotiating state, in which the interface responds to PAgP packets it receives but does not start PAgP packet negotiation. This setting minimizes the transmission of PAgP packets.
desirable	Places an interface into an active negotiating state, in which the interface starts negotiations with other interfaces by sending PAgP packets.
on	Forces the interface into an EtherChannel without PAgP or LACP. With the on mode, a usable EtherChannel exists only when an interface group in the on mode is connected to another interface group in the on mode.
passive	Places an interface into a passive negotiating state, in which the interface responds to LACP packets that it receives, but does not start LACP packet negotiation. This setting minimizes the transmission of LACP packets

“Interfaces can form an EtherChannel when they are in different PAgP modes as long as the modes are compatible”

Default EtherChannel Configuration

Table 30-2 shows the default EtherChannel configuration.

Table 30-2 Default EtherChannel Configuration
Feature	Default Setting
Channel groups	None assigned.
PAgP mode	No default.
PAgP learn method	Aggregate-port learning on all interfaces.
PAgP priority	128 on all interfaces. (Changing this value has no effect.)
LACP learn method	Aggregate-port learning on all interfaces.
LACP priority	32768 on all interfaces.
Load balancing	Load distribution on the switch is based on the source-MAC address of the incoming packet.

EtherChannel Configuration Guidelines

http://www.cisco.com/en/US/docs/switches/lan/catalyst2950/software/release/12.1_22_ea2/configuration/guide/swethchl.html#wp1021856

“Do not configure a GigaStack GBIC port as part of an EtherChannel.”

“Configure all interfaces in an EtherChannel to operate at the same speeds and duplex modes.”

“Do not configure a secure port as part of an EtherChannel.”

http://www.linkedin.com/answers/technology/information-technology/computer-networking/TCH_ITS_CNW/255109-12236013

PAgP is cisco proprietary which has 2 implications:
1-Interoperability with other network gear from different vendors is not guaranteed.
2-More importantly, your vmware servers may not be able to leverage the etherchannel

“Cisco's recommandations for VMWare connection is to use the mode "on", not LACP or PAGP”

“ESX does not support PAgP or LACP, you need to set Etherchannel to NONEGOTIATE”

http://www.networking-forum.com/viewtopic.php?t=22447

“I need to get in the habit of using LACP. I've been using "on" exclusively and it's bitten me a couple times.”

“Channeling unconditionally can cause a loop if the other side isn't configured.”

http://www.networks-wetworks.com/2010/06/etherchannel-quick-and-dirty.html

For ports to be members of the same EtherChannel, there are some restrictions. Ports must:

• Belong to the same VLAN
• Have identical STP settings
• Have identical speed/duplex settings
• Note: In addition, if the EtherChannel is to be used as a trunking interface, all ports must be in trunking mode, have the same native VLAN, and pass the same set of VLANs.

EtherChannel Negotiation: PAgP vs. LACP

There are two EtherChannel negotiation protocols.  Port Aggregation Protocol (PAgP) is a Cisco-proprietary protocol, while Link Aggregation Control Protocol (LACP) is standards based.

PAgP dynamically modifies the EtherChannel if one of the ports’ VLAN, speed, etc. is changed so that all of the links in the EtherChannel match. PAgP can be configured in active mode (desirable), which actively attempts negotiation.  Passive mode (auto, the default) only negotiates an EtherChannel if the far end initiates it.

LACP assigns roles to end points.  The switch with the lowest system priority makes decisions about what ports will participate in the EtherChannel at any given time.  If you’re familiar with STP, this is similar to the way the Root Bridge is elected.  Ports are selected and become active in the EtherChannel according to their port priority. LACP Active mode (active) – actively negotiates, while passive mode (passive) negotiates only if the far end initiates it.

Etherchannel Performance Issues with….

https://supportforums.cisco.com/thread/2067911

http://pktmaniac.info/2011/06/design-for-failure-it-helps/

“Recently there was a post on Linkedin about port aggregation and the use of negotiation. It’s really disturbing seeing how many people recommend not using LACP or PaGP to configure aggregated ports (port-channels|etherchannels). Seems to be common misconception that if two switches are back-to-back configuring an on mode is acceptable.”