Author Archives: jgreig

So, I decided to start looking into what cards we had in production setups and investigate what sort of throughput we should be expecting on the line cards that we use within the Cisco 6500 and 7600 chassis’ that we run.

First off, the following is a general list of the card models that we tend to run in production along with a few other SFP modules that I won’t include here:

WS-X6148A-GE-TX
WS-X6548-GE-TX
WS-X6748-GE-TX

All the cards listed are 48 port line cards for the Cisco 6500/7600 series chassis’, however, there’s some big differences between the cards that you may not be aware of.

Chassis connection types
The backplane on the chassis allows all the cards to talk to one another and to talk to the supervisor engine. There’s a good article on wikipedia about this here

Classic :- 32Gb/s Shared Bus Half-Duplex
CEF256 :- 8Gb/s Full Duplex dedicated 'fabric' connection + shared bus connection
CEF720 :- 2x 20Gb/s Full Duplex dedicated 'fabric' connections + shared bus connection

There are also distributed versions that I won’t cover here.

Back to the cards

Going back to the cards themselves. Cisco don’t make it too easy in my opinion to find the details on the line cards throughput features, particularly from the product pages on the line cards themselves. So after a bit of research, below is a brief summary of the features of some of the cards that I’ve found:-

6148A
* 'classic' line cards and aren't fabric enabled
* 6 ASICs per card
* 8 ports per ASIC
* 1Gb/s per ASIC
* 32GB/s Half-Duplex connection to backplane

6548 
* 256 Fabric Enabled line cards
* 6 ASICs per card
* 8 ports per ASIC
* 1Gb/s per ASIC
* Dedicated full duplex 8Gb/s link between other cards

6748
* 720 Fabric Enabled line cards
* 4 ASICs per card
* 12 ports per ASIC (ports 1-12, 13-24, 25-36. 37-48)
* Each ASIC can do 10Gb/s
* Dedicated full duplex (40Gb/s) 2x 20GB/s links between other cards

The WS-X6148A-GE-TX line card and WS-X6548-GE-TX line card are pretty much the same card and are just as oversubscribed as one another, though, the WS-X6148A-GE-TX does have larger port buffers and the ability to set a higher MTU, this aside the WS-X6548-GE-TX benefits from having a dedicated Full-Duplex link to the backplane allowing it to shift the data quicker.

Conclusion

After the research I have done if you have fairly low bandwidth customers WS-X6548-GE-TX (cef256) line cards will be fine. However, if you’re providing pseudo wires, mpls etc and dealing with higher bandwidth customers cef720 cards such as the WS-X6748-GE-TX cards are more suitable.

One final useful command, if you want to see whether your cards are connected to the crossbar (Fabric) backplane or the classic bus, try this:-

core# show fabric switching-mode
Global switching mode is Compact
dCEF mode is not enforced for system to operate
Fabric module is not  required for system to operate
Modules are allowed to operate in bus mode
Truncated mode is allowed, due to presence of CEF720 module

Module Slot     Switching Mode
    1                 Crossbar
    2                 Crossbar
    3                 Crossbar
    4                 Crossbar
    5                     dCEF
    7                 Crossbar

One final quote from Cisco

“Connectivity Problem or Packet Loss with WS-X6548-GE-TX and WS-X6148-GE-TX Modules used in a Server Farm

When you use either the WS-X6548-GE-TX or WS-X6148-GE-TX modules, there is a possibility that individual port utilization can lead to connectivity problems or packet loss on the surrounding interfaces. Especially when you use EtherChannel and Remote Switched Port Analyzer (RSPAN) in these line cards, you can potentially see the slow response due to packet loss. These line cards are oversubscription cards that are designed to extend gigabit to the desktop and might not be ideal for server farm connectivity. On these modules there is a single 1-Gigabit Ethernet uplink from the port ASIC that supports eight ports. These cards share a 1 Mb buffer between a group of ports (1-8, 9-16, 17-24, 25-32, 33-40, and 41-48) since each block of eight ports is 8:1 oversubscribed.”

Any feedback, corrections, suggestions etc more than welcome.

References:-
Understanding Quality of Service on the Catalyst 6500 Switch
Release Notes for Cisco IOS 12.2SX
Release Notes for Cisco IOS 15.0SY
Cisco 6500 Architecture white paper