Cisco wireless problems, multicast failures

3

This post is written entirely out of frustration.  For what seems like months I’ve been on-and-off troubleshooting wireless connectivity issues with Cisco.

I’ll give a little background first.


At Mozilla’s main campus I’m using a Cisco 3845 ISR with two NM-WLC Wireless LAN controllers and have a total of 9 APs covering two buildings.

I broadcast two SSIDs – a guest one and a WPA/WPA2 Enterprise one.  Both wireless networks are bridged through the ISR onto the appropriate wired network through a BVI.

Problem #1

My first issue was mostly around client authentication.  Mozilla has a heavy percentage of Mac users and most had some sort of issue authenticating.  This problem became worse when the MacBook Airs came out and with some of the new gen MacBook Pros.  None of the Airs could authentication and a large number of the Pros started failing.  And not a single iPhone could authenticate.

Cisco’s default response was to:

  1. Update my wireless drivers on OSX
  2. Update the firmware on the WLC

#1 is impossible, #2 I did and no fix.  Finally after a month of pushing and two days of bringing in Aruba gear to prove to Cisco it wasn’t an OSX issue, Cisco found a solution.  The default EAP timeout was set to one second with a one second retry.  You had one second to type your password correctly and you had one chance to retry it.  Changing both of those to something more reasonable resolved most of the issues for Airs, Pros and iPhones.

(I don’t believe this was well documented – it’s not exposed through the webui WLC interface either and took TAC a long time to come up with this recommendation. Look for config advanced eap identity-request-timeout & config advanced eap identity-request-retries.)

Problem #2

The second problem is more involved and has been a problem since day one but hasn’t really been end-user affecting.  Most users will notice that wired users can not see wireless users’ iTunes libraries (and visa versa).

That’s just a symptom of the problem. Anything that relies on mDNS/Bonjour fails to work between wired and wireless users, including finding network-based Time Machine servers.

This manifested itself again when certain users couldn’t sync their Things content with their iPhone.  In troubleshooting, we (Justin) noticed that it used multicast to try to find devices to sync with.

I’ve narrowed down the problem to the following:

  1. multicast traffic is not forwarded intra-WLC or inter-WLC
  2. mulitcast traffic is not bridged out the BVI

From a wired host I ran:

tcpdump -n ip multicast and ether host 00:17:f2:09:d8:ea

and am unable to see any multicast data from my wireless host (it’s entirely possible that I don’t understand mDNS or how to use tcpdump well enough to troubleshoot this either).  As best as I can tell, the WLC is configured to process multicast:

(BS-WLC01) >show network sum

RF-Network Name............................. mozilla
Web Mode.................................... Disable
Secure Web Mode............................. Enable
Secure Web Mode Cipher-Option High.......... Disable
Secure Shell (ssh).......................... Enable
Telnet...................................... Disable
Ethernet Multicast Mode..................... Enable   Mode: Mcast  239.0.1.2
Ethernet Broadcast Mode..................... Enable

Cisco appears to have no clue on this either. The last response from TAC on this was:

I checked our query and found no response as of this time. I researched and found no similar devices in combination related to the matter. Be assured that I will make necessary follow-up and will provide you an update as soon as I receive a reply.

This worked without problems when I had that Aruba hardware for a couple days so I know this is not an OSX client issue – I wasable to stream from my iTunes library on my MacBook (wireless, on Aruba) to my wired desktop.

Cisco, why is this so hard to get working?!

Categories: Mozilla, Networking

3 responses

  1. hansen wrote on :

    So why do you use Cisco?

    Why not play with some of the WRT firmwares or setup a FreeBSD box with antennas?

  2. docca wrote on :

    This is typical of Cisco. I’ve had similar support experiences with a large customer. All that their TAC would do was basically check their knowledgebase, and say something like “nope, we do not know what it is, but we’ll keep working on it”. Not even a sniffer on the network was taken, just logs from their client software. And then they stop talking to you after a few weeks…

  3. Justin wrote on :

    I know this is an old post but…

    I had similar problem to your #1 with old hardware. I basically had to disable a bunch of default settings and then make changes through the CLI to complete the changes. Emailing you a document TAC actually sent to me.

    On #2, APs and WLCs have to be configured for multicast. The best link I can find on it is http://www.cisco.com/en/US/tech/tk722/tk809/technologies_configuration_example09186a00807cc10d.shtml

    Which is pretty much just a sample config and I’m not sure what your setup is.