mrz's noise

noise from a mozilla IT/Operations wrangler

Archive for Networking

World IPv6 Day, Chickens, Eggs & Mozilla

Mozilla has a long history of being a technology leader on the web and open standards. A short, and certainly not exhaustive, list includes: Firefox (of course) HTML5 Geolocation Do Not Track IT/Operations is often the unglamorous side of Mozilla. We work in the shadows and you generally don’t see us unless something’s broken. However,… Read more

Tags:

Categories: Networking

Phoenix to San Jose, in 18ms

[root@ip-ns01 ~]# mtr www.mozilla.com –report ip-ns01.phx.mozilla.org Snt: 10 Loss% Last Avg Best Wrst StDev 10.8.75.1 0.0% 0.4 0.4 0.4 0.4 0.0 v500.core1.phx.mozilla.net 0.0% 1.1 1.3 1.0 3.1 0.6 xe-1-1-0.border1.phx.mozilla.net 0.0% 0.7 0.7 0.7 0.8 0.0 64.124.201.177 0.0% 1.1 1.1 1.1 1.1 0.0 ge-0-3-0.mpr3.lax9.us.above.net 0.0% 9.4 13.0 9.4 44.3 11.0 xe-0-1-0.er1.lax9.us.above.net 0.0% 9.5 13.7 9.4 51.4… Read more

Tags: ,

Categories: Mozilla, Networking

Geolocation & Zeus ZXTM 6.0

Mozilla’s North American store has been down in maintenance mode (you can read about the why) but not the International Store. The old store used to redirect non-North American users to the International Store, but unfortunately that redirect wasn’t carried over when we took the store offline. I was inspired Sunday night to put together… Read more

Tags: ,

Categories: load balancing, Mozilla

More traffic analysis with NetFlow

I’ve been working on a couple capacity planning projects and have been knee deep in bandwidth metrics. That, combined with turning up Level3 yesterday got me looking more into where that bandwidth is coming from (and Reed asked). The first chart shows a breakdown by protocol. Shouldn’t be any surprise that 82% of Mozilla’s traffic… Read more

Tags: ,

Categories: Mozilla, Networking

Level3, post BGP turn up

Derek turned up BGP peering with Level3 this morning out of San Jose (this was supposed to be part of last night’s maintenance but he ran into turn up problems on Level3’s side). We’ve grown (bandwidth-wise) to the extent that having two transit providers wasn’t sufficient. Should either fail, I wouldn’t have felt comfortable pushing… Read more

Categories: Mozilla, Networking

Switch IOS Upgrade Post-Mortem

Last night Derek attempted to upgrade IOS on core1 & core2 to pickup software support for Cisco’s ACE module. We ran into several issues and postponed the upgrade on core2 until those issues are resolved. The switch’s Compact Flash cards weren’t formatted in a format that rommon (the low level boot loader) could read and… Read more

Categories: Mozilla, Networking

Uptime

I’m fairly conservative when it comes to upgrading switches. I generally only upgrade to pick up security fixes. It’s rare that I’ll upgrade just to stay current. Tonight’s one of those rare exceptions. Derek is upgrading both core1 and core2 to pickup software support for Cisco’s ACE module. We’re planning on using this in conjunction… Read more

Categories: Mozilla, Networking

Load Balancer performance issues, fxfeeds.mozilla.org & versioncheck

I mentioned briefly in Monday’s meeting the performance issues we’re having with our load balancers.  Since then, we’ve been hustling to turn up something in the short term to handle Thursday’s Major Update and Firefox 3.0.5/2.0.0.19 release (see here). For a number of months we’ve been looking at Zeus and their ZXTM product.  It has… Read more

Tags: , ,

Categories: load balancing, Mozilla