In order to enhance security, improve redundancy, and facilitate upgrades for the Toronto office we will need to conduct a downtime at the Mountain View office that will take our test machines offline, requiring us to close all development trees. This will take place on Sunday, September 19th, and is expected to last for up to 7 hours, from 10am PT until 5pm PT, though if things go well we might be done much sooner.
We will notify firstname.lastname@example.org of our progress as necessary during the day, and keep Tinderbox up to date with the tree status; this is our planned schedule (all times PDT):
1000 – Close all trees on Tinderbox
1015 – Release Engineering confirms systems are ready to go offline
1015 – Phase I: upgrade firewall, and wired switches at Mountain View
At this point all the infrastructure in the Mountain View office (performance and unit test infrastructure, as well as some builders) will lose their connection to the machines that report their progress. The trees will appear to be broken if people look on Tinderbox.
1230 – Phase I Complete
1230 – Release Engineering will prepare systems to come back online
1230 – Phase II: upgrade office wireless infrastructure
Infrastructure connectivity is expected to be restored within 1.5 hours, and the Release Engineering group will be on hand to bring the systems back up. They will wait for a full “green run” before re-opening the trees. This normally takes 2-3 hours.
1400 – Phase II Complete
1400 – Release Engineering will begin to bring remaining systems online
1700 – Release Engineering will re-open all trees
Feel free to comment directly if you see issues past the planned downtime.