Thoughts on working at Mozilla and the Firefox release process
Some thoughts on my life at Mozilla as I head into our company-wide work week here in Portland! My first year at Mozilla I spent managing the huge volume of bugs, updating docs on how to triage incoming bugs and helping out with Bugzilla itself. For my second year I’ve been more closely tied into the Firefox release process, and I switched from being on the A-Team (Automation and Tools) to desktop Firefox QA, following Firefox 31 and then Firefox 34 through their beginning to release.
I also spent countless hours fooling around with Socorro, filing bugs on the highest volume crash signatures for Firefox, and then updating the bugs and verifying fixes, especially for startup crashes or crashes associated with “my” releases.
Over this process I’ve really enjoyed working with everyone I’ve met online and in person. The constant change of Mozilla environments and the somewhat anarchic processes are completely fascinating. Though sometimes unnerving. I have spent these two years becoming more of a generalist. I have to talk with end users, developers on every Mozilla engineering team, project or product managers if they exist, other QA teams, my own Firefox/Desktop QA team, the people maintaining all the tools that all of us use, release management, and release engineering. Much of the actual QA has been done by our Romanian team so I have coordinated a lot with them and hope to meet them all some day. When I need to dig into a bug or feature and figure out how it works, it means poking around in documentation or in the code or talking with people, then documenting whatever it is.
This is a job (and a workplace) suited for people who can cope with rapid change and shifting ground. Without a very specific focus, expertise is difficult. People complain a lot about this. But I kind of like it! And I’m constantly impressed by how well our processes work and what we produce. There are specific people who I think of as total rock stars of knowing things. Long time experts like bz or dbaron. Or dmajor who does amazing crash investigations. I am in their secret fan club. If there were bugzilla “likes” I would be liking up a storm on there! It’s really amazing how well the various engineering teams work together. As we scramble around to improve things and as we nitpick I want to keep that firmly in mind. And, as we are all a bit burned out from dealing with the issues from the 33 Firefox release and many point releases, then the 10th anniversary release, then last minute scrambling to incorporate surprise/secret changes to Firefox 34 because of our new deal with Yahoo.
I was in a position to see some of the hard work of the execs, product and design people, and engineers as well as going through some rounds of iterating very quickly with them these last 2 weeks (and seeing gavin and lmandel keep track of that rapid pace) Then seeing just part of what the release team needed to do, and knowing that from their perspective they also could see what IT and infrastructure teams had to do in response. So many ripples of teamwork and people thinking things through, discussing, building an ever-changing consensus reality.
I find that for every moment I feel low for not knowing something, or making a mistake (in public no less) there are more moments when I know something no one else knows in a particular context or am able to add something productive because I’m bringing a generalist perspective that includes my years of background as a developer.
As an author and editor I am reminded of what it takes to edit an anthology. My usual role is as a general editor with a vision: putting out a call, inviting people to contribute, working back and forth with authors, tracking all the things necessary (quite a lot to track, more than you’d imagine!) and shifting back and forth from the details of different versions of a story or author bios, to how the different pieces of the book fit together. And in that equation you also, if you’re lucky, have a brilliant and meticulous copy editor. At Aqueduct Press I worked twice with Kath Wilham. I would have gone 10 rounds of nitpick and Kath would still find things wrong. My personal feeling for my past year’s work is that my job as test plan lead on a release has been 75% “editor” and 25% copy editor.
If I had a choice I would always go with the editor and “glue” work rather than the final gatekeeper work. That last moment of signing off on any of the stages of a release freak me out. I’m not enough of a nitpicker and in fact, have zero background in QA — instead, 20 years as a lone wolf (or nearly so) developer, a noisy, somewhat half-assed one who has never *had* QA to work with much less *doing* QA. It’s been extremely interesting, and it has also been part of my goal for working in big teams. The other thing I note from this stressful last 6 weeks is that, consistent with other situations for me, in a temporary intense “crisis” mode my skills shine out. Like with the shifting-every-30-minutes disaster relief landscape, where I had great tactical and logistical skills and ended up as a good leader. My problem is that I can’t sustain that level of awareness, productivity, or activeness, both physically and mentally, for all that long. I go till I drop, crash & burn. The trick is knowing that’s going to happen, communicating it beforehand, and having other people to back you up. The real trick which I hope to improve at is knowing my limits and not crashing and burning at all. On the other hand, I like knowing that in a hard situation I have this ability to tap into. It just isn’t something I can expect to do all the time, and isn’t sustainable. One thing I miss about my “old” life is building actual tools. These last 2 years have been times of building human and institutional infrastructure for me much more than making something more obvious and concrete. I would like to write another book and to build a useful open source tool of some sort. Either for Mozilla, for an anti-harassment tool suite, or for Ingress…. 🙂 Alternatively I have a long term idea about open source hardware project for mobility scooters and powerchairs. That may never happen, so at the least, I’m resolving to write up my outline of what should happen, in case someone else has the energy and time to do it.
In 2013 and 2014 I mentored three interns for the GNOME-OPW project at Mozilla. Thanks so much to Tiziana Selitto, Maja Frydrychowicz, and Francesca Ciceri for being awesome to work with! As well as the entire GNOME-OPW team at Mozilla and beyond. I spoke about the OPW projects this summer at Open Source Bridge and look forward to more work as a mentor and guide in the future.
Meanwhile! I started a nonprofit in my “spare time”! It’s Double Union, a feminist, women-only hacker and maker space in San Francisco and it has around 150 members. I’m so proud of everything that DU has become. And I continue my work on the advisory boards of two other feminist organizations, Ada Initiative and GimpGirl as well as work in the backchannels of Geekfeminism.org. I wrote several articles for Model View Culture this year and advised WisCon on security issues and threat modeling. I read hundreds of books, mostly science fiction, fantasy, and history. I followed the awesome work my partner Danny does with EFF, as well. These things are not just an important part of my life, they also make my work at Mozilla more valuable because I am bringing perspectives from these communities to the table in all my work.
The other day I thought of another analogy for my last year’s work at Mozilla that made me laugh pretty hard. I feel personally like the messy “glue” Perl scripts it used to be my job to write to connect tools and data. Part of that coding landscape was because we didn’t have very good practices or design patterns but part of it I see as inevitable in our field. We need human judgement and routing and communication to make complicated systems work, as well as good processes.
I think for 2015 I will be working more closely with the e10s team as well as keeping on with crash analysis and keeping an eye on the release train.
Mad respect and appreciation to everyone at Mozilla!!
