It’s been awhile since the last Fennec QA Update by Joel, so we felt that now was as good a time as any to provide another update. This is especially true as we’re getting closer to a final release with the Fennec 1.0 Beta 5 out the door now. Team MOQA has been really busy making Fennec the best mobile browser it can be over the past few months. With all the effort we’ve put in for quality execution on manual and automated testing, we knew the project was getting somewhere. But we had no idea how far until we started playing around with Bugzilla’s report charts tool. Here’s what we found:
Basically, we literally and figuratively went crazy in August and September with the number of bugs verified, but it opened up a whole bunch of usability issues in the project that started to bring the quality of the project around in October. The number of bugs fixed per week in the project before August was 7-8, but since August its gone up to 37-38. Now, this can be attributed to a whole bunch of things, but at the end of the day a person has to ask themselves if the overall quality of the project they’re working on has gotten better through their hard work. I think its safe to say that such a huge jump in bugs in a fixed state was attributed to developers having a larger number of bugs to work on that could be fixed…and that’s something we can hang our hats on.
With that said, we’re not done yet. Team MOQA has a couple more things up our sleeves that will really shore up some of the loose ends relating to quality and they’re coming hard and fast. So be ready for some hawtness with your Mozilla-powered mobile browsing in the near-future.
Things to Look For:
- WinMo Talos up and running soon
- Developing an extension to developer browser-chrome tests
Things Done:
- A robust system to move test and performance automation to any new platforms that crop up in the future (oh, and they will on the mobile front).
- xpcshell unit tests up and running
- We now have Release Test Tracking Pages for every release
- A String Guide (It’s a subgroup within the testrun) for localizers to find Fennec UI elements that correspond with the strings they localize in .dtd and .properties files within the mobile-browser source code.
Raw Stats (By Team MOQA since Joel Maher’s last Fennec QA Update on 6/30/09):
- 1092 bugs verified
- 276 bugs filed
- 64 Bugs filed in Testdays
That’s an impressive chart, especially the 90 degree angle right around 8-23. Nice job guys, keeping the verification count in preparation for 1.0. I’m curious, has any of these been reopened because of regressions found? That would a good stat to track, and also the severity of them. Also, were any security bugs included in this numbers?
Congratulations and bravo to all the efforts and tremendous undertakings over the past few months for all of MoQA. With a 1.0 release on the horizon, it’s clear how far it has come. Hopefully testing will be easier post 1.0 with all the goodies and tools in parallel development :) Keep up the good work MoQA
-Aaron
Good stuff, the chart only works for people with a Bugzilla cookie with “editbugs” privileges. Maybe next time it’s better to include a static screenshot… :-/
Thanks for the information, It’s changed now.
Actually, the number of re-opened bugs has increased dramatically as well. Check it out: https://bugzilla.mozilla.org/reports.cgi?product=Fennec&datasets=REOPENED%3A
As for security bugs, there’s a dearth of them as we have higher priority bugs coming in for functionality, performance and plugins before heading to RC.
Aye, we have some goodies and tools that Clint’s team is working on or at least thinking about. If you have any ideas, please feel free to give me a heads up on them as I’m planning on quality needs for 2010.