It’s here! It’s here! If you are reading this, then the Test Competition is finally here!
No, you don’t need to have registered in advance to play, just to win the big prizes and have your name in lights.
Either way, here’re the rules of play.
The Schedule
The formals part of the competition may be over, but we are keeping the rules out here as a sort of open-source test competition prep. So if you’d like to find some bugs, read on – we’ll keep a couple of prizes held back for late bug reports that no one else found.
Or, if you are reading this weeks later and it is “too late”, don’t worry, you can still follow the directions below to create an informal test competition with some friends. Be sure to have someone picked out as a judge in advance, or just have fun.
To make your own competition run by these rules, you’ll probably want about three hours of time.
That’s a Long Time!
Not really. You’ll spend the first few minutes reading this post, then interact with the “customer”, decide what to test, come up with a strategy, do actual testing, and file bugs. About an hour and a half into the exercise, you’ll need to start thinking about the formal test report that is due at the end of the contest, and how to make sure you have it turned in before the time runs out.
We suspect you’ll be pretty busy.
Websites to Test
In the three-hour period, you will test one or more of the following sites:
NBA.com
NRGGlobal.com (Login and pre-registration required. Sorry.)
QuickEasySurvey.com
CorkBoard.me
Just like real testing, it won’t be possible to test everything, so your team will have to choose which sites to invest how much time on and why. The final deliverables will be bugs (log any, at any time) and a test report, focused on one of these sites, designed to inform the ‘customer’ of the status of the site, suggested fixes, and their importance.
But wait! You ask. What is important? What do my customers care about?
Interacting With Customers
We have created a streaming YouTube site. For three hours, from 10AM Eastern to 1PM, Matt Heusser, our lead judge, will act as ‘customer’ on audio and video; you can ask questions using the comment/chat features of YouTube. Use this forum to ask and get answers to questions about priority/risk, consequences of failure, project details, and so on. We suggest that after reading this post your team spend a couple of minutes coming up with questions, then jump onto the stream to interact with the customer. The quality of that conversation is a part of the score!
Performance Testing
We are asking teams to load/performance test QuickEasySurvey.com; you can discuss performance expectations with the customer. The main goal is to find out how many simultaneous customers the site will support. (We expect the site to handle at least 25 simultaneous users). Your report should identify your findings, or how the site performs under increasingly load over time. The report should also include your method — what you did to test and why you believe that test was accurate, along with any potential bottlenecks, recommendations, major risks, and quick wins you are able to identify.
Deliverables
Teams will produce three major deliverables: Bug reports, a functional test project report and an optional performance test project report. Teams that skip performance testing will be severely handicapped, as it represents 50% of total score.
The bug reports are easy; each team should have directions on how to file bugs emailed to them in advance. If you would like to play but are not registered, you can create a list of bugs you found and email it, along with your team name, to SWTestCompetition@gmail.com with a subject of “Bug Reports for (Team_Name)”.
The content and format of that report is up to each team, but it should help the decision maker figure out if he is ready to ‘ship’ the product, if it needs more fixes, and what to invest in next, and so on. Ideally, the report would also include how you decided what to test, and how your team spent it’s time.
For example, say you spent 25 minutes on each site but decide to write a final report on QuickEasySurvey.com. Your report would start by describing the state of QuickEasySurvey, then might include why you felt it was the most important, then you might include some detail on your strategy and what you found on the other sites.
At the end of the functional competition you will email your report to SWTestCompetition@gmail.com with a subject of “Functional Test Report for (Team_Name)”, and you’ll do the same for performance testing.
The Judging
The total score possible for the entire competition is 100 points; fifty for functional testing, and fifty for performance testing. Functional test categories include importance of Bugs Filed, The quality of bug reports, how reproducible those reports are, the quality of the test report, it’s accuracy, and the quality and accuracy of the functional/load test report. A small amount of bonus points are available for how the teams interact with the judges, and teamwork if visible.
You
Should now have everything you need to get out there and test! If not, leave a comment.
See you in YouTube, in the comments, on the email.
But wherever we see you, we hope to see you … testing.
If you can’t play, come back next week for the scores, awards, and after-action reports; we think it’s going to be quite a show.