Exploratory testing, Session based testing, Scripted testing…concertedly

In my last assignment a mix of exploratory, session based and scripted testing helped won an award, appreciation by managers & stake holders at our organization. The quotes from our award certificate

QA team has come up with some intelligent testing techniques to validate the product. The customer test team was not able to find even a single critical bug after delivery

Though the award was a small one in terms of the award categories our organization have, it meant a lot to me and the team, because this was the first time the team used exploratory approach. The assignment was not pure exploratory or session based, in fact it was supposed to be scripted, but a mix of all three exploratory, session based and scripted testing helped us learn, enjoy, and meet our mission.

The assignment started off with developing test cases. We wrote test cases based on the initial requirement document provided. We did come up with a large number of test cases, of which most were re-written, some ended obsolete, deferred, because of changes in software or hardware or firmware or incorrect requirement or technical limitations or schedule limitations or etc. The test cases as part of the process were then reviewed by the stakeholders and were signed off.

The releases started flowing from the development team to the testing team. In the first 2 releases we executed the test cases found some bugs and were quite happy. After the second build release for testing, the bugs started decreasing and we felt bored to execute the same tests. To add to the misery, our hardware boards got limited and we were 5 testers with 1 board to test on.

This to me was the right time to get in exploratory and session based testing.
• I divided the team into 2 with 2 testers in each team,
• Session time was set to 1.5 hrs per session, mission for the session was provided before the session would start.
• After one team completes the exploratory session, the notes, bugs, issues, data used, tasks performed would be discussed with me, while the other team sets out to meet the mission.
Initially I did observe couple of testers finding it hard to meet the destination (mission) without a GPS (test cases), but observed 2 other testers enjoying their testing, switched the pairs in the team, and the bugs started flowing. This session based testing allowed everyone to be occupied with testing, enjoy testing and also helped me keep track on all the proceedings.

Though exploratory was much more fun and the rate at which bugs were found was huge, after some releases, when we had no new features left to be added to the software the bugs reported by team started dropping, when we sat together, I felt the ideas running out in the team – we initially focused on exploring the system with functional and domain related missions. After the discussion we changed the focus from functional & domain to Flow tests - explained the team on what to be targeted in flow tests and what could be the observations to note running flow tests – and what a change, the energy was back, they liked testing the same module again without any new changes in it.

From then on test case execution was approached as an activity to provide only test case execution report, but there were times wherein a test case would remind us of a requirement we missed in our exploratory, a test scenario in one of the test case we missed to run when we followed only exploratory approach. I feel the reason why we could have missed out a requirement or a scenario, while exploring could be because the missions were not planned as carefully as they should have been. Well, a lesson for the next phase.

We are thru with the release, and are pretty happy on meeting our mission.

Hope this little summary inspires other teams to have a look at exploratory, and how you could fit it in your context.