Later I started thinking more on this license plate heuristic and I chalked out points, why this worked so well at that point.
1. Time: The time I chose to follow the bike with similar license plate was around 12:00am; most of us might be heading home at that time and seldom to other locations.
2. Rider’s gear: She had a helmet, backpack and an jacket on, no gloves, no saddle bags, no gear of her made me think she could be on a long ride, in fact her appearance looked like she was returning home from her office.
3. The bike: She rode a bike which delivered similar power to that of my bike, so following the bike was much easier.
Now, imagine some one using the license plate heuristic at 9AM, when most of us might be heading to office or college, etc. or imagine following a biker who has his touring gear on, or imagine following a bike which is more powerful or slower, when compared to the bike you ride. The probability you might not reach your destination will be high. Or why would one even use this heuristic in the day light when you could ask some one the route (people who speak a different language and those who follow best particle might still use it :)).
So, I feel using heuristics are good but deciding which heuristic to use when is very essential. In other words it again highlights the importance of context in our day to day problem solving or be it in the field of testing.
Let me ask you this: Which training school teaches how to respond to such situations?
If it wasn't taught anywhere how did we learn?
There are a number of ways we learnt it and I am trying to list them and draw parallels to testing of why most of us fail to self teach testing?
Get curious about it: I first got curious to learn about it because I wanted to do it. When I read the biking portals and their travelogues, met bikers, heard them share their experiences about their biking trip, it made us imagine the challenges faced, the physical strength required, the concentration to keep reflexes active and most importantly have lots of fun, if we were to do it, too. This kicked some juices to flow within us and made us quiz and question more about the itineraries, ride, etc.
If someone describes their experience of testing software, that makes us think we too should have that kind of an experience, the juices have started to flow.
Listen to people: A most part of what we did when we asked them questions was to listen to them very carefully. We didn't say, "Hey, I went on a ride there and don't think it works that way".
When someone is sharing their testing story with us and if we start concluding the story before they do, we aren't anywhere close to listening.
Prepare: We try making a checklist of things that we need to have or carry like riding gear, equipment, clothing & accessories, first aid kit, etc, preparing our motorcycle, preparing ourselves before we start off a bike riding trip to some place. We ask around the experienced ones and use their ideas + form our own ideas of what we might need. For instance, riding gear with thermal liner might not be needed in certain regions, based on the weather conditions but when we are off to a place like the mighty Himalayas, we definitely need it. We try wearing the stuff that we bought to check how we feel about it or to see if we need to exchange. We may buy maps, search over internet a 100 times +, meet riders who have traveled before to know more about the place. We are happy to know the same information from several sources but never happy to say, "I know it and I don't need to see it"
Preparing to test is essential. Sometimes our testing looks sloppy because our preparation was bad. We probably didn't have a setup close to the customer environment or we don't know about it at all. Learning new things could be a part of our preparation.
Mistakes and Lessons: The human curiosity sometimes makes a biker not listen to what the guide says or try new things that might not be safe. In a rare case it could be fatal but mostly the biker has some bruises or end up choking the bike for a while. After the trip is over and the wounds are healing, the biker thinks over the mistake and learns to be extra cautious. Even if the biker followed the guide there is no assurance that the biker won’t make a mistake.
If we make mistakes, we must find special joy to learn from it and own our mistakes. But rarely with test case execution someone else is the owner and not me.
Please share best practices for testing team?, testing centralized team?, knowledge sharing for testers?, etc – how many times have we all heard about this or have seen this as a discussion topic in various forums.
I wonder isn’t it a best practice by best practicing testers to share their best practice on Google?
Or should someone write a paper about “Best practices to search in Google for best practices in testing”?
Couple of weeks back, I was invited to provide a career talk on software testing at Don Bosco Institute of technology engineering college in Bangalore. I was sceptical initially since I had never talked in a college and I was not sure if I could even deliver it, but it sounded like a good challenge and so I accepted the invitation.
Friday was the day when I had to deliver the talk and I was occupied the whole week, Thursday I some how managed to leave early to home and was at home by 7:30 pm. luckily my wife was stuck at office and I had the whole house to shout and prepare for the next day session. When I sat down in front of my laptop I did not know where to start, the challenges in front of me were I could not use technical jargons, I cannot assume that they know testing, I cannot assume that they know software development process, what do I explain to them in software testing? How will I build rapport with the audience? Shall I crack some jokes? How will I face the professors, who I was sure would be pretty aged and experienced? Etc,
Loads of questions and I was looking for answers. I thought I shall reach out for Pradeep Soundararajan, my mentor, but decided against it since I felt I wanted to crack this challenge myself I am sure even Pradeep would be much happier to know how I cracked the challenge rather than seeking help from him.
I initially thought of developing slides to assist me in the presentation, but later ruled it out, and planned to prepare a checklist just for my mind. I did prepare a presentation of one slide with my blog address in it.
Friday, sharp 3:30 pm I was at Don Bosco campus and was thrilled to see my posters flashed on the compound walls, well, it made me nervous as well, the expectation I had set for myself raised higher. The overwhelming reception I got on the way to the seminar hall made me more nervous and the professors around me gave me a sort of look, may be because I am much younger.
I stood on the stage and saw more than 100 students looking at me, most of the stuff I planned and had made a checklist of looked like loosing its way. My first words to the audience “Hi All, this is the first time I am addressing such a big gathering, so my legs might shiver or I might miss/loose some words, but I might also get better as the time goes on”. Then I looked at the checklist, I felt I have to first engage the audience and thought the checklist I prepared the previous night might not work here in the same sequence.
My checklist asked me to talk about some of the History’s worst software bugs,but on the stage I did not feel like starting with it. I could hear noise from the back seats, and so I asked one of the students sitting at the back what his name was? He replied to my question nervously may be because everyone else were looking at him. I then came up with a scenario wherein Raju (name changed) had saved his pocket money for over 6 months to gift a mobile phone to his girl friend for valentines, now I felt I had more people listening to me. I went on “raju, the day before valentines walks to the store selects the mobile which he had done a lot of research on and gets it gift wrapped. On the 14th of Feb he gifts it to his girl, the girl so excited to see a big box, calls her friends over before she unwraps it, she unwraps the box, looks at the mobile and feels top of the world. She removes the mobile phone from the box, try powering it ON and what she hears a nasty noise which almost deafened her and her friend’s ears. – I asked raju how would you feel and I asked the audience how would they feel and this set the tone for my talk.
I was no longer nervous, I then put the same bug in a life critical context and then went on talking about the History’s worst software bugs and in turn justify why we need software testing.
I felt more confident with each minute rolling by and I felt I had more audience listening to me as the talk went on.
At the end of the session I was pretty happy that I could deliver a talk to 100+ students for approx 45 min, but what pleased me more was that there were questions from the audience both the professors and the students.
I liked questions like
- How much does a software tester earn?
- How can I become a software tester?
- Is only software tested?
But one question was fantastic
- How can you ensure that there will be no bugs after you test a product?
This experience taught me a lot of lessons.
- The first one is never hesitate to take on a new challenge; you might surprise yourself.
- Research and homework before a presentation is required, but one should be prepared to alter/modify the flow/content during the presentation if required.
- You feel confident on stage when you sense more audience is listening to you, so it’s very important to engage the audience.
- No slides for the presentation actually helped me, but I found the checklist very useful.
The mission was simple hunt for bugs, and so did we. The session was fantastic; bugs started flowing from the first minute, and all three of us had loads of fun. The test techniques varied individually and the discussions we had during the test session been great.
Once we were thru with the session, Manoj and me left and Ajay did almost most of the post production. So, it would be nice to read the full report from his blog @ http://enjoytesting.blogspot.com/2009/08/trio-testing-at-distance-part-1.html
So, next time if you are not feeling sleepy or feel like practicing your test skills with other testers or feel like learning some thing new or feel like sharing your test ideas, or just curious to know what this is all about. Drop a mail to firstname.lastname@example.org
Thanks Manoj and Ajay for helping me sleep. (This statement is a bit controversial :)
Bangalore Weekend Testers: Fun, Learn & Contribute - Pradeep Soundararajan
BWT - Do you wanna Rock? - Parimala Shankaraiah
Are you Ready? Join : Bangalore Weekend Testers - Ajay Balamurugadas
Here comes BWT – Learn, Test and have Fun - M.V.Manoj
My small attempt to faction all reports
Pradeep Soundararajan's report
Santhosh S Tuppad’s report
Shikhar K Singh’s report
Rahul Mirakhur’s report
Enjoy reading these fantastic reports, and hope to see more of you in the next BWST.
The most wonderful thing about context driven testing community I have observed is they are just an email away from you. They love sharing their testing experiences, they love sharing their secrets, and most importantly they love to see a better test community.
“My blog List” lists some of those testers who I respect a lot and have learnt from. The list is nowhere complete, and has been growing the day I started it.
Thank you all.
I have a game in my cell phone called “Krish Cricket PRO Challenge” in the game cricket greats like even Lara, Gilchrist bat right handed – is this is a bug?. Well, I am confused because though they are right handed the game is a lot of fun to play and I do not mind them batting right handed or in other words it does not bug me. So is this a bug?
The game came bundled with other applications when I bought the cell phone. Initially I felt very odd to see Lara play right handed, and I cursed the team who built this game. Could the code be simple if all batsmen in the game are right handed? May be the team who built the game did not invest enough in the software since it would be bundled free? But, what if I had to buy the game and in the demo Lara bats right handed, I would have never bought it. So is this a bug?
Lara batting right handed definitely does threaten the value of the product and so I feel this definition by James Bach and Michael Bolton “A bug is anything about the product that threatens its value” (I came across this definition in Bug Advocacy Slides from BBST) suits this context. So my next question to you all is – are definitions contextual? Or have I have failed in understanding the first definition by James & Michael?
I glanced thru the mission statement, test scenarios, bugs logged, testers own analysis/review of the assignment, the answers to the difficult questions like
• What did I do well?
• What did I not do well?
• What wasn’t I able to do?
• Why wasn’t I able to do whatever I couldn’t do?
• How much time did I pit in on daily basis?
• What did I learn about what I don’t know during these days?
• What tools did I use in this project & what tools did I discover?
I felt very happy; I looked for the author name and asked Pradeep for more information about the author and the reply by Pradeep was a “fresher” who has just completed the Finishing school from Edista.
WOW! I doubt if I might have written such a good report as a fresher for my first testing assignment.
Check out the report for yourselves @ http://www.scribd.com/full/13774644?access_key=key-13o6fneq87daonc2luix
From the report I did not see a trainee documenting some stuff for the sake of a certificate or a trainee not interested in testing, forced into the role going thru the formalities of a course. I felt the report is written by a tester, who is passionate about what he is learning that to me is a great sign for Indian testing.
Jai ho etifinishingschool
If you support such an initiative please e-mail to email@example.com with Your Name, Your designation, Your organization name, Your web address and ETI would add you to their current list of supporters and publish it in their website.
“QA team has come up with some intelligent testing techniques to validate the product. The customer test team was not able to find even a single critical bug after delivery”
Though the award was a small one in terms of the award categories our organization have, it meant a lot to me and the team, because this was the first time the team used exploratory approach. The assignment was not pure exploratory or session based, in fact it was supposed to be scripted, but a mix of all three exploratory, session based and scripted testing helped us learn, enjoy, and meet our mission.
The assignment started off with developing test cases. We wrote test cases based on the initial requirement document provided. We did come up with a large number of test cases, of which most were re-written, some ended obsolete, deferred, because of changes in software or hardware or firmware or incorrect requirement or technical limitations or schedule limitations or etc. The test cases as part of the process were then reviewed by the stakeholders and were signed off.
The releases started flowing from the development team to the testing team. In the first 2 releases we executed the test cases found some bugs and were quite happy. After the second build release for testing, the bugs started decreasing and we felt bored to execute the same tests. To add to the misery, our hardware boards got limited and we were 5 testers with 1 board to test on.
This to me was the right time to get in exploratory and session based testing.
• I divided the team into 2 with 2 testers in each team,
• Session time was set to 1.5 hrs per session, mission for the session was provided before the session would start.
• After one team completes the exploratory session, the notes, bugs, issues, data used, tasks performed would be discussed with me, while the other team sets out to meet the mission.
Initially I did observe couple of testers finding it hard to meet the destination (mission) without a GPS (test cases), but observed 2 other testers enjoying their testing, switched the pairs in the team, and the bugs started flowing. This session based testing allowed everyone to be occupied with testing, enjoy testing and also helped me keep track on all the proceedings.
Though exploratory was much more fun and the rate at which bugs were found was huge, after some releases, when we had no new features left to be added to the software the bugs reported by team started dropping, when we sat together, I felt the ideas running out in the team – we initially focused on exploring the system with functional and domain related missions. After the discussion we changed the focus from functional & domain to Flow tests - explained the team on what to be targeted in flow tests and what could be the observations to note running flow tests – and what a change, the energy was back, they liked testing the same module again without any new changes in it.
From then on test case execution was approached as an activity to provide only test case execution report, but there were times wherein a test case would remind us of a requirement we missed in our exploratory, a test scenario in one of the test case we missed to run when we followed only exploratory approach. I feel the reason why we could have missed out a requirement or a scenario, while exploring could be because the missions were not planned as carefully as they should have been. Well, a lesson for the next phase.
We are thru with the release, and are pretty happy on meeting our mission.
Hope this little summary inspires other teams to have a look at exploratory, and how you could fit it in your context.
We knew that our internet connectivity thru LAN is slow, but had to observe and document the result on the same network.
After stating the assumptions about connectivity, speed, throughput available I started testing, in 2-3 min I decided not to use our bug tracker but instead write a story in a word document. What did my story contain – every step (test) performed on the application sequentially. An extract from it would look like
1) Reboot the < >
2) Click on < > for the first time. It takes approximately 13 seconds to display all the images in the < >
3) Click on < > and login to < >. The < > takes approximately 6-7 minutes to load 15 images.
4) When initially started it displayed loading x/37 images but after loading 15 images the message disappears and no other images are downloaded
5) Click on < >. It takes 4 seconds to get enabled
The story was easier to write because the application was terribly slow and I could think myself as the nicest/kindest customer our product could ever find.
Was the story helpful to my development team and the concerned team?
Well, they were surprised the way the story ended “30 button clicks – wait for the functionality to complete – took almost 1.5 hrs”. But liked the way the story was narrated, it helped them analyze each step and know the actual issues.
Now my question is - I initially set out to test the application functionally, but changed my tests based on the performance of the application and almost calculated response time for each function. But the final report (story) I documented after almost 1.5 hrs was helpful to both my development and the concerned department. Also, I could save time by documenting it as a story and not as individual bugs – I also doubt whether individual bugs could have revealed the story the same way the story could.
So, what do you call this testing?
Disclaimer: All the blogs shared by me are my ideas, my thought, my understanding of the subject and does not represent any of my employer’s ideas, thought, plans or strategies.
Last week at Software Testing Club, I came across a discussion started by Michele Smith on “How do you keep Fresh Ideas Flowing?” for testing.
I felt it is one of the most important questions but sadly very rarely discussed in our community.
This was my take on the question, what’s yours?
I have had team members complaining that it's boring to test the same program with no new features. I have also observed a pattern, testers who complain this way, usually are very enthusiastic when they begin testing but loose their interest may be after 1-2 weeks or so irrespective of the module they test. One of the reason I feel they get bored is because they run out of ideas to test. As a test lead one of my challenges have always been to keep myself and my team on the look out for information always.
Like in a recent assignment I found the number of bugs reported by my team dropping, when we sat together, I felt the ideas running out in the team – we initially focused on exploring the system with functional and domain related missions. After the discussion we changed the focus from functional & domain to Flow tests - explained the team on what to be targeted in flow tests and what could be the observations to note running flow tests – and what a change I observed I felt the energy was back, they liked testing the same module again without any new changes. If they again run out of energy I still have user testing, risk, stress, variability, etc lined up. (I have found The Heuristic Test Strategy Model by James Bach extremely helpful in fact I have a print out of the model posted in my bay along with several others which keeps reminding me of the endless ideas/possibilities to test a product :)
When we feel like we are running out of ideas I feel we should change our approach/thinking a bit and the ideas start rolling.
Disclaimer: All the blogs shared by me are my ideas, my thought, my understanding of the subject and does not represent any of my employer’s ideas, thought, plans or strategies.