A reply, that grew long.

I wanted to post this as a reply to the blog post “I know you'd have a test case for this !!!” by Ajay, but since the reply grew long, I posted it on my blog.

I agree with James Bach stating that this is a platform issue. It’s very clear that the issue reported here is more to do with the platform than the application itself.

This makes me wonder why Windows XP was chosen to run such an application. There could have been many reasons like ease of compatibility, familiarity, db connectivity, interoperability, ease of development, etc. However one needs to keep in mind that Win XP is not designed to just run in the background, it is an end-user interface for applications. So, Win XP should have been customized to meet the needs of deployment.

Now let me try answering the questions posted in the blog.

  • Is it a bug? How risky is it to ignore such messages?
    • If I go by the definition used by James and Michael “A bug is something that bugs somebody who matters” – yes I would call it a bug.
  • How risky could this be
    • Well, if there is a 24/7 support answering the travellers queries I do not feel it is risky, since the affected traveller could get their query answered, also knowing this balloon from Win XP, it hides after 10s or so, I do not feel it a very risky bug.
  • Is the purpose of the display served?
    • Yes, the purpose of display is definitely served, if there is a pop-up marginally blocking the display for a few seconds, I would not go to the extent of questioning the display. I agree that there could be % of customers who might be annoyed, but definitely the display is served.
  • If the overlapping of the message on the display is a bug, will you fix it?
    • To me this is a redundant question apart from “will you fix it” part. Will I fix it – since this issue could be resolved by disabling the informative balloons in Win XP, I would fix it. The cost to fix it very low, so I guess it could be a straight forward decision.
  • What if it is not fixed?
    • If it’s not fixed, it might continue to annoy a % of users.
  • Which tester will think of these kinds of tests?
    • It’s interesting how everything and anything boils down to a tester, may be because he is the one who touches the product last before it’s shipped. I feel issues like this could be minimized when the engineers (dev & testers & deployment) have enough domain knowledge or aware of the deployment or know their exact mission. This check might be of highest priority in the checklist for engineers from SCADA or HMI domain (Well, they might have a test case for this:).
  • Do you wear the hat of a non-tester and say: Hmmm, there is a workaround. I'll not fix it.
    • Who is a non-tester in this context?
  • I do not know how to fix it.
    • Are you kidding me? :)
  • It might be a bug but it is a limitation of the technology. [Cannot fix]
    • WHT? I seriously have no words to say.

On further scouting for similar information over the web, I came across this very informative link on how to turn Win XP to a HMI platform. In the article author writes about different parameters such as performance, security, disabling un-wanted balloons to make Win XP suitable for HMI platform.

Comments

Santhosh Tuppad said…
@Sharath,
Good explanation. I was impressed with the investigation you did before just having your say. You talked about disabling the pop-up which is true. This is a flexibility which is given to an end-user and this is not really a fix but changing the configuration. You talked about calling it a bug if you refer it to James Bach and Michael Bolton's definition.

After many days of peeping into your blog I found a cool post. Keep it up.

All the best for your future posts too *smiles*.

Thanks,
Santhosh Shivanand Tuppad
March 16, 2010 at 11:37 PM
Ajay Balamurugadas said…
Thanks for your time Sharath.

>> How risky could this be
Well, if there is a 24/7 support answering the travellers queries I do not feel it is risky, since the affected traveller could get their query answered, also knowing this balloon from Win XP, it hides after 10s or so, I do not feel it a very risky bug.

In this context, as a user, I want to get my ticket ASAP. So, when the officer is keying in the ticket details & I'm unable to see it on the display, does the risk of this bug increase?

Even if the balloon displays itself for 10 secs, I may be the customer whose ticket details are displayed for that time interval.

Regards,
Ajay Balamurugadas
March 18, 2010 at 9:26 AM
Madhukar Jain said…
Hi Sharath,

>> How risky could this be
Well, if there is a 24/7 support answering the travelers queries I do not feel it is risky, since the affected traveler could get their query answered, also knowing this balloon from Win XP, it hides after 10s or so, I do not feel it a very risky bug.

In the above context as a traveler I am waiting on the window to get my ticket and if this Pop up blocks my view then I consider this as an issue as it matters the most to me as an user. May be it lasts only for 10 secs.

Imagine if a customer has this popup and 10 secs are wasted per customer, now multiply the number of times this popup occurs in a day with 10 secs and multiplied by 365 days a year, i will say it might pose that so much time was lost which could have been utilized in selling tickets to other customers or if it is a query window then could have been used to answer the calls.

When you mention about 24/7 support answering the queries then a person buying the ticket when his turn comes will not be able to leave his turn in order to go to the Inquiry Window as there is a long queue on that window too or if the Ticket Seller himself has to answer the query then it’s a waste of time for the ticket seller to tell the amount just because that the Display Board has a popup on it ( Time wasted again).

Imagine the amount of money which the Ticket seller can cheat the customers who are not so literate because when the pop up blocks the view then the Ticket seller might return less amount of change to a traveler and since the person is in a hurry for train he might not check the price of ticket as he was unable to see it on display so he assumes that he has been given the right change back.
Many things we can consider here, it all depends on the context and here the context is users. Hence as per my Views it’s an issue which has to be fixed and cannot be left unattended.

Regards,
Madhukar.
March 18, 2010 at 10:41 AM
Sharath Byregowda said…
@ Ajay
If the officer is keying in the ticket details, what other information a user seeks, will not the officer answer the user query.

@ Madhukar
I really like the way you have provided various scenarios that could come up in the context. I loved the ticket seller cheating example the most (but honestly I do not feel our govt officials are that bad:)

Anyways, if you look at my answers to the following questions in the post, I would like to get this issue fixed as well. But, to me the bug is not risky at all, since the users have a workaround.

To me the word "risky" is a very strong word in this context.

regards,
Sharath
March 18, 2010 at 12:06 PM
Madhukar Jain said…
Hi Sharath,

Yes i also agree that this Bug is not risky, however if i take the context of time critical missions where every nano second counts then this Bug is very risky in that context.

Here its a usability feature which does not allows the Ticket Seller to see the amount when the pop up comes. If the ticket seller is selling many tickets then he need to see the exact price for the number of tickets hence he might need to close that pop up when ever it comes in order to see the price.

Its not critical bug in this context.

The Debate goes On and On :)

Happy Testing..

Madhukar
March 18, 2010 at 12:22 PM
Santhosh Tuppad said…
@Ajay & Madhukar,
Good points. I liked them :)

Sometimes what might not irritate you might irritate other users. Different people are different perspective about this bug talked by Ajay :)

So it is a bug but severity depends on the context :)

Thanks,
Santhosh Shivanand Tuppad
March 22, 2010 at 11:42 AM
Post a Comment

Popular posts from this blog

Exploratory testing, Session based testing, Scripted testing…concertedly

In my last assignment a mix of exploratory, session based and scripted testing helped won an award, appreciation by managers & stake holders at our organization. The quotes from our award certificate “ QA team has come up with some intelligent testing techniques to validate the product. The customer test team was not able to find even a single critical bug after delivery ” Though the award was a small one in terms of the award categories our organization have, it meant a lot to me and the team, because this was the first time the team used exploratory approach. The assignment was not pure exploratory or session based, in fact it was supposed to be scripted, but a mix of all three exploratory, session based and scripted testing helped us learn, enjoy, and meet our mission. The assignment started off with developing test cases. We wrote test cases based on the initial requirement document provided. We did come up with a large number of test cases, of which most were re-written, some
Read more

What do you do when you find a bug?

What do you do when you find a bug? Hold on do not answer it right now. Let me first set the context. You are part of an agile team. The developers sit opposite to you. You are testing a feature in a session. You are time bound and have the good habit of capturing notes when you test. When you now find a bug what do you do? Assuming that you have taken enough notes you go on to write a good bug description and may capture a screen shot, video of the bug and continue with your session. At the end of the session do you Open the defect tracking tool (wait for it to load), enter all the fields for a bug, upload necessary evidence and have the tool share the defects with your dev team? Or Open the defect tracking tool (wait for it to load), enter all fields for a bug, upload necessary evidence and then compose an email to the dev team with the bug id’s/descriptions? Or Walk to the developer share your notes; reproduce it if required, work together to identify the problem? I work in
Read more

Regression Checks + Regression testing = Regression testing?!

One of the questions I find hard to answer when I use exploratory test techniques in my project is,”How do you run regression tests? How do you prioritise them? Do you automate them? Do you document regression test cases?” And then a bunch of questions follow. Let me define the context. I work closely with developers; we use either Scrum or Kanban - which ever best suits our project, product owner, business. We do not believe in individual metrics, blame games, stringent processes, and heavy documentation. All we care about is to deliver high quality software. This does not mean that there is no process, metrics or documentation. We have them all, but they are there to meet our needs and to help those who will come onboard to support/maintain the software. Regression Checks At the start of a story, testers pair up with developers to develop feature files using SpecFlow. Some call it Acceptance Test Driven Development and others Behaviour Driven Development [I am yet to figure out the d
Read more