TestToTester

Using Bug Magent to test APIs

I get extremely irritated when people ask me the toolset I use to test without trying to understand/learn what I am testing and how I intend to use a specific tool.
My choice of tool and how I use it very much depends on the testing objective and the context.

Below is one such example when I used Bug Magnet, a chrome/firefox extension to test APIs.

Test objective/mission: Explore the email field added to the JSON payload.

I started off the test session with a 5 min brainstorming. I mostly use mind map for this.

  • what regex is used in the code?
  • what inputs can break it?
  • what inputs are business critical?
  • response code when it fails? 400?
  • error message returned on failure?
  • min/max length?
  • multiple email addresses?
  • duplicates? does it matter?
  • do we care if it is a *valid/fake email address? 
  • what about domains such as mailinator?
  • (Context:) Given we are not going gung-ho on validation. What could be useful at this moment?

From the brainstorming session I decided to break my test session into two.

  1. I wanted to first focus on just the validation around regex and 
  2. a follow up session to gain information on questions outside validation


So refactored the test objective to - Test the email field in the JSON payload for validation.

Started off the session by defining my test data

  • Business critical test data, the email addresses that should definitely be accepted. 
  • Pairing with the dev helped me learn the regex used to validate email address. This helped me add more tests around the boundaries of the regex. 
  • Also, I was aware and wanted to make use of the amazing list in Bug Magent for email valid/invalid addresses. I had previously used it for testing a UI functionality.


Yes, bug magnet from its webpage is an exploratory testing assistant for Chrome and Firefox. But I wanted to make use of it's email valid/invalid addresses to validate a field in JSON payload.

Navigating to the bug magnet installed folder path revealed the below list in config.json

"E-mail addresses": {
    "Valid" :{
      "Simple": "email@domain.com",
      "Dot in the address": "firstname.lastname@domain.com",
      "Subdomain": "email@subdomain.domain.com",
      "Plus in address": "firstname+lastname@domain.com",
      "Numeric domain": "email@123.123.123.123",
      "Square bracket around IP address": "email@[123.123.123.123]",
      "Unnecessary quotes around address": "\"email\"@domain.com",
      "Necessary quotes around address": "\"email..email\"@domain.com",
      "Numeric address": "1234567890@domain.com",
      "Dash in domain": "email@domain-one.com",
      "Underscore": "_______@domain.com",
      ">3 char TLD": "email@domain.name",
      "2 char TLD": "email@domain.co.jp",
      "Dash in address": "firstname-lastname@domain.com",
      "Intranet": "name@localhost",
      "Non-ascii Email" : "nathan@学生优惠.com"
    },
    "Invalid": {
      "No @ or domain": "plainaddress",
      "Missing @": "email.domain.com",
      "Missing address": "@domain.com",
      "Garbage": "#@%^%#$@#$@#.com",
      "Copy/paste from address book with name": "Joe Smith ",
      "Superfluous text": "email@domain.com (Joe Smith)",
      "Two @": "email@domain@domain.com",
      "Leading dot in address": ".email@domain.com",
      "Trailing dot in address": "email.@domain.com",
      "Multiple dots": "email..email@domain.com",
      "Unicode chars in address": "あいうえお@domain.com",
      "Leading dash in domain": "email@-domain.com",
      "Leading dot in domain": "email@.domain.com",
      "Invalid IP format": "email@111.222.333.44444",
      "Multiple dots in the domain": "email@domain..com"
    }
  },

I then added the email addresses (business critical and regex boundaries) to the above list and converted it into a CSV file. A small sample below

email_addresses
firstname+lastname@domain.com
email@123.123.123.123
xxx@yyy.com

The next step was to turn the email field in the JSON payload into a variable and run it via postman collection.

{
    "docEmail":"{{email_addresses}}"
}

The test revealed many inconsistencies, some values were validated to return 400 but some just threw 500. The generic error message was not helpful either.

--------------------------------------------end of session-------------------------------------------

So, yes a tester could give you any of the below tools plus more if you ask them just their toolset

- Mindmup
- Bug Magent
- Big list of naughty strings >> https://github.com/minimaxir/big-list-of-naughty-strings
- Atom
- Postman
- Fiddler
- Insomnia
- J-meter
- Charles Proxy
- pyresttest
- dev tools
- newman, etc

but you will never learn how they use it!

Coaching by doing

Recently, I was not happy with the bug reports on one of my team. The title was generic, there was little or too much information in its description, logs, response was not formatted and the attachments at times made very little sense.

My initial thought was to forward 'how to write good bug reports' blog links from the web but something in me did not wanted me to do it. They explored well and the bugs they found were very good. Also, I did not feel a meeting or a workshop targeting only bug writing skills would have the right impact on this team.

So instead of direct coaching or sharing blog links I started to reproduce the bugs locally and started editing the bug title/description/code format/attachments, etc in the bug reports. I did not mention anything to the team.

After couple of weeks I started noticing a change. The bug reports got better!
The title was appropriate, description much improved, code well formatted with screen shots and there was also gifs in attachment.

When I thanked the team for observing and improving the bug descriptions they mentioned how much my changes to the bug description helped them understand the - why I am doing it and the value in it. They wanted to get better too but did not know how. The changes served them as examples on how to approach it. This was extremely satisfying to me too because I was able to influence my team by just doing it.

Test with me @

Tweets