Blogs

Testing Algorithms is now teaching its automated test case design technique...

11/18/2016

“Share your knowledge. It is a way to achieve immortality.” - Dalai Lama

Few years back, I and my friend Palash started a research to find out whether, during a software development, there is an easy way to capture business requirements that would automatically translate those requirements to an optimum set of test cases covering maximum test combinations. Last year we were able to find a solution and that's when we started Testing Algorithms, LLC.

In today's world, more than half of the defects originate from ambiguous and unclear business requirements. Moreover, in most cases the projects end up spending a lot of time and effort in over-testing the application under development. Our unique and patent-pending automated test case design solution claims that it creates an optimum set of test cases 10-times faster and covers 100% (under certain assumptions) of the requirement combinations. Refer to our case studies.

The purpose of our organization is not to selfishly accumulate money by selling the licences of the product that we have created. We believe in helping each other, especially to the business analysts and software tester community whom we professionally grew with, is the right way of living. We think that our unique skill, which we have developed during our years of research, should be shared with each and every tester of the planet with a hope that it will make their lives a little better.

With that objective, we made our product and techniques available to everyone in August 2016. Anyone, who is interested in trying this new process, can register in our website to get a copy of our product. We also put in place a way to give us return gift if our users find it useful. Refer to our videos.

But we recently received some feedback from our users where they said that a little more guidance on using the solution would be even more helpful. As we are both teachers, we decided to start a training program for all who are interested in using our techniques individually or in their organizations, against a nominal fee as compensation for our time. As always, the product is and will be free. And you will be able to keep your knowledge and use them to stand out for the rest of your professional life.

As part of this course, we are planning to have 2 sessions of 2 hours each on Saturday & Sunday morning EST/CST. Also, it will be an online training via Skype with screen-sharing and lots of hands on assignments for practice. The course fee per person will be approximately $200. Certificates will be issued to all participants at the end of the course to recognize them and to help them getting the training fee reimbursed by their employers, if needed. We are planning to have our first training sessions in the beginning of January 2017.

I would love to hear your thoughts and whether you will be interested in attending the course. Feel free to let me know if you have any questions and/or suggestions.

P.S. No matter whether you take this course or not, I am always available (abhimanyu@testingalgorithms.com) if you need any assistance in using our solution.

Our first eBook is available!

10/6/2016

We, testers, test software applications to find bugs. We are called bug-hunters. But if we take a step back and look at the mirror, we will see ourselves as someone much more important than mere bug-hunters. We are the trust-builders. Just like the bomb disposal squad.

When a bomb is disarmed, civilians are no more scared to use the street where the bomb was originally found. That's because people trust the bomb disposal squad and the techniques they use to disarm it. Well, in a similar way, the business users trust the testers and their testing processes at the time of using the software application in their day-to-day business.

But the question is, how do we disarm a bomb? How do we ensure that it is not dangerous anymore? Is it a science or an art? Can we increase the reliability and efficiency of a techniques we use? Can we disarm it faster? Can a robot do it?

This book is the outcome of the philosophical investigation to find answers of all these software testing questions. To read this book, a little knowledge of a software development process is helpful but not necessary.

**The book is available in amazon.com:
https://www.amazon.com/dp/B01LXRTVDW/ref=rdr_kindle_ext_tmb

The book is available in testingalgorithms.com for our users:
https://www.testingalgorithms.com/ebook.html

Note:
All users of the automated test case design solution of Testing Algorithms get the eBook for free. They can choose to give return gifts based on the perceived value of the book.

Link to register as an user of Testing Algorithms:**
https://www.testingalgorithms.com/registration.html

Inferences from Testing Challenges Survey

7/11/2016

Participants:

Participants of the survey were mostly:
1. Part of dedicated testing teams (92% of the organizations)
2. Following Agile methodology (89% of the organizations)
3. People who are not in managerial positions (77% are Test Analysts/Leads)
4. Located in Asia & Europe (76% of the participants)
5. From bigger organizations (67% has >200 employees, 59% has >500 employees)

Observations:

6. Requirement Analysis (64%) is a bigger area of improvement than Automated Test Execution (61%).

7. Test Data Creation (45%) is a bigger area of improvement than Status Reporting (38%).

8. Test Case Creation (42%) is a bigger area of improvement than Manual Test Execution (32%).

9. Defect Management (26%) is not a big problem in most of the organizations.

10. Combining points 6-9, it looks like the test execution, defect management and status reporting process has been streamlined quite a bit. However, requirement analysis & test planning is emerging as the primary pain area in many organizations.

11. Only 41% organizations use Model Based Testing tools for test case creation and only 36% use a Test Data Management tool. This confirms our hypothesis in point 10.

12. Automated Regression Testing has been adopted by most organizations (80%), but still 87% participants feel that testing takes more time. This also confirms our hypothesis in point 10.

13. Even if 80% organizations are using Automated Regression Testing, 74% says that testing phase exceeds their budget. This indicates either or both of the following:

a. Automated Regression Testing didn’t reach its break-even point;
b. Automated Regression Testing is not cost effective.

14. Most importantly, 92% of organizations fail to prevent defect leakage to the next testing environment, in spite of spending more time, spending more money and utilizing strong test management, test management and automated testing processes.

Conclusion:

The primary problems are in requirement analysis and test design processes. These continue to be manual processes since time began and its quality is believe to be dependent on human skills vs. automated techniques. There are various tools and techniques that are currently available that can make requirement analysis and test design much faster and better. It’s high time testing organizations start moving towards the direction of automated test case design. In fact, I predict automated test case design to be one of the most demanding topics in software testing area in the next decade.

Fishing and Software Testing - An Analogy

5/31/2016

Original: I am a fisherman. I mostly work in waterfall.
Translation: I am a software tester. I mostly work in waterfall.

Original: My goal is to catch all fishes in a pond.
Translation: My goal is to identify all defects in a software application.

Original: I aim to catch the bigger fishes first.
Translation: I aim to identify the critical defects first.

Original: I plan to navigate through all geographical coordinates of the pond.
Translation: I plan to cover all business functions of the application as part of my testing.

Original: I don't know how deep the pond is at various geographical coordinates.
Translation: I don't know how critical the various business functions are.

Original: I met with Department of Natural Resources and created a depth map of the pond.
Translation: I met with the Business Users/Analysts and created a criticality map of the business functions.

Original: I looked at the list of fish species present in the pond to determine my fishing strategy.
Translation: I looked at the various requirements of the application to determine my testing strategy.

Original: I have created a plan on how deep I will place my bait in each coordinate.
Translation: I have created all my test cases with expected results.

Original: I have collected various baits required for fishing.
Translation: I have arranged for various test data required for fishing.

Original: I have reached in front of the pond to start fishing.
Translation: The application under test has been deployed to the test environment.

Original: I made sure that my boat is in working condition.
Translation: I made sure that I have all required access to the application.

Original: During test execution, I often change my directions to coordinates where fishes are more probable.
Translation: During test execution, I often re-prioritize my testing to focus on defect-prone business functions.

Original: I inform my captain about the number of fishes caught on an hourly basis.
Translation: I publish daily/weekly test execution status reports during test execution.

Original: Sometimes fishes move from one coordinate to another coordinate that has been covered by me earlier.
Translation: Sometimes one defect fix breaks a functionality that has been tested by me earlier.

Original: So, I keep an eye on coordinates that I covered earlier while moving to new coordinates.
Translation: So, I assess the need for regression testing of functionalities tested earlier after each defect fix.

Original: Sometimes I catch turtles by mistake that I eventually return to the pond and thus waste my time.
Translation: Sometimes I waste my time on non-defects that eventually get rejected.

Original: When I am done, I move to another environment.
Translation: When I am done, I move to another environment.

Is software testing a different game of poker?

5/11/2016

As per the ISTQB study material, the definition of software testing is "a process of executing a program or application with the intent of finding the software bugs."

But to me, software testing is a different poker game where a tester 'pokes' an application under test using various input actions and data combinations and compares the observed behavior of the application with its expected behavior. The primary dilemma in determining a test strategy is 'how to intelligently poke the application considering limited resources and time?’ To me, this sounds more like an optimization problem.

I have seen many testing teams come up with test cases for an application without giving much attention to the optimization aspect of the problem. Traditionally, the complete set of functionalities of the application under test is broken down into transactions, then into test scenarios and then into test cases with steps and expected results. However, this manual exercise often leads to a big number of repetitions and missed requirements in the test suite created. This is because a structured thought process and an effective optimization technique is usually missing in the process.

If we look at software testing as a special case of the poking game mentioned above, a mathematical optimization problem can be formulated that will generate minimum number of test cases that eliminates all gaps in test coverage.

We, at www.testingalgorithms.com, have come up with a process that makes software design process much faster, better and cheaper.