I grew up in a big city in India. A hot one!

Air-conditioners were not that uncommon in my childhood. However, there was one problem. It used to get very cold inside whenever the outside temperature dropped in the middle of the night. One had to wake up, get up and adjust the temperature before going back to bed.

Then the age of remote controls came. And the air-conditioners started coming with remote controls so that one wouldn't have to get up to adjust the temperature. However, they still had to wake up!

The evolution of test automation is very similar to the air-conditioners.

In our earlier days, we had record-playback which had to be adjusted every time there was a change in requirement (just like the outside temperature). Then came automation frameworks which made the maintenance of automation scripts easier. Just like the remote control.

But still a tester needs to wake up and adjust the inside temperature manually!

Then a fundamental concept of Physics changed the whole air-conditioning industry.

Thermostats!

Now we don't have to even wake up for a temperature adjustment!

We, at Testing Algorithms, are working on a Thermostat for test automation. We were able to link test automation scripts directly with the business requirements so that if one changes, the other changes automatically.

If you are interested to see our solution, watch our video.

If you want to read our previous article on this topic, please read this post.

​If you are looking for more information, please contact us.

​Testing Algorithms is...

... neither a solution for automated test execution, nor for test management. However, it easily integrates with both types of tools.

... neither associated with, nor a subsidiary of any other company. Me and my friend, Palash, came up with this solution based on our years of research on requirement analysis and test planning and formed Testing Algorithms in October, 2015.

... neither a product company, nor a service company. Our business model is based on teaching, training and consultancy to individuals and organizations.

... neither a non-profit, nor a charity organization. However, we believe in helping the testing community and therefore offer our automated test case design solution to our believers against return gifts.
​

​During execution of test cases, wouldn't a statement like the one above, for each pending test cases, be helpful?

Also, wouldn't it be even more helpful if these probabilities are revised automatically every time a test case is executed and its outcome is recorded?

Well, I tried to summarize some of the benefits below.

1. Test Prioritization: Fail probabilities assigned to each test case would enable testers to determine the order of execution in such a way that defects (especially the critical and high defects) are identified at the earliest. This will give the developers enough time to fix those defects.

2. Stopping Rule: Creation of rules like "stop testing if all remaining test cases have fail probabilities < 10%!" before the test execution starts would be possible. This would help the testing team to avoid over-testing, by objectively and quantitatively deciding when to stop. This would also help in determining the extent of regression testing for a release.

3. Effort Estimation: At the time of estimating the testing effort for a project and the number of resources required, usually a fixed percentage (e.g., 15%) of the total testing effort is assumed for defect re-testing. However, most of the time we under-estimate it and thus experience a tremendous amount of time-crunch towards the end of testing. With these fail probabilities; defect re-testing effort could be determined more accurately.

Makes sense?

Now, the question is, it is possible to calculate these fail probabilities?

And if so, how?

Very recently I helped a friend in the analysis of a completely different problem (not even related to software testing). An organization shared a list of their employees. Our task was to calculate the probability of attrition for each employees based on their demographic information, salary information and survey responses so that the organization can take necessary steps to prevent attrition.

This analysis was done using some statistical models and the accuracy of the models were very high in terms of predictability.

And, while doing this analysis, I discovered something else that would be help in software quality assurance!

I found that, at the time of test execution, the determination of fail probabilities for test cases, based on various attributes of the test cases, is the exact same problem!

We, at Testing Algorithms, are working on creating a framework where the fail probability of test cases (generated by our patent-pending automated requirement analysis and test case design solution) can be automatically calculated and revised during test execution.

If you are interested to know more, feel free to contact us. We would be happy to talk to you about this.

This is an Agile question. Why cars have brakes?
 
The purpose of a car is to move people and things from one place to another. And the performance of a car is generally measured by parameters like how fast it can go and how quickly it can pick up speed.
 
But, if that is only the case then why just having a gas pedal is not enough? Brakes waste both time and gas anyway! Why cars have brakes then?
 
I think we all know the answer. It's for safety.
 
And I am actually talking about documentation in Agile.
 
Document is a waste unless it is necessary for safety.
 
But when documentation becomes necessary for safety?
 
Well, I have seen situations where everyone perceived a user story (and its acceptance criteria) completely differently. It ended up in a car accident because it didn't have a brake!
 
May be this is one of the reasons why most scrum teams are very stressed in today's world.
 
Agile Manifesto says "Working software over comprehensive documentation". And most of us tend to ignore the word comprehensive, consciously or unconsciously.
 
What would make this easier for everyone is, if we had a process to create enough documentation (just to bring everyone on the same page) that doesn't waste any time and effort.
 
For example, if we had a process where, on clicking a button, optimum set of test cases in the organization-specific format, requirement traceability matrix and standardized use cases and process models are instantly (i.e., automatically) created from the user stories (written in a specific way), it would have made things much easier.
 
And this is exactly what we, at Testing Algorithms, are trying to accomplish based on our years of research. Please note, our purpose is to help the software development community and our solution is a gift to all. However, you can choose any amount (including zero) for giving us return gift, if our solution helps your organization.
 
Please register if you are interested to try something new.

I have studied Statistics as a student and then spent most of my professional life doing software testing! It's always been my curiosity to find out how analytics can help in identifying various patterns while testing a software under development. One of the specific areas that always attracted me was the test management where I thought inferences about the performances of the development and testing teams could be drawn based on statistical data analysis. And, as part of that investigation, I found out that the information contained in the logged defects tell us many stories.

In one of my previous projects, we were using a standard defect management tool where individual defects were following the following workflow:

While looking at the defect metrics, I always made the following assumptions about existing defects:

1. Defects staying in New status for a long time indicates that there might be a problem with the defect triage process. Either triage should be performed more frequently, or there are many requirement gaps that are causing the confusion about whether an anomaly is a defect or not.

2. Defects staying in Open or Reopen status for a long time indicates that there might be a problem with the performance of the development team. Clearly, they are not able to fix defects effectively and efficiently. Either the development team is understaffed, or they need more technical training.

3. Defects staying in Fixed status for a long time indicates that there might be a problem with the performance of the code deployment team. It is an indication that the testers can't retest the defect yet because the updated code is not available in the test environment.

4. Defects staying in Retest status for a long time indicates that there might be a problem with the performance of the testing team. Clearly, they are not able to retest defects effectively and efficiently. Either the testing team is understaffed, or they need more training on testing processes or the application under test.

So I found out a way to measure how long the defects are staying in these statuses so that inferences can be made about the root cause and appropriate actions can be taken to improve the speed and/or quality of testing.

I noticed that the defect management tool captures and maintains the history of status changes for all defects. And also, a report containing that history can be exported to excel format. So, using my little coding knowledge, I was able to write a small program that parsed through that excel report and captured the duration of stays of all defects in various statuses. The output of my program looked something like this:

New: 1, 5, 3, 7, 5, 2, 3, 1, 7. (Note: There was total 9 defects)

Open/Reopen: 5, 8, 4, 2, 6, 8, 7, 8, 9, 4, 2. (Note: 1 defect got reopened 2 times)

Fixed: 4, 5, 2, 7, 4, 6, 8, 3, 2, 9, 8. (Note: 9 defects were fixed 11 times)

Retest: 3, 6, 7, 2, 6, 8, 4, 6, 3, 9, 8. (Note: 9 defects were retested 11 times)

I assumed a Gaussian or Normal distribution for all the duration data. A Normal distribution looks like this:

So, while doing testing, I constructed normal distribution for each of the 4 sets of data (i.e., for New, Open/Reopen, Fixed & Retest) on a daily basis to understand how this duration is increasing or decreasing over time. And to to do this, I used box plot diagrams. Following is a example where box plots are constructed on a daily basis:

Note that, the individual boxes are separate normal distributions and viewed from the top. Anyway, I kept plotting the distributions in this way for all of the 4 sets of data (i.e., for New, Open/Reopen, Fixed & Retest) separately. This helped me identifying the pattern and to guess the root cause of the problem so that I could take necessary actions to bring the speed and quality of testing on track, as and when needed.

I have tried this technique many times after that in both Waterfall & Agile projects and it helped me every time!

Over the last few years, many organizations have realized that software development is not a production process; rather it is more of a Research and Development process, as Agile Manifesto indicated in 2001. Since then, many approaches and techniques were crafted that have been successfully used and still being used in these organizations. 

Business Driven Development (BDD) is one of them, and probably the most popular today. It uses a Given-When-Then (i.e., Gherkin) statement that serves multiple purposes:

1. Elaborates the product feature with details and examples,
2. Aligns the Definition of Done (DOD) with business users,
3. Specifies the acceptance criteria of a user story, and
4. Accelerates Test Driven Development (TDD) using test automation.

However, the current practice in the industry is to write the Given-When-Then statements manually, as a team.

But what if we go up one more level of abstraction? In other words, can we capture the feature, or the behavior, of the software under development using a much simpler model and generate not only the Given-When-Then statements, but also the optimized manual test cases in Quality Center format, various types of requirement traceability matrices, test  case priority matrix, traditional use cases and process flow diagrams?

We, at Testing Algorithms, came up with a solution for that based on our years of research. It is a methodology that helps in structured thinking and representation of the application behavior so that the optimization and visualization can be taken up by a mathematical algorithm. Based on our case studies, this model is able to create all artifacts, including better quality test cases, ten times faster.

Agile is about innovation. And we believe in helping each other to grow as a community. So we have decided to let everyone use our research-based approach for free. Our website is https://www.testingalgorithms.com. 

If you want to try our requirement analysis and test case design  solution, please register using https://www.testingalgorithms.com/registration.html.

For any questions, contact us at https://www.testingalgorithms.com/contact-us.html.

​"Alone we can do so little; together we can do so much." - Helen Keller

Testing algorithms is slowly becoming a community where software testers learn together to make a difference. We challenge the conventional believes and practices of software testing to make our work easier, yet effective. 

With the worldwide popularity of Agile methodology, the lines between a business analyst, a developer and a tester are fading day by day. At this juncture, it is essential to have perspectives of each other to make software development projects successful. Because of this reason, cross-certification is becoming very common phenomenon where a tester is learning technical languages and a developer is learning quality assurance techniques, for example. 

ISTQB Foundation Level is one of the most popular certifications in software quality assurance and thousands of people appear for this test every year. We, at Testing Algorithms, wanted to help the aspiring candidates of ISTQB Foundation Level by providing a simple excel tool to practice with 500+ sample questions.

​In order to access this tool, please register using the following link:

https://www.testingalgorithms.com/registration.html

With this registration, you can also access the unique automated test case design solution invented by Testing Algorithms and the eBook "We Are All Pokers: A Different Perspective on Software Testing".

Note that, all tools, products and books of Testing Algorithms are free and will be free forever. However, you can give us "return gift" with whatever amount you thing they are worth. 

If you have any questions, please contact us as support@testingalgorithms.com.

All the best for your ISTQB Foundation Level!

“Share your knowledge. It is a way to achieve immortality.” - Dalai Lama

Few years back, I and my friend Palash started a research to find out whether, during a software development, there is an easy way to capture business requirements that would automatically translate those requirements to an optimum set of test cases covering maximum test combinations. Last year we were able to find a solution and that's when we started Testing Algorithms, LLC.

In today's world, more than half of the defects originate from ambiguous and unclear business requirements. Moreover, in most cases the projects end up spending a lot of time and effort in over-testing the application under development. Our unique and patent-pending automated test case design solution claims that it creates an optimum set of test cases 10-times faster and covers 100% (under certain assumptions) of the requirement combinations. Refer to our case studies.

The purpose of our organization is not to selfishly accumulate money by selling the licences of the product that we have created. We believe in helping each other, especially to the business analysts and software tester community whom we professionally grew with, is the right way of living. We think that our unique skill, which we have developed during our years of research, should be shared with each and every tester of the planet with a hope that it will make their lives a little better.

With that objective, we made our product and techniques available to everyone in August 2016. Anyone, who is interested in trying this new process, can register in our website to get a copy of our product. We also put in place a way to give us return gift if our users find it useful. Refer to our videos.

But we recently received some feedback from our users where they said that a little more guidance on using the solution would be even more helpful. As we are both teachers, we decided to start a training program for all who are interested in using our techniques individually or in their organizations, against a nominal fee as compensation for our time. As always, the product is and will be free. And you will be able to keep your knowledge and use them to stand out for the rest of your professional life.

As part of this course, we are planning to have 2 sessions of 2 hours each on Saturday & Sunday morning EST/CST. Also, it will be an online training via Skype with screen-sharing and lots of hands on assignments for practice. The course fee per person will be approximately $200. Certificates will be issued to all participants at the end of the course to recognize them and to help them getting the training fee reimbursed by their employers, if needed. We are planning to have our first training sessions in the beginning of January 2017.

I would love to hear your thoughts and whether you will be interested in attending the course. Feel free to let me know if you have any questions and/or suggestions.

P.S. No matter whether you take this course or not, I am always available (abhimanyu@testingalgorithms.com) if you need any assistance in using our solution.

We, testers, test software applications to find bugs. We are called bug-hunters. But if we take a step back and look at the mirror, we will see ourselves as someone much more important than mere bug-hunters. We are the trust-builders. Just like the bomb disposal squad. 

When a bomb is disarmed, civilians are no more scared to use the street where the bomb was originally found. That's because people trust the bomb disposal squad and the techniques they use to disarm it. Well, in a similar way, the business users trust the testers and their testing processes at the time of using the software application in their day-to-day business. 

But the question is, how do we disarm a bomb? How do we ensure that it is not dangerous anymore? Is it a science or an art? Can we increase the reliability and efficiency of a techniques we use? Can we disarm it faster? Can a robot do it? 

This book is the outcome of the philosophical investigation to find answers of all these software testing questions. To read this book, a little knowledge of a software development process is helpful but not necessary.

The book is available in amazon.com:
https://www.amazon.com/dp/B01LXRTVDW/ref=rdr_kindle_ext_tmb

The book is available in testingalgorithms.com for our users:
https://www.testingalgorithms.com/ebook.html

Note:
All users of the automated test case design solution of Testing Algorithms get the eBook for free. They can choose to give return gifts based on the perceived value of the book.

Link to register as an user of Testing Algorithms:
https://www.testingalgorithms.com/registration.html 

Few days back, my family was invited to a birthday party. It was the fifth birthday of one of our new neighbor's son.

My son wanted to buy a big set of Lego for him because "he seems like a nice guy". So we bought it.

The birthday boy was so happy and excited receiving our gift! We had a great time there.

Then, after a few hours, when it was time to say goodbye, he offered his favorite video game to my son. As a return gift.

They are good friends since then.

I found something very interesting in that transaction. Our neighbors didn't have to invite us, we didn't have to buy a big gift and then they didn't have to be so generous about the return gift! All our actions were our own choices.

Me and my friend, Palash, thought a lot about this and decided to try the same concept to run our small business, Testing Algorithms.

We believe that our automated test case design product can reduce 90% of test case creation and documentation time if used with right philosophy, attitude and technique.

So we want to give this to all as gifts, just like the Lego, if we are invited to do so.

Please note, return gift is not mandatory, but appreciated.

​I heard the following story a few months back.

A wise man, living in a small town, invited all residents of his hometown in his house. There were around a hundred people who showed up on the scheduled day. He had one of his bigger rooms already filled up with hundreds of balloons. Each balloon had name of a guest written on it.

He asked everyone to enter the room and find the balloon that had her name on it in three minutes. He declared a handsome prize for those who would be able to find theirs.

He started his stopwatch and everyone started rushing to look at each and every balloon to check if those had their names. When the watch stopped after three minutes, unfortunately, everyone came out of the room without their own balloons. Three minutes was too short a time for this exercise!

Then the host, the wise man, asked everyone to repeat the exercise, but this time he showed everyone a trick. The trick was to pick up a balloon, read the name on it, find the person and hand over the balloon to her, then pick up the next balloon and repeat the same steps.

The exercise was repeated and this time everyone was able to come out of the room with their own balloons within three minutes.

During our journey of creating our product and our company, we realized that most companies follow the first approach to sell their products or services. However, we want to take the alternate approach, as shown by the wise man of the story. Our automated test case design solution will be ready for everyone, who believes in our approach and philosophy, in a few weeks, for free.

However, we are a small company with a few people who want to make a living by doing what we believe in and what we love to do. And we understand that the value perception of our solution would be different for different users. Therefore, we will have a ‘pay what you want’ provision where anyone can contribute any amount according to how much value they see in our methodology and how much this is helping in your test design. We would appreciate any contribution and we promise that your contribution will be spend only on enhancing the product and customer experience.

One more thing.

We will keep providing the support and answering all your questions via email. We plan to have a feedback mechanism so that our solution can be improved based on your suggestions. We also plan to have a LinkedIn group created only for our users so that all communications are transparent to everyone belonging to the user community. However, if anyone needs any training or consultation about our methodology or product, or any other verbal assistance, we will charge for our times. I hope that is reasonable for a small startup company with no funding.

This is not a marketing gimmick. We, the founders of Testing Algorithms, believe that entrepreneurship is about caring for others, growing as a community and trying to make a difference. So we just want to take the first step of handing over the first balloon to you. I am sure you all will join us in this journey if you believe in our philosophy and approach.

My brother was obsessed with cars when he was a kid. He built a huge collection of model cars of various colors, shapes and categories over many years of his childhood. At some point of time, he even started designing newer models and thought of selling them to the toy companies so that they can actually make those cars!

I must admit, he did an awesome job as a designer when he was a third grader!

However, when we both look back now, we realize that there was one fundamental problem with those designs. They were merely sketches, not designs. They were more arts than science. They were about how he sees the cars, instead of how the cars work. There were absolutely no measurements about anything that were specified in the design.

Based on my experience of interacting with various software testing teams, I feel the same way about the test cases creation process. The creation and review of test cases are still very subjective and thus qualitative. They are more art than science. No one cares to quantitatively measure the quality of test cases, as part of the review process, before they are actually executed. The only time the quality of test cases are assessed, unfortunately, is when there is a bug found in production and management has doubt about how much was the test coverage when the application was tested in lower environments.

Very recently I co-founded Testing Algorithms, LLC. with a friend to help organizations with an automated test case design algorithm that makes the test case creation a scientific, logical, quantitative and automated process, as opposed to artistic, subjective, qualitative and manual one.

​Participants:
 
Participants of the survey were mostly:
1.      Part of dedicated testing teams (92% of the organizations)
2.      Following Agile methodology (89% of the organizations)
3.      People who are not in managerial positions (77% are Test Analysts/Leads)
4.      Located in Asia & Europe (76% of the participants)
5.      From bigger organizations (67% has >200 employees, 59% has >500 employees)

Observations:
 
6.      Requirement Analysis (64%) is a bigger area of improvement than Automated Test Execution (61%).

7.      Test Data Creation (45%) is a bigger area of improvement than Status Reporting (38%).

8.      Test Case Creation (42%) is a bigger area of improvement than Manual Test Execution (32%).

9.      Defect Management (26%) is not a big problem in most of the organizations.

10.   Combining points 6-9, it looks like the test execution, defect management and status reporting process has been streamlined quite a bit. However, requirement analysis & test planning is emerging as the primary pain area in many organizations.

11.   Only 41% organizations use Model Based Testing tools for test case creation and only 36% use a Test Data Management tool. This confirms our hypothesis in point 10.

12.   Automated Regression Testing has been adopted by most organizations (80%), but still 87% participants feel that testing takes more time. This also confirms our hypothesis in point 10.

13.   Even if 80% organizations are using Automated Regression Testing, 74% says that testing phase exceeds their budget. This indicates either or both of the following:

a.      Automated Regression Testing didn’t reach its break-even point;
b.      Automated Regression Testing is not cost effective.

14.   Most importantly, 92% of organizations fail to prevent defect leakage to the next testing environment, in spite of spending more time, spending more money and utilizing strong test management, test management and automated testing processes.

Conclusion:
 
The primary problems are in requirement analysis and test design processes. These continue to be manual processes since time began and its quality is believe to be dependent on human skills vs. automated techniques. There are various tools and techniques that are currently available that can make requirement analysis and test design much faster and better. It’s high time testing organizations start moving towards the direction of automated test case design. In fact, I predict automated test case design to be one of the most demanding topics in software testing area in the next decade.

​...unless we become artists or inventors!

Let's look at the history of corporate jobs.

In 1960's and 1970’s, typewriting and shorthand was one of the niche skills worldwide. This is because the job of a secretary was in high demand in all industries. Microsoft Office killed that job in 1990's.

The story is similar for the telephone operators.

Factory workers were replaced by machines across the world as well. A couple of years back we visited Hershey's and took a tour inside the chocolate factory. We were amazed by how commercially packaged chocolate bars are being produced starting from cocoa beans without a single human intervention.

The latest inclusion to the list is the self-driven cars. I wonder if drivers all over the world are going to lose their jobs in next few years or decades.

I am a selfish person and have a habit of connecting everything I say with what I do. So, please bear with me...

If we look at the historical pattern, any mechanical, logical and analytical jobs have slowly been taken over by machines over the years.

But, fortunately, machines have two big limitations.

1. They can't connect with human emotions to produce arts; and
2. They can't invent other machines.
 
Now, what does this mean to the software testing world? (Here you go!)

Very simple. Manual testing jobs are going to go away in a few years or decades. And this is because this is a mechanical, logical and analytical job.

What is the way out then?

Be an artist or an inventor (or both). Keep in mind the 'survival of the fittest' theory of the great Charles Darwin.

You might argue that not everyone is an artist or an inventor.

If I could borrow some thoughts from the great author and public speaker Seth Godin, I would ask you, how do you know that you are not an artist or an inventor? Did you try to be one? Thomas Alva Edison was not an inventor until he invented the light bulb! Mark Twain was not a writer before he wrote Tom Sawyer!

You don't have to be Thomas Alva Edison or Mark Twain. But you can start making small differences in the work that you do, however insignificant that might be. A difference that only a human can make. That will make you an artist. That will make you an inventor.

1602:
​​
It is a great day in William Shakespeare's life! He just finished writing “Hamlet”! Now it's time to start rehearsing!

But he needs so many copies of the script for all the actors!

So, everyone sat down for months to have a copy of the entire play for themselves.

Some of them were really struggling to read the author's handwriting...
 
1921:

Charlie Chaplin just finalized the screenplay of “The Kid” a few days back.

He is considering buying a typewriter to copy the script so that it is legible.

It is much faster too. He only has a few weeks before the shooting starts.

But a typewriter is very expensive these days and he also needs to find someone who can type...
 
1952:

Mr. Chaplin found a better way!

A typewriter uses something called Carbon Paper that makes multiple copies in one round of typing.

He is both excited and nervous about “Great Dictator”! The shooting is starting next week.

He wishes he could start before the Fuehrer knows about it!
​
He is all ready, except for the scripts...
 
1982:

Steven Spielberg just handed over the full screenplay of “E.T.” to his office boy.

Available copies of the script can't be used because he had to make a few changes in the last minute.

He needs the photocopies positively by tomorrow morning!

John Barrymore's 6-year old granddaughter is auditioning tomorrow after lunch...
 
2004:

Fortunately, this time Steven wrote the entire screenplay for “Catch Me If You Can” in Microsoft Word!

Both Tom and Leonardo are very demanding actors! Why on earth do they need the script in next half hour??!!

Oops!!! The printer is out of paper...

A verb is a word in a sentence that expresses an action, an occurrence, or a state of being.

A noun is a word that is the name of something and is typically used in a sentence as subject or object of a verb.

A subject noun is a noun that performs the action of the verb in the sentence.
​
An object noun is a noun that receives the action of the verb.

A few years back I learned this technique from a great Business Analyst I worked very closely with. It was very early in my career as a tester at that time. She taught me that whenever we gather, analyze, interpret or elaborate business requirements of a software applications, we should always start with these three words (or phrases): a verb, a subject noun and an object noun, to identify all the business functions that the application is supposed to perform. Once we have the list of all the functions documented in this form, we should then elaborate on the individual functions to come up with the details of the requirements and/or test cases.

Since then, I has been tremendously helpful for me in writing good quality test cases!

Let's take the example of the deposit function in an ATM machine. 

Here, the verb is 'deposit'.

Now, who deposits and what is deposited?

Clearly, an account holder deposits.

But which account holders? Checking Account? Credit Cards? Mortgage? Domestic? International? 

Here comes the need for elaboration. We should document very explicitly what types of account holders can use this function.

Similarly, what is being deposited? The answer is simple. Money.

But, in which form? Cash? Check? This again needs elaboration.

This process can be extended to identify all business functions of the ATM machine.

Note that, this technique is not specific to any methodology and can be used in Agile, Waterfall, Iterative or anything similar.

Here are a few examples:

Account holder withdraws money
Account holder views balance
Guest opens account
Administrator prints report, etc.

Over the years, this technique has been of great help to me as a tester to identify all positive and negative test conditions.

I was buying a car for my wife a few years back. We did a lot of homework and decided on the brand, model and color of the car that we wanted. While entering the showroom, we were pretty sure about what exactly we were looking for and we thought it would be a very quick transaction.

Then the dealer started giving us hundreds of options beyond the brand, model and color of the car. Tilted glasses or not, moonroof or not, alloy wheels or not, and the list went on and on. The one that caught our attention was, whether leather seats or not. Although the earlier questions seemed more or less straight forward, this question was a tough one for both of us.

After a long analysis, we finally decided to go for it!!!

And then came this additional information from the dealer. For the mid west weather, we would definitely need heated seats if we want leather ones.

Eventually, we didn't buy the one with leather seats because, considering the additional value we were getting, this double investment was not worth to us.

Sometimes, in software testing world, we do not realize that we make similar uninformed choices. We first buy licenses of expensive test design, test automation or test management tools and then we pay a lot of money either to hire skilled people or to train existing associates so that the tool can be used in the organization; without even analyzing whether this double investment is worth the additional value or not.

Testing Algorithms, LLC. offers a simple automated test case design solution that doesn't need any specialized and technical skills that a Model Based Testing tool needs. Even a non-technical person can use this methodology to create better test cases 10x faster.

Visit www.testingalgorithms.com for more details.

Do you remember the hilarious scene in the movie 'Modern Times' where a machine was being demonstrated to feed the factory workers? This was one of my favorite scenes when I was a child.

Now, in 2016, we embraced automation in every parts of our lives. But still we laugh our lungs out every single time we see that scene.

Why?

We invented very simple machines, like a dishwasher, to very complex ones, like a GPS. So why do we have problem with a feeding machine?

In the movie, it was rejected as it failed to work because of technical problems. But I am sure, that is not the reason why no one ever tried to build and sell it since then. We sure had advanced technologies that would have made this machine perfect. Then why?

Here are my thoughts.

Historically, we automated processes that had a definite starting point, definite end point and some very definite steps in between. Like a washing machine, or an electric fan. None of these machines needed to make any decision in the process. 

Then we gradually learned how a machine can make logical decisions based on the information supplied by its user. Like an ATM machine.

Now a days, we talk about artificial intelligence where a machine knows how to gather information and make decisions based on that. Like a GPS.

But, the point is, we do not want to automate human choices and preferences! We do not want to automate the process to determine what to cook for dinner or what to wear in a party! And, we certainly do not want to automate eating!

We recently started Testing Algorithms, LLC. that offers an automated test case design solution for software applications. We started this, because we believe that test case creation is not about human choices and preferences; rather, it is more about taking logical decisions based on available information.

​There are, of course, exceptions to this common man philosophy. If you try to sell a NASCAR driver a self-driven car, do you think they will buy it?

In the early age of computers, when I was growing up, one of my uncles decided to install a computer in his existing small business. He took some basic computer courses on emails, word editors, spreadsheets etc. and started using the computer in his daily operations.

In no time, he started seeing the benefits of a computer over papers and slowly became an expert in spreadsheets. He often used to teach me and my cousins the basics of computer. We named him 'computer uncle' and used to hide from him whenever he was around to avoid those sessions.

One day, after a few years, when I was visiting him, I saw him working in front of his favorite computer. There was a big excel worksheet, full of numbers, open in the computer screen and he was doing some calculations using those numbers with his pocket calculator!

Surprised, I asked him, "Do you know that you can do the same calculation using the excel formulas? You won't have to enter the numbers all over again in your calculator." He calmly replied, "Yes, I know that. But I don't trust excel!"

I didn't know what to say!!! He continued, "We should not trust machines for these types of human work. They might be fast, but they are inaccurate."

A couple of decades went by. In the mean time I started working in software industry and gained some experience in software testing. Now I sometimes feel that most of us are still like my computer uncle. We still believe that some processes are still very human and they couldn't and shouldn't be automated because that will produce inaccurate results.

I and my friend, Palash, recently created Testing Algorithms to try to make a difference in the requirement analysis and test design process, which is still believed to be a human process. Our website is www.testingalgorithms.com.

P. S. When I called computer uncle a few months back to inform about Testing Algorithms, he was in the middle of a chess game with his computer. 

A software application is very similar to a building.

In a building, a person can visit various rooms and use connectors and doors to go to other rooms. Rooms are of different colors and sizes and have their own purposes and personalities. Some doors are even locked and only those having the right authority have the keys. 

At the time of building one, a story-by-story approach is usually taken by the builders. They first build one story and then they build the next one on top of it.

(Do you see the similarity? In Agile software development we call them user stories!!!)

In the example of the above picture, the builders developed each story perfectly on top of each other. They put every effort to make sure that each individual story is as per the customer's expectation.

But, in this whole process, what they failed to do was, to check the foundation of the entire building after each story was built! No one even noticed that there was an increasing problem with the center of gravity of the building!!!

The case of Agile software development is very similar. Sometimes techniques like paired programming and TDD make the testers lose focus on the bigger picture.

This is why Testing Algorithms recommends the evolution of a regression test suite parallel to the software under development. The regression suite should be enhanced, prioritized, executed and maintained as part of each sprint.

Do we know how a fisherman estimates the total number of fishes in a lake?

Here are the steps they follow:

1. They add certain number of fishes, uniquely marked, to the lake.
2. After a few days, they come back and capture a fixed number of fishes.
3. In their bucket of captured fishes, they determine the proportion of marked fishes.
4. They repeat the same steps for multiple times.

Let's use the following notations to explain this mathematically:

F = Total number of fishes in the pond
M = Number of marked fishes added to the lake
f = Number of fishes captured in a single iteration
m = Number of marked fishes captured in a single iteration
x = Estimated proportion of marked fishes in one iteration.

Clearly, for a single iteration, x = m/f. 

Therefore, an estimated number of fishes (i.e., F) in the pond could be determined using the below equation:

M/(M+F) = m/f

Or, in other words, F = M*(f-m)/m.

However, if we consider n trials (i.e., iterations), then the entire problem boils down to an statistical estimation problem for Binomial distribution where x follows Binomial(n, p = M/(M+F)).

Note that in the above picture, n = 20.

But the question is, how does this prediction method apply in the context software testing?

Well, the Business Analysts and/or developers can intentionally inject some defects without informing the testers. Then, by monitoring the defects being incrementally found by the testers and applying the above formula, total number of defects might be predicted.

Thoughts?

Believe it or not, zero!!! It's only the representation of test cases that changed! 

Testing is all about simulating a user's experience. And test design is all about planning to simulate as many variations of user experiences as possible in a limited time and effort.

In last two decades, the simulation of user experiences evolved to more mature processes over time. It started with record and playback and passed through reusable functions, data-driven frameworks, keyword-driven frameworks etc. The latest entry in this list is codeless automation.

But what had changed in test design? Nothing!!! It is still the same process of testers thinking about various test conditions and documenting, as it was 20 years ago. With the advent of Agile, however, the format of test cases got changed. In present days, we mostly use the given-when-then format test cases that can be easily integrated with the test automation tools.

In early years, the purpose of test case development was to make sure that a stranger, with no knowledge of software testing, should be able to execute the test cases and find defects. Later Agile brought a tester's role much closer to the role of a Business Analyst or a developer. But did that make the test design process capable of finding more defects?

The question is, instead of focusing more on 'how', shouldn't we address the 'what' part of the problem first? Visit www.testingalgorithms.com for similar posts.