Blogs

Self-maintaining Automated Test Scripts

4/3/2017

I grew up in a big city in India. A hot one!

Air-conditioners were not that uncommon in my childhood. However, there was one problem. It used to get very cold inside whenever the outside temperature dropped in the middle of the night. One had to wake up, get up and adjust the temperature before going back to bed.

Then the age of remote controls came. And the air-conditioners started coming with remote controls so that one wouldn't have to get up to adjust the temperature. However, they still had to wake up!

The evolution of test automation is very similar to the air-conditioners.

In our earlier days, we had record-playback which had to be adjusted every time there was a change in requirement (just like the outside temperature). Then came automation frameworks which made the maintenance of automation scripts easier. Just like the remote control.

But still a tester needs to wake up and adjust the inside temperature manually!

Then a fundamental concept of Physics changed the whole air-conditioning industry.

Thermostats!

Now we don't have to even wake up for a temperature adjustment!

We, at Testing Algorithms, are working on a Thermostat for test automation. We were able to link test automation scripts directly with the business requirements so that if one changes, the other changes automatically.

If you are interested to see our solution, watch our video.

If you want to read our previous article on this topic, please read this post.

If you are looking for more information, please contact us.

What Testing Algorithms is not?

1/3/2017

Testing Algorithms is...

... neither a solution for automated test execution, nor for test management. However, it easily integrates with both types of tools.

... neither associated with, nor a subsidiary of any other company. Me and my friend, Palash, came up with this solution based on our years of research on requirement analysis and test planning and formed Testing Algorithms in October, 2015.

... neither a product company, nor a service company. Our business model is based on teaching, training and consultancy to individuals and organizations.

... neither a non-profit, nor a charity organization. However, we believe in helping the testing community and therefore offer our automated test case design solution to our believers against return gifts.

Why cars have brakes?

12/15/2016

This is an Agile question. Why cars have brakes?

The purpose of a car is to move people and things from one place to another. And the performance of a car is generally measured by parameters like how fast it can go and how quickly it can pick up speed.

But, if that is only the case then why just having a gas pedal is not enough? Brakes waste both time and gas anyway! Why cars have brakes then?

I think we all know the answer. It's for safety.

And I am actually talking about documentation in Agile.

Document is a waste unless it is necessary for safety.

But when documentation becomes necessary for safety?

Well, I have seen situations where everyone perceived a user story (and its acceptance criteria) completely differently. It ended up in a car accident because it didn't have a brake!

May be this is one of the reasons why most scrum teams are very stressed in today's world.

Agile Manifesto says "Working software over comprehensive documentation". And most of us tend to ignore the word comprehensive, consciously or unconsciously.

What would make this easier for everyone is, if we had a process to create enough documentation (just to bring everyone on the same page) that doesn't waste any time and effort.

For example, if we had a process where, on clicking a button, optimum set of test cases in the organization-specific format, requirement traceability matrix and standardized use cases and process models are instantly (i.e., automatically) created from the user stories (written in a specific way), it would have made things much easier.

And this is exactly what we, at Testing Algorithms, are trying to accomplish based on our years of research. Please note, our purpose is to help the software development community and our solution is a gift to all. However, you can choose any amount (including zero) for giving us return gift, if our solution helps your organization.

Please register if you are interested to try something new.

Our first eBook is available!

10/6/2016

We, testers, test software applications to find bugs. We are called bug-hunters. But if we take a step back and look at the mirror, we will see ourselves as someone much more important than mere bug-hunters. We are the trust-builders. Just like the bomb disposal squad.

When a bomb is disarmed, civilians are no more scared to use the street where the bomb was originally found. That's because people trust the bomb disposal squad and the techniques they use to disarm it. Well, in a similar way, the business users trust the testers and their testing processes at the time of using the software application in their day-to-day business.

But the question is, how do we disarm a bomb? How do we ensure that it is not dangerous anymore? Is it a science or an art? Can we increase the reliability and efficiency of a techniques we use? Can we disarm it faster? Can a robot do it?

This book is the outcome of the philosophical investigation to find answers of all these software testing questions. To read this book, a little knowledge of a software development process is helpful but not necessary.

**The book is available in amazon.com:
https://www.amazon.com/dp/B01LXRTVDW/ref=rdr_kindle_ext_tmb

The book is available in testingalgorithms.com for our users:
https://www.testingalgorithms.com/ebook.html

Note:
All users of the automated test case design solution of Testing Algorithms get the eBook for free. They can choose to give return gifts based on the perceived value of the book.

Link to register as an user of Testing Algorithms:**
https://www.testingalgorithms.com/registration.html

We don’t have a price tag in our product...

8/1/2016

I heard the following story a few months back.

A wise man, living in a small town, invited all residents of his hometown in his house. There were around a hundred people who showed up on the scheduled day. He had one of his bigger rooms already filled up with hundreds of balloons. Each balloon had name of a guest written on it.

He asked everyone to enter the room and find the balloon that had her name on it in three minutes. He declared a handsome prize for those who would be able to find theirs.

He started his stopwatch and everyone started rushing to look at each and every balloon to check if those had their names. When the watch stopped after three minutes, unfortunately, everyone came out of the room without their own balloons. Three minutes was too short a time for this exercise!

Then the host, the wise man, asked everyone to repeat the exercise, but this time he showed everyone a trick. The trick was to pick up a balloon, read the name on it, find the person and hand over the balloon to her, then pick up the next balloon and repeat the same steps.

The exercise was repeated and this time everyone was able to come out of the room with their own balloons within three minutes.

During our journey of creating our product and our company, we realized that most companies follow the first approach to sell their products or services. However, we want to take the alternate approach, as shown by the wise man of the story. Our automated test case design solution will be ready for everyone, who believes in our approach and philosophy, in a few weeks, for free.

However, we are a small company with a few people who want to make a living by doing what we believe in and what we love to do. And we understand that the value perception of our solution would be different for different users. Therefore, we will have a ‘pay what you want’ provision where anyone can contribute any amount according to how much value they see in our methodology and how much this is helping in your test design. We would appreciate any contribution and we promise that your contribution will be spend only on enhancing the product and customer experience.

One more thing.

We will keep providing the support and answering all your questions via email. We plan to have a feedback mechanism so that our solution can be improved based on your suggestions. We also plan to have a LinkedIn group created only for our users so that all communications are transparent to everyone belonging to the user community. However, if anyone needs any training or consultation about our methodology or product, or any other verbal assistance, we will charge for our times. I hope that is reasonable for a small startup company with no funding.

This is not a marketing gimmick. We, the founders of Testing Algorithms, believe that entrepreneurship is about caring for others, growing as a community and trying to make a difference. So we just want to take the first step of handing over the first balloon to you. I am sure you all will join us in this journey if you believe in our philosophy and approach.

Test design is a science, not an art…

7/25/2016

My brother was obsessed with cars when he was a kid. He built a huge collection of model cars of various colors, shapes and categories over many years of his childhood. At some point of time, he even started designing newer models and thought of selling them to the toy companies so that they can actually make those cars!

I must admit, he did an awesome job as a designer when he was a third grader!

However, when we both look back now, we realize that there was one fundamental problem with those designs. They were merely sketches, not designs. They were more arts than science. They were about how he sees the cars, instead of how the cars work. There were absolutely no measurements about anything that were specified in the design.

Based on my experience of interacting with various software testing teams, I feel the same way about the test cases creation process. The creation and review of test cases are still very subjective and thus qualitative. They are more art than science. No one cares to quantitatively measure the quality of test cases, as part of the review process, before they are actually executed. The only time the quality of test cases are assessed, unfortunately, is when there is a bug found in production and management has doubt about how much was the test coverage when the application was tested in lower environments.

Very recently I co-founded Testing Algorithms, LLC. with a friend to help organizations with an automated test case design algorithm that makes the test case creation a scientific, logical, quantitative and automated process, as opposed to artistic, subjective, qualitative and manual one.

Losing jobs to machines is inevitable...

7/11/2016

...unless we become artists or inventors!

Let's look at the history of corporate jobs.

In 1960's and 1970’s, typewriting and shorthand was one of the niche skills worldwide. This is because the job of a secretary was in high demand in all industries. Microsoft Office killed that job in 1990's.

The story is similar for the telephone operators.

Factory workers were replaced by machines across the world as well. A couple of years back we visited Hershey's and took a tour inside the chocolate factory. We were amazed by how commercially packaged chocolate bars are being produced starting from cocoa beans without a single human intervention.

The latest inclusion to the list is the self-driven cars. I wonder if drivers all over the world are going to lose their jobs in next few years or decades.

I am a selfish person and have a habit of connecting everything I say with what I do. So, please bear with me...

If we look at the historical pattern, any mechanical, logical and analytical jobs have slowly been taken over by machines over the years.

But, fortunately, machines have two big limitations.

1. They can't connect with human emotions to produce arts; and
2. They can't invent other machines.

Now, what does this mean to the software testing world? (Here you go!)

Very simple. Manual testing jobs are going to go away in a few years or decades. And this is because this is a mechanical, logical and analytical job.

What is the way out then?

Be an artist or an inventor (or both). Keep in mind the 'survival of the fittest' theory of the great Charles Darwin.

You might argue that not everyone is an artist or an inventor.

If I could borrow some thoughts from the great author and public speaker Seth Godin, I would ask you, how do you know that you are not an artist or an inventor? Did you try to be one? Thomas Alva Edison was not an inventor until he invented the light bulb! Mark Twain was not a writer before he wrote Tom Sawyer!

You don't have to be Thomas Alva Edison or Mark Twain. But you can start making small differences in the work that you do, however insignificant that might be. A difference that only a human can make. That will make you an artist. That will make you an inventor.

Automation is a journey, not a destination...

7/6/2016

1602:

It is a great day in William Shakespeare's life! He just finished writing “Hamlet”! Now it's time to start rehearsing!

But he needs so many copies of the script for all the actors!

So, everyone sat down for months to have a copy of the entire play for themselves.

Some of them were really struggling to read the author's handwriting...

1921:

Charlie Chaplin just finalized the screenplay of “The Kid” a few days back.

He is considering buying a typewriter to copy the script so that it is legible.

It is much faster too. He only has a few weeks before the shooting starts.

But a typewriter is very expensive these days and he also needs to find someone who can type...

1952:

Mr. Chaplin found a better way!

A typewriter uses something called Carbon Paper that makes multiple copies in one round of typing.

He is both excited and nervous about “Great Dictator”! The shooting is starting next week.

He wishes he could start before the Fuehrer knows about it!

He is all ready, except for the scripts...

1982:

Steven Spielberg just handed over the full screenplay of “E.T.” to his office boy.

Available copies of the script can't be used because he had to make a few changes in the last minute.

He needs the photocopies positively by tomorrow morning!

John Barrymore's 6-year old granddaughter is auditioning tomorrow after lunch...

2004:

Fortunately, this time Steven wrote the entire screenplay for “Catch Me If You Can” in Microsoft Word!

Both Tom and Leonardo are very demanding actors! Why on earth do they need the script in next half hour??!!

Oops!!! The printer is out of paper...

How much do we automate in life? And why?

6/22/2016

Do you remember the hilarious scene in the movie 'Modern Times' where a machine was being demonstrated to feed the factory workers? This was one of my favorite scenes when I was a child.

Now, in 2016, we embraced automation in every parts of our lives. But still we laugh our lungs out every single time we see that scene.

Why?

We invented very simple machines, like a dishwasher, to very complex ones, like a GPS. So why do we have problem with a feeding machine?

In the movie, it was rejected as it failed to work because of technical problems. But I am sure, that is not the reason why no one ever tried to build and sell it since then. We sure had advanced technologies that would have made this machine perfect. Then why?

Here are my thoughts.

Historically, we automated processes that had a definite starting point, definite end point and some very definite steps in between. Like a washing machine, or an electric fan. None of these machines needed to make any decision in the process.

Then we gradually learned how a machine can make logical decisions based on the information supplied by its user. Like an ATM machine.

Now a days, we talk about artificial intelligence where a machine knows how to gather information and make decisions based on that. Like a GPS.

But, the point is, we do not want to automate human choices and preferences! We do not want to automate the process to determine what to cook for dinner or what to wear in a party! And, we certainly do not want to automate eating!

We recently started Testing Algorithms, LLC. that offers an automated test case design solution for software applications. We started this, because we believe that test case creation is not about human choices and preferences; rather, it is more about taking logical decisions based on available information.

There are, of course, exceptions to this common man philosophy. If you try to sell a NASCAR driver a self-driven car, do you think they will buy it?

The man who doubted a computer's addition ability...

6/21/2016

In the early age of computers, when I was growing up, one of my uncles decided to install a computer in his existing small business. He took some basic computer courses on emails, word editors, spreadsheets etc. and started using the computer in his daily operations.

In no time, he started seeing the benefits of a computer over papers and slowly became an expert in spreadsheets. He often used to teach me and my cousins the basics of computer. We named him 'computer uncle' and used to hide from him whenever he was around to avoid those sessions.

One day, after a few years, when I was visiting him, I saw him working in front of his favorite computer. There was a big excel worksheet, full of numbers, open in the computer screen and he was doing some calculations using those numbers with his pocket calculator!

Surprised, I asked him, "Do you know that you can do the same calculation using the excel formulas? You won't have to enter the numbers all over again in your calculator." He calmly replied, "Yes, I know that. But I don't trust excel!"

I didn't know what to say!!! He continued, "We should not trust machines for these types of human work. They might be fast, but they are inaccurate."

A couple of decades went by. In the mean time I started working in software industry and gained some experience in software testing. Now I sometimes feel that most of us are still like my computer uncle. We still believe that some processes are still very human and they couldn't and shouldn't be automated because that will produce inaccurate results.

I and my friend, Palash, recently created Testing Algorithms to try to make a difference in the requirement analysis and test design process, which is still believed to be a human process. Our website is www.testingalgorithms.com.

P. S. When I called computer uncle a few months back to inform about Testing Algorithms, he was in the middle of a chess game with his computer.

A story of a simple solution...

6/14/2016

How much the test design process evolved in last 20 years?

6/8/2016

Believe it or not, zero!!! It's only the representation of test cases that changed!

Testing is all about simulating a user's experience. And test design is all about planning to simulate as many variations of user experiences as possible in a limited time and effort.

In last two decades, the simulation of user experiences evolved to more mature processes over time. It started with record and playback and passed through reusable functions, data-driven frameworks, keyword-driven frameworks etc. The latest entry in this list is codeless automation.

But what had changed in test design? Nothing!!! It is still the same process of testers thinking about various test conditions and documenting, as it was 20 years ago. With the advent of Agile, however, the format of test cases got changed. In present days, we mostly use the given-when-then format test cases that can be easily integrated with the test automation tools.

In early years, the purpose of test case development was to make sure that a stranger, with no knowledge of software testing, should be able to execute the test cases and find defects. Later Agile brought a tester's role much closer to the role of a Business Analyst or a developer. But did that make the test design process capable of finding more defects?

The question is, instead of focusing more on 'how', shouldn't we address the 'what' part of the problem first? Visit www.testingalgorithms.com for similar posts.

What is unique about Testing Algorithms?

5/24/2016

Struggling with software quality? Not having a dedicated testing team? Testing processes are not standardized? Many requirement defects found during testing?

Testing Algorithms is not a QA outsourcing company.

It is a research based organization that aims to improve requirement gathering and test design processes across industry, in collaboration with top Universities of USA.

We help you create the most effective & efficient testing approach that drastically reduces overall testing time and effort and increases test coverage using optimization algorithms!

Share your requirement documents to try out our innovative service for free.

Want us to sign an NDA? Contact support@testingalgorithms.com.

Is the software testing game over yet?

5/17/2016

"Its a machine, and you know that it is stupid!"

These were the exact words that Gary Kasparov said before his games with Deep Blue!

Do you think the same way about requirement analysis and automated test case design?

Try our unique solution at www.testingalgorithms.com before answering the question!

1140 test cases in 10 hours? Believe it!

3/9/2016

Testing Algorithms create high quality test cases using a proprietary automated process by analyzing the application behavior. This results in:

1. Complete overlap of test design with requirement gathering process, eliminating the need of having a separate test design phase
2. Reduction of test execution time and effort, as minimum number of test cases cover maximum requirements

In one of our recent projects, we were supposed to create complete set of test cases for an eCommerce website. The conventional approach of test design (used by other vendors) estimated a few thousands of test cases and more than a thousand hours just for manual test case creation.

Testing Algorithms created a total of 1140 test cases in Quality Center format (i.e., with detailed test steps, expected results and requirement traceability) covering all pairwise interactions. In addition to that, all business models (e.g., UML format Use Cases, Swim-lane Diagrams and Process Flows Diagrams including primary and alternate paths) were also created and shared with the customer in no additional effort, time and cost.

Most interesting part of the project was, those 1140 test cases (and business models) were created in just 10 hours.