Software Engineer’s Blog

Software Engineering weblog

Life cycle of a test case

An interesting question on “”States” of a Test Case” in Software Testing & Quality Assurance group in Linkedin. This got me reflecting back on my test management practices.

Following are my comments:

I presume that question is not limited to test case execution. In such case, my suggestion would be to consider what I call ‘life cycle of a test case’.

Some of the states in the life cycle of a test case would be:
1. Identified: Test case is first identified in consideration of various factors. At this stage, test case may have only the name and description.
2. Designed: Test case is further designed with details and elaborated with test procedure/manual test script
3. Design-reviewed: This is a state where test case designed is reviewed and accepted.
4. Executed: Test case is executed.
5. Result-reviewed: Result reported by a human tester or a tool may require moderation
6. Closed: Test cases no longer valid

As we progress, some test cases may get further refined into many related test cases and some may go obsolete overtime. Maintaining state and related information helps in effective test management

December 22, 2009 Posted by | software engineering, Software Quality, Software Testing, Testing | Leave a comment

Cheese is moving again!

Landscape of world around is changing drastically with innovations like Sixth Sense, Project Natal , and multi-touch man-machine interfaces. Software is opening up new frontiers in collaboration. What is more, Cloud computing is changing the deployment and usage model of software taking it more towards a service paradigm.

I believe, these augurs well for the world at large as it brings technology closer to people. For me? I have different hats. Good for most hats but not so good for others

It is fascinating for the curious onlooker in me. It is good for my business as it provides new possibilities in improving my customers’ business. It is good for software developer in as I get to work with something new, different and probably even unique. It is a new challenge for the architect in me, as I grapple with the unknown.

It alarms the tester in me. I know, I will be in the middle of it in no time at all; even before the world comes to terms with it. The world understand the need for architect, developer and user to learn and adapt but the tester is expected to get into the situation and has to come to terms with it on their own. For instance, what are the techniques that hold good? Where are my tools?

Well, we like it or not, the cheese is moving. I am diving in…. better to be proactive than reactive, right?

December 1, 2009 Posted by | Agile testing, Natural User Interface, NUI, software engineering, Software Quality, Software Testing, Surface, Testing, User Experience, UX | Leave a comment

A Magazine on Software Test Automation

Though software test automation has been around for quite some time by now, success with test automation is far and few. Test automation space is replete with marketing hype, technical jargon, and claims of “silver bullet” solutions. Adding to the woes are idiosyncrasies of test automation tools used. Scene is sufficiently confusing to an innocent but curiously enticing to the gullible.

In my experience, failures are partly due to i) expectations based on hype rather than facts from the field, ii) half-hearted but expensive approach to automation as an extension of manual testing and iii) lack of business focus. These are, by no means, unique to software test automation; we have seen this repeated many times over in the past

I think, it is time for the dust to settle with increased demand for quality in software. I have come across a magazine focused exclusively on automated testing.

I am glad to see such an effort; it is need of the hour. I hope this magazine would help in harvesting success patterns, promoting knowledge exchange, scientific spirit and objective analysis in test automation space.

August 27, 2009 Posted by | Functional testing, Software Quality, Software Testing, Test automation, Test Automation Architecture, Test Automation Framework, Testing | Leave a comment

Consortium for IT Software Quality

As software permeates every aspect of daily life and as we depend more and more on software in all walks of life, quality of software gets to be of utmost importance.

World has come a long way from the early perceptions of quality as a natural consequence of following a good process. Agile community has asserted umpteen times, reflecting in manifesto of software craftsmanship and agile manifesto, quality of software that add value to users in their context is far more importance than religious, and often dogmatic, adherence to process.

People, process and automation are three different, orthogonal but often inseparably interrelated, factors of what goes into making of a quality software. These form different dimensions that call for attention of a manager of software development project, and not conflicting goals as they are sometime made out to be. Managing software development project is not progressing on one dimension at the cost of another but working with a dynamic balancing of these dimensions through the course of project

Three dimensions of software quality

Three dimensions of software quality

It is interesting to note, in this context, emergence of a consortium focused on software quality. It is a step in the right direction. I hope that this would help in making life better for its users and customers

August 20, 2009 Posted by | Agile, Product Engineering, Software architecture, software engineering, Software Factory, Software Quality, Software Testing, Testing, Uncategorized, Unified Process | 1 Comment

Presenting success at RSDC 2009

It feels great to selected to presented at IBM Rational Software Development Conference 2009. Unfortunately, due to personal priorities, I would not be able to make it to the conference.

Success (QM15: Effective Test Automation of SAP Implementations) would be presented by my manager Vipin Kumar (Managing Director and Software Engineering Evangelist, Astra Infotech Pvt Ltd), who has played a crucial role all through for the success of this project

March 5, 2009 Posted by | Functional testing, IBM Rational, Rational, Rational Functional Tester, Rational Quality Manager, Rational Software Development Conference, RSDC, SAP Testing, Software Testing, Test automation, Testing | Leave a comment

Some observations on software testing and test automation practices

Following are my comments on “What do you think about the future of Software Testing?” in International Association of Software Architects group in Linkedin

Software testing was undervalued but no longer. Yes, there are people who undervalue software testing .

There is a tendency to overlook important of testing in immature software development environments. This typically happens in organizations or projects in a reactive (or worse, firefighting) mode. Immediacy of business imperatives in such cases being to churn out code (as if quality is of someone else’s concern….. An interesting thought in this line ).

Testing is an integral part of any development activity, and hence an integral part of software development as well.

Test driven development is a great way forward, it brings focus on testing quite upfront and makes it almost inescapable

I agree with the bottom line from Stephen with a minor refinement: Adherence to craftsmanship and good coding practices, plus the use of a good automation tool. I would also like to add agile values and principles to the list.

Refinement being “good automation” rather than “good automation tool”. What we need is automation, and tool is incidental. Purpose could also be served by in-house automation effort. In-house automation could also result in a tool. I am making refinement as purchase of tool generally comes with expectation of a “silver bullet”

February 4, 2009 Posted by | Agile, Agile testing, Software Quality, Software Testing, Test automation, Testing | Leave a comment

Testing Alone Doesn’t Guarantee Quality

Following are my comments on “The Application Quality Dilemma: Why Testing Alone Doesn’t Guarantee Quality?” in Software Testing & Quality Assurance group in Linkedin

A good quality software cannot be produced by processes and tools; it can only be done by people. Hence, software development is essentially a teamwork.

Process, standards, methodologies, best practices and tools are only enablers.
It is naive to assume that testing alone can guarantee quality software. After all, examinations alone does not make a great student come out of school/college!

Testing is essential but one of the many such essential activities that goes into making a quality product; not just software.

I agree with above statements:
Testing is a function that the business cannot do without

It is important by the companies to acknowledge that testing is a critical part of the SDLC… that should be done with clearly defined responsibility, authority, resouces and ownership

Test tools alone does not guarantee anything other than a cost to the company… I agree that vendors have contributed to the testing tools but I believe unrealistic expectations are also part of the problem…

This is something I had been working with over more than 7 years now and, hence, close to my heart:

* Expectations, misunderstandings and disappointments related to test automation is common in many first time automation efforts. Actual upfront effort involved is hardly realized upfront by novices

* Tool is only a means for an end; end being test automation which in turn is for better productivity (sometimes misinterpreted as elimination of human effort), repeatability, predictability and control

* One of the first tasks in my test automation assignments has been setting the expectations right

* What is being automated is testing, and not test development

* Just as a development environment helps a programmer in development of program (but does not develop program replacing programmer), testing tool can help a tester develop test (does not develop test replacing tester)

* I have not seen any test automation (or any automation project, for that matter) successful without setting the expectations upfront

Tailpiece: A typical (read “failed”) test automation project reminds me of a machine to help men eat in the Charlie Chaplin movie “modern times”… We seem to have invented a software equivalent in test automation


February 4, 2009 Posted by | software engineering, Software Quality, Software Testing, Testing, Uncategorized | Leave a comment

Are Developers good testers… and can test better than a QA guy…??

Following are my comments on “Are Developers good testers… and can test better than a QA guy…??” in Software Testing & Quality Assurance group in Linkedin

I continue to be a designer, programmer, and tester. My experience is:

Coding as a mix of translation, derivation, and synthesis; essentially a creative and optimistic track. Testing is an act of what I call a “constructive destruction”.

Slips from development are not, typically, intentional. My code is my baby and I would like it be the best… this is also cause, at times, for developer-tester chasm; question on my baby is considered as question on me.

Human mind is at work and it has limitations. Human mind works best on terrain it is familiar with. You train it on coding; it will adapt to that. You train it on testing; it will adapt to that. It is difficult to be both at the same time.

Again, typically programmer works on smaller unit, whether he/she is aware of and involved in larger picture or not. Focus of testing, therefore, from developer is restricted to that unit (or a similar unit from peer). This is and needs to be done

Testing of the whole software is different from testing individual unit. Perspective now has to change to whole system and its desired business value. Typically difficult for a person to be both at the same time.

If trained well, one person can do both, in two different points in time but details, perspective and approach needs to change

January 8, 2009 Posted by | Agile, Agile testing, Software Testing, Testing, Uncategorized | Leave a comment

A Tester’s Observations in an Agile Environment

Following are my comments on ‘A Tester’s Observations in an Agile Environment’ in Software Testing & Quality Assurance group in Linkedin

I have seen successful and failed agile environments. Problems occur when fundamentals are compromised.

Agile approach has put focus back on individuals, rather than process/documentation; quite rightly so.

Therefore, success in agile approach depends on motivation (very critical) and skill levels of individuals involved, team synergy and agility, and very short feedback/correction cycles. Quite ironically though, I have seen in many such projects go on downward spiral with process (I do not mean documentation here) getting ritualistic and viscosity setting in the team (may be, a human tendency). This happens over time, as project complexity increases, defeating the very foundation on which agile approach is built. Given the above, I consider effect on testing is natural and incidental, though disastrous; meaning, root of the problem lies elsewhere. I consider this tendency fatal, unless immediate remedial steps are taken.

I had been asking a stock question to many agile teams that I have come across. Question is, does you testers ACTIVELY participate in the meetings. On probing (and, only on probing), it comes out that testers participate but participation is rather ritualistic. They do not dare to ask uncomfortable questions. It is important to ask uncomfortable questions, even if sugarcoated, and get answers to those questions. After all, no one else but only testers can claim to know the software as it is (rather than, what it should be/have been), its shortcomings/pitfalls, testability issues

It does not matter to me, as a tester, whether answers (to the uncomfortable questions that I ask) come to me as documents or it comes from discussions but if answers not coming from project team, irrespective of methodologies, project is running on dangerous track.

Then, it is time for a serious jolt; a jolt from above. If jolt does not occur naturally, it needs to be made to happen. In my experience, I find that someone, somewhere is always listening. But we need to set alarm bell ringing!

January 8, 2009 Posted by | Agile, Agile testing, Functional testing, software engineering, Software Testing, Testing | Leave a comment

A retrospection!

Looking back, last two years have been personally very taxing. Specifically, 2007 was traumatic personally. Though some of the traumatic experience continued into 2008, it was more of recovery personally and a time for change and consolidation Professionally. And, it was a mixed bag from the world around me.

World around me:
Chandrayan and India’s achievements in Beijing Olympics after a long while are on top of my mind on a positive note. Yet another unforgettable development is a Sister Alphonsa reaching sainthood. Happiness of these, however, marred to some extent by Mumbai bomb attack and global economic recession.

My cousin Jayan Varma became world’s fastest percussive bassist. On the negative, my father’s health suffered serious jolt. By god’s grace, timely intervention and good medical care, he has recovered to a great extent. May be unrealistic to expect at his old age but, as a son, I expected him to be ever active. Now, he along with my mother require constant attention. I am yet to figure out what best I can do for them.

2008 was also a year when I could connect back many of my old friends. Sunil, Ashok, Suresh, Romy, and Narayanan from my college days, and my friends Arun, Unnikrishnan, Madhavan, Zacharia, Varughese and Vaideeswaran from days of my first job. I could also connect to Amit and Shanmugham from CMC Ltd, Hyderabad.

This was also a year when Mr. Naresh Kumar Reddy passed away. I was shocked to hear that sad news. I was reporting to him during later part of my tenure at CMC Ltd. I learned quite a bit of management lessons working with him. Time has to say how effective I was in their application. Pray to God that his soul may rest in peace.

On professional front, in 2008, I have worked closely with Vikram Sarabhai Space Center, Trivandrum (Ministry of Defence, Government of India), and Weapons and Electronics Systems Engineering Establishment (Ministry of Defence, Government of India), Delhi helping them adopt software engineering best practices, methodologies and tools. As a departure from the past, I had started traveling outside India on business, with a visit to Qatar in October 2007. This was followed by visit to Saudi Arabia in March-April 2008 and a visit to Nigeria in June-July 2008. In the meantime, I was working for a customer in Germany remotely from India for test automation of SAP implementation. I visited Germany, by November 2008, to commence yet another project on test automation of SAP implementation.

Overall, I am happy looking back as indicated in old farewell note to Rational that I had written in 2002, having achieved what I had set out to.

My professional high in 2008 were:
1. Publishing case studies of success from India, a result of hard work in the past
2. Paper presentation at Rational Software Development Conference in Munich, Germany
3. Publishing case studies of success in test automation of SAP implementation in Germany
4. Release of Ready for IBM Rational Software plug-in Astra Model Creator and Astra Test Automat, with best practice compliance

2008 was also a year when I could connect back many of my old friends. Sunil, Ashok, Suresh, Romy, and Narayanan from my college days, and my friends Arun, Unnikrishnan, Madhavan, Zacharia, Varughese and Vaideeswaran from days of my first job.

I shall write on what I look forward to in 2009 and further in my note tomorrow

My favorite song for the year 2008 reflection (movie: Aap Ki Kasam):

December 31, 2008 Posted by | software engineering, Software Quality, Testing, Uncategorized | Leave a comment