Crucis Consulting Limited

Testing times

Daishik
24 Jun 2022
0

The third shortcut we'll look at in our bullish effort to gain time and reduce cost, is by forgoing or minimising quality assurance (QA).

What is QA? Basically, it is the practice of evaluating whether a product or service meets the intended qualitative and quantitative requirement. QA is met by tests of various types to evaluate various aesthetic, functional and non-functional considerations.

QA can take many forms, and when it comes to software not all developers or even testers have the skills to perform every type of test.

The most obvious would be functional testing: if I click X will it do Y? If I submit invalid data, will it stop me? Does it return my search results? Does it sort and filter as intended?

There’s also non-functional testing: how long does it take to bring back my search results? Is the application using too much memory or processing time? Will the server be able to cope with anticipated demand?

Consider security and penetration testing. Is the system resilient to common forms of cyber-attack? Might the system be leaking system details or customer data that could be appropriated by criminals?

Apart from the above technical aspects, what about testing look and feel, user interface and brand standards? Does the application meet the design objectives? Are logos, calls to action, and copyright messages clear and visible in the right places on screen? How does the application scale on different screen resolutions and devices? Is the application accessible to people with disabilities?

To test....

QA is not simply about finding bugs or pushing features to breaking point. Tests verify whether requirements have been met and ensure that the customer gets what they were promised.

Testing can identify potential issues that may have been missed during analysis and subsequent discussions. They may even indicate areas of improvement, security and privacy concerns, or guide the business to new features.

QA is not an optional bolt-on to the process of creating a product or feature - it is an integral part of it. In principle, good teams integrate QA into the development plan, and perform QA while they are developing, not just after.

When integrated into the development process, QA affords stakeholders an opportunity to catch issues and consider mitigations at an early stage: perhaps a previously modelled operation doesn't quite work as intended, or an alternative option becomes viable with a small adjustment. The team can use this information to rework or quickly change direction on the feature before going too far down the wrong path.

Or not to test....

You may (or not) be surprised to learn that despite all the above, in practice many organisations consider QA to be just another of those tedious, but somewhat discretionary exercises. One of those things that just need to be checked-off before delivery – after all, it's up to the developers to write bug-free code, right?

In any case, any really important issues will be discovered somehow during development or delivery. Won't they?

So as a maverick looking to cut some corners, how should you break the QA rule?

Many businesses schedule testing in the gap between completion of development and delivery. Not surprisingly, for a host of reasons (see the previous articles in this series) development runs almost up to the wire, shrinking the time to test the product.

When testing is finally done, and sometimes if it is done, it is rushed, uncoordinated and unstructured. The risk is that critical problems may not be discovered, and potentially valuable conversations and corrections never take place.

As a defence against total disaster, if you do intend to forgo or under-resource your QA, you must ensure your developers conduct QA whilst they're writing the code. (If you're really serious about "tearing up the rulebook", you'll also want developers to conduct analysis alongside the coding!)

You should know though, that developers frequently don’t have the skills or time to conduct adequate testing, and QA professionals (and analysts) often have a deeper understanding of the requirements, scope and user expectations than the coders.

So what happens when QA is not a core part of project delivery?

The most obvious outcomes are undiscovered errors (bugs) and missed requirements. I’ve noticed that even where found, issues may not get logged, triaged, or tracked. Even if you find yourself with essentially bug-free code, you may instead end up with incorrect or inadequate application hardware or platform resources, or security problems.

However, you'll be pleased to know that in most cases your gamble will pay off!

In my experience - provided you have a professional development team - it's very rare for the wheels to completely come off in a production environment due to poor QA.

It is also my experience that many organisations are proud of how they manage to keep their unique "Sellotape and strings" software projects going. They wear it as a badge, as though it were a testament to their status within their peculiar business niche or the uniqueness of their business model.

Privately they (the developers working in the software delivery teams) almost all fully acknowledge the consequences in bugs, live incidents, staff stress and retention, financial waste, and even reputational harm.

In fact, my experience is that organisations with really good, structured QA are the exception rather than the norm – even within companies that rely on bespoke software for a core part of their business.

So how do you manage the risks and still succeed? We’ll explore that in the next article.

Click on the links below for the other articles in this series

Tearing up the rulebook - We know our business is unique, so let's just tear up the rules and do everything in our own, unique way!
There is no Plan A! - Cutting back on discussions, workshops and planning of work, or even skipping them entirely, is an important shortcut that could reap dividends!
Analyse this! - A popular shortcut is to leap over the analysis phase, with the hope that issues will be easily resolved during implementation
Getting away with it - So you've cut corners and played fast and loose with best practices. Here's how you can still succeed in spite of the consequences!
The gods among us - The ultimate, and most effective solution to managing the fallout from shortcuts. But there is a price to pay!