A Bug vs QA

Over on IPVM’s LinkedIn group there’s a discussion about compensation if a client encounters defective software or hardware. Vincent Tong brought up a point which was readily dismissed as a meaningless, semantic discussion – he pointed out that you need to discriminate between QA problems and bugs.

Here’s my defense of Vincent.

The term “bug” comes from the days when little critters – real bugs – would short electrical connections and cause machines to stop working. It’s a funny mental image, and so when poor soldering or routing on boards caused devices to malfunction, it was also called “bugs”. Naturally, programmers also wanted to blame someone else, so we decided to call mistakes “bugs” too.

Now, plenty of companies have experienced faltering, physical circuits. This week I “re-flowed” an old PS3. What happens is that the soldering between the chips and the motherboards come loose, to re-flow you melt the solder-balls ever so slightly to make them re-connect, the re-apply thermal grease and re-assemble. The issue is not entirely uncommon, but the question is – is it a bug?

To the end user, the semantic discussion is useless – the damn thing won’t work, but to manufacturers the distinction is extremely important. If you want to have a symbiotic relationship with your manufacturer, you should know, and appreciate, the difference too.

When people ask me if our software will work under XYZ conditions, I have to make an educated guess. First of all, the conditions are never described fully – did you specify the firmware of the camera, the brand and firmware of the switches, the cable lengths, the OS version, the language settings on the OS, the EFI version, the network adapter and so on. You don’t. And nor do we test every single permutation of hardware and software out there – it would simply be impossible to do.

The cell phone is broken!

I know that some of you are probably thinking – if they only used [insert favorite development model here] then it would be bug-free! My question is then – when was the last time you saw a 100% error free software that did something meaningful? Even my Ubuntu server needs updates every now and again – As soon as you offer advanced human interaction, it becomes real hard to avoid bugs.

This is why we need real good integrators, that educate the customer to pick tried and tested combinations of hardware, software and settings that we know to work well. But sometimes we encounter a combination of settings that just doesn’t work, and then we look like idiots.

I was told it could haul 2000 pounds!

This is completely different from a camera with a defective soldering or routing on the motherboard. It is completely different from a damaged CMOS sensor. When these things are discovered, the manufacturer should send you a replacement of the same device, and go whip the people assembling the damn thing. If you get too many of these duds and you don’t handle them well, the market will/should soon put them out of business. When you encounter a software bug, someone has to change the code, recompile and send you a new binary file.

This is the difference – the resolution to a bug is a different product, while a resolution to a QA problem is the same product (only this time, assembled correctly).

Alternatively, clients can and will routinely withhold payment if the system (in its totality) is not working well. If an integrator decides to make some obscure combination of hardware and software to make the sale without ever doing even basic testing before deployment, then theres a chance that things are not going to work. A great integrator will know what works and where to tread with care. Gunslingers will make the unreasonable expectation that every single combination must work, and demand compensation if it doesn’t – regardless of how exotic the permutation is.

You may also encounter contracts where the integrator is penalized if the system is not delivered to spec and on time, but that requires that you actually spec everything out in advance, and in 99.99% of those cases the manufacturer will be able to deliver (as you are able to test beforehand) and if they don’t they will find some detail in the contract that bails them out.


Author: prescienta

Prescientas ruler

2 thoughts on “A Bug vs QA”

  1. I am not against drawing distinctions but It was off topic and created this weird side discussion. If someone want to start a dedicated discussion, it’s worth talking about (e.g., this post).

  2. Indeed it was OT, and I had no problem with the discussion being stopped right then and there. I just did not feel that it warranted a whole new thread within the IPVM group, and I suspect the discussion could get rather heated and unproductive.

    I use this blog to process my own thoughts and frustrations, and in this case I felt that Vincents point was being dismissed as a “useless semantic discussion”, someone even said that they suspected he work for Arecont, and someone else that they would never buy a product from the company that he then DID work for… that is a little over the top I think, and I needed to vent a little.

    In writing you always come across a little more arrogant than you really mean to be. When people have an issue (and are frustrated) and you communicate via emails or via forums it can quickly escalate into “oh, this guy is a total a-hole”, when all he is trying to do, is argue his side of the issue.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s