Over on IPVM’s LinkedIn group there’s a discussion about compensation if a client encounters defective software or hardware. Vincent Tong brought up a point which was readily dismissed as a meaningless, semantic discussion – he pointed out that you need to discriminate between QA problems and bugs.
Here’s my defense of Vincent.
The term “bug” comes from the days when little critters – real bugs – would short electrical connections and cause machines to stop working. It’s a funny mental image, and so when poor soldering or routing on boards caused devices to malfunction, it was also called “bugs”. Naturally, programmers also wanted to blame someone else, so we decided to call mistakes “bugs” too.
Now, plenty of companies have experienced faltering, physical circuits. This week I “re-flowed” an old PS3. What happens is that the soldering between the chips and the motherboards come loose, to re-flow you melt the solder-balls ever so slightly to make them re-connect, the re-apply thermal grease and re-assemble. The issue is not entirely uncommon, but the question is – is it a bug?
To the end user, the semantic discussion is useless – the damn thing won’t work, but to manufacturers the distinction is extremely important. If you want to have a symbiotic relationship with your manufacturer, you should know, and appreciate, the difference too.
When people ask me if our software will work under XYZ conditions, I have to make an educated guess. First of all, the conditions are never described fully – did you specify the firmware of the camera, the brand and firmware of the switches, the cable lengths, the OS version, the language settings on the OS, the EFI version, the network adapter and so on. You don’t. And nor do we test every single permutation of hardware and software out there – it would simply be impossible to do.
I know that some of you are probably thinking – if they only used [insert favorite development model here] then it would be bug-free! My question is then – when was the last time you saw a 100% error free software that did something meaningful? Even my Ubuntu server needs updates every now and again – As soon as you offer advanced human interaction, it becomes real hard to avoid bugs.
This is why we need real good integrators, that educate the customer to pick tried and tested combinations of hardware, software and settings that we know to work well. But sometimes we encounter a combination of settings that just doesn’t work, and then we look like idiots.
This is completely different from a camera with a defective soldering or routing on the motherboard. It is completely different from a damaged CMOS sensor. When these things are discovered, the manufacturer should send you a replacement of the same device, and go whip the people assembling the damn thing. If you get too many of these duds and you don’t handle them well, the market will/should soon put them out of business. When you encounter a software bug, someone has to change the code, recompile and send you a new binary file.
This is the difference – the resolution to a bug is a different product, while a resolution to a QA problem is the same product (only this time, assembled correctly).
Alternatively, clients can and will routinely withhold payment if the system (in its totality) is not working well. If an integrator decides to make some obscure combination of hardware and software to make the sale without ever doing even basic testing before deployment, then theres a chance that things are not going to work. A great integrator will know what works and where to tread with care. Gunslingers will make the unreasonable expectation that every single combination must work, and demand compensation if it doesn’t – regardless of how exotic the permutation is.
You may also encounter contracts where the integrator is penalized if the system is not delivered to spec and on time, but that requires that you actually spec everything out in advance, and in 99.99% of those cases the manufacturer will be able to deliver (as you are able to test beforehand) and if they don’t they will find some detail in the contract that bails them out.