Let’s be clear; UI issues are real issues, and should be treated as such, but they are different in the sense that a poor UI can be usable, given enough practice and training, but technical issues (such as crashes) will render a product useless.
For the NVR writer the test is pretty simple. The NVR must acquire video and audio when instructed to do so, and it should be able to deliver the acquired data to the end user whenever they feel compelled to gaze at it. Sure, there are plenty more things the NVR need to do reliably, but the crux is that the testing is pretty static and deterministic.
Enter the grey area of analytics – it is much more difficult to test an analytics solution. Sure, we could set up a very wide range of reference tests that might be used to measure the quality of the algorithms (assuming the thing does not crash), but these tests might be “gamed” by the vendor, or then number of required tests would make the testing prohibitively expensive. I am sure there would be a market for an analytics test generator, simulating fog, rain, reflections, shadows, vibration and so on ad infinitum at almost no cost, but that’s beside the point.
My concern is that sometimes the actual, real performance is masked by poor UI; the operator blames the poor performance not on the algorithms, but instead themselves for not configuring, calibrating and nursing the system enough. When the vendors techies come on site they are always able to tweak the system to capture the events – “see, you just have to mask out this area, set up this rule, disable this other rule” and so on. The problem is that 2 days later, when it’s raining AND there is fog, the analytics come up short again. You tweak the rules, and now you’ve broken the setup for reflections and shadows and so it goes on and on.
This dark merry-go-round of tweaking is sometimes masked by the poor (crap?) UI. So this brings us to my argument: If the UI had been stellar, logical, easy to use, intuitive and all that, then the operator would quickly realize that the system is flawed and should be tossed. But if the UI is complex, weird, counterintuitive it takes longer for the end user to realize that something is fundamentally wrong, and subsequently they might stay in the business for longer.
Sure, at times things are just inherently complex (the complexities of a program can never be removed by replacing text-entry with drag’n drop), NAT setups and port forwarding is always a little difficult to handle in an elegant way and so on, and naturally analytics do require a fair bit of massaging before they work entirely as intended (if you have realistic expectations!! – which reminds me of a book I read recently on the financial meltdown; if people demand snakeoil, snakeoil will be provided).