Safe, Easy, Advanced

You can only pick 2 though.

Admitting mistakes is hard; it’s so hard that people will pay good money just to be told that they are not to blame for the mistake. That someone else is responsible for their stupidity. And sometimes they’re right, sometimes not.

Anton Yelchin was only 27 when he died, he left his car in neutral on an incline. The car started rolling and killed him. Since it would be unbearable to accept that Anton simply made a mistake, lawsuits were filed.

Another suitor claimed that the lever was too complex for people to operate, therefore the manufacturer is liable for the damage that occurs when people don’t operate them correctly. The car had rolled over her foot, and while there were no broken bones, she was now experienced “escalating pains”, and demanded reparations. One argument was that the car did not have the same feature as a more expensive BMW.

Tragically, every year more than 30 kids are forgotten in cars and die. When I bring this up with people, everyone says “it won’t ever happen to us”, and so there’s zero motivation to spend extra on such a precaution. The manufacturers know this, and since there’s also liability risk, they are not offering it. So, every year, kids bake to death in cars. It’s a gruesome fate for the kids, but the parents will never recover either.

Is it wrong to “blame the victim”?

I think the word “blame” has too many negative connotations associated to be useful in this context. Did the person’s action/inaction cause the outcome? If the answer is a resounding yes, then sure…  we can say that we “blame the victim”.

It’s obviously a gray area. If a car manufacturer decides that P should mean neutral and N should mean park, and writes about this in their manual and tells the customers as they sign the contract, then I wouldn’t blame an operator for making the mistake. The question is – would a person of “normal intelligence” be more than likely to make the same mistake?

In our industry, not only are we moving the yard-post of what “normal intelligence” means. Some of the most hysterical actors are using the bad practices of the layman and arguing that the equipment, therefore, can’t be used by professionals.

It feels like it’s entirely reasonable to argue no-one should drive 18-wheelers because random people bought modified trucks at suspect prices in a grey market and then went ahead and caused problems for a lot of people.

As professionals, we’re expected to have “higher intelligence” when it comes to handling the equipment. You can’t call yourself “professional” if you have to rely on some hack and his gang to educate you online or through their “university”. And you sure as hell can’t dismiss the usability of a device based on what random amateurs do with it.

So what gives? You have a bunch of people who act like amateurs but feel like “professionals” because they are paying good money for this industry’s equivalent of 4chan and got paid to install a few units here and there.

It seems to me that the hysterical chicken-littles of this industry are conflating their own audiences with what actual professionals are experiencing. E.g. if someone suggests using a non-standard port to “protect their installation”, then you know that the guy is not professional (doesn’t mean he’s not paid, just means he’s not competent).

And that’s at the core of this debacle: people that are incompetent, feel entitled to be called professionals, and when they make mistakes that pros would never make, it’s the fault of the equipment and it’s not suitable for professionals either.

So, as I’ve stated numerous times, I have a Hikvision and an Axis camera sitting here on my desk. Both have default admin passwords, yet I have not been the victim of any hack – ever. The Hikvision camera has decent optics (for a surveillance camera) and provides an acceptable image at a much lower cost than the “more secure” option.

And I’ll agree that getting video from that camera to my phone, sans VPN is not “easy” for the layman. But it doesn’t have to be. It just has to be easy for the thousands of competent integrators know what to do, and more importantly, what not to do.

That said; the PoC of the HikVision authentication bypass string should cause heads to roll at Hikvisions (and Dahuas) R&D department. Either there’s no code-review (bad) or there was, and they ignored it (even worse). There’s just no excuse for that kind of crap to be present in the code. Certainly not at this day and age.

 

Advertisements

Debtors Prison

There’s a wonderful term called “technical debt”. It’s what you accrue when you make dumb mistakes, and instead of correcting the mistake, and taking the hit up front, you take out a small loan, patch up the crap with spittle and cardboard, and ship the product.

kid_credit
Yay! Free money!!!

Outside R&D technical debt doesn’t seem to matter. It’s like taking your family to a restaurant and racking up more debt; the kids don’t care, to them, the little credit card is a magical piece of plastic, and the kids are wondering why you don’t use it more often. If they had the card, it would be new PlayStations and drones every day.

Technical debt is a product killer; as the competition heats up, the company wants to “rev the engine”, but all the hacks and quick fixes mean that as soon as you step on the gas, the damn thing falls apart. The gunk and duct tape that gave you a small lead out of the gate, but in the long run, the weight of all that debt will catch up. It’s like a car that does 0-60 in 3 seconds but then dies after 1 mile of racing. Sure it might enter the race again, limp along for a few rounds, then back to the garage, until it eventually gives up and drops out.

Duct Tape Car Fix - 03
Might get you home, but you won’t win the race with this fix

Why does this happen?

A company may masquerade as a software company and simply pile more and more resources into “just fix it” and “we need” tasks that ignore the real need to properly replace the intake pipe shown above. “If it works, why are you replacing it”, the suit will ask, “my customer needs a sunroof, and you’re wasting time on fixing something that already works!”.

So, it’s probably wise to look at the circumstances that caused the company to take on the debt in the first place. An actual software company might take technical debt very seriously, and very early on they will schedule time for 3 distinct tasks:

  1. Ongoing development of the existing product (warts and all),
  2. Continued re-architecting and refactoring of modules,
  3. Development of the next generation product/platform

Any given team (dependent on size, competency, motivation, and guidance) will be able to deliver some amount of work X. The company sells a solution that requires the work Y. Given that Y < X, the difference can be spent on #2 and #3. The bigger the difference, the better the quality of subsequent releases of the product. If the difference is small, then (absent team changes), the product will stagnate. If Y > X then the product will not fulfill the expectations of the customer. To bridge the gap until the team can deliver an X > Y, you might take on some “bridge debt”. But if the bridge debt is perpetual (Y always grows as fast or faster than X), then you’re in trouble. If Y > X for too long, then X might actually shrink as well, which is a really bad sign.

Proper software architecture is designed so that when more (competent) manpower is added, X grows. Poor architecture can lead to the opposite result. And naturally, incompetent maintenance of the architecture itself (an inevitable result of a quick-fix culture), will eventually lead to the problematic situation where adding people lead to lower throughput.

A different kind of “debt” is the inability to properly value the IP you’ve developed. The cost of development is very different from the value of the outcome. E.g. a company may spend thousands of hours developing a custom log handler, but the value of such a thing is probably very low. This is hard to accept for the people involved, and it often leads to friction when someone points out that the outcome of 1000 hours of work is actually worthless (or possibly even provides a net negative value for the product). A lot of (additional) time may be spent trying to persuade ourselves that we didn’t just flush 1000 hours down the drain, as we’re more inclined to believe a soothing lie than the painful truth.

Solutions?

A company that wants to solve the debt problem must first take a good look at its core values. Not the values it pretends to have, but the actual values; what makes management smile and how it handles the information given to them. Does management frown when a scalability issue is discovered, do they yell and slam doors, points out 20 times that “we will lose the customer if we don’t fix this now!”. The team lead hurries down the hallway, and the team pulls out cans of Pringles and the start ripping off pieces of tape.

The behavior might make the manager feel good. The chest-beating alpha-manager put those damn developers in their place, and got this shit done!. However, over the long run, it will lead to 3 things : 1) Developers will do a “quick fix”, because management wants this fixed quickly, rather than correctly, 2) Developers will stop providing “bad news”, and 3) developers that value correctness and quality will leave.

To the manager, the “quality developer” is not an asset at all. It’s just someone who wants to delay everything to fix an intake that is already working “perfectly”. So over time, the company will get more and more duct-tapers and hacks, and fewer craftsmen and artisans.

The only good thing about technical debt (for a coder) is that it belongs to the company, and not to the employees. Once they’re gone, they don’t have to worry about it anymore. Those that remain do, and they now have to work even harder to pay it back.

debt_mountain2

The Parts of an IP Camera

To understand where the IP camera market is headed, I think it’s important to understand how one of these things are put together.

Like most high tech devices, each product is really an amalgamation of parts from different manufacturers. In fact many products are the result of tight, but perhaps unappreciated, collaboration of several (sometimes competing) companies. I’d recommend listening to Freakonomics rundown of the “I, Pencil” essay (starts 7 minutes in).

So, an IP camera is not a pencil, but just like all pencil manufacturers don’t manufacture every single part of the pencil, but instead, they purchase the parts (graphite, brass, paint and so on) and every manufacturer puts the pencil together following roughly the same pattern.

And, so, when it comes to IP cameras, they too are composed of parts that are available to everyone who wants to start making cameras.

You’ll need a couple of things: A lens, a sensor, some circuitry and some code.

You’re not going to start making your own lenses or sensors, are you? Probably not, so you’ll get the lenses from a lens maker (and they may even outsource their manufacturing process even further), and the sensor from either Sony or Canon.

You’re not going to design your own CPU either (unless you’re Axis). Today, you’d be better off grabbing an ARM platform and use that to drive the sensor and interface. The other advantage is that ARM is well supported in the software world, so you’re already halfway there.

Now that you have the basics, you need to write some code to get it all working together. If you went the ARM route, it’s pretty simple to get a linux kernel running. Well.. “simple” is depends on your level of skill, but finding a few geeks who can do this shouldn’t take long. So you grab the Linux kernel, add Apache or perhaps GoAhead, you can add gStreamer too (do check the link, it is a great presentation by Axis) . The next thing you know, you have a jumble of cables and breadboards, burns on your fingers from the soldering iron, you haven’t seen your kids in 4 days and the smell is getting a little hard to stomach.

On top of that, you need to wrap this in an enclosure. There’s regulations to follow, tests that need to be carried out and so on. Then you have the nightmare of maintaining all those pieces of code, and trust me – if you wrote everything yourself, it would take even longer and be much harder to test and maintain.

What if there was a company, that could do all of the above? And just stick my name on the box? After all, my company would pick the same lens, the same sensor, the same board and the same software, so why not do it?

I have no intention of starting production of a Raspberry Pi Zero based IP camera, but I know that I can make one for ~$40 (and that’s buying all the parts retail). Not only will this thing work as an IP camera, it can work as a full fledged stand-alone VMS.

In other words, the question is: if some washed up coder in Copenhagen can build a fully functional “IP camera” for $40, I think you’re going to face a tough time if you’ve based your entire organization around selling your cheapest cameras for $250+ (they may be “even more good enough”, but who cares?).

Obviously, my camera is not going to be materially different from the other guy’s cameras. We’re all going to use the same bits and pieces, including software, even the damn protocols are going to be the same.

So, I think we’re going to see a race to the bottom in terms of prices. The cameras will look and perform almost identically across brands, use the same protocols, and be completely interchangeable, much to the chagrin of the incumbents, so the USP for the brands in this realm will have be something else.

VMS Software, perhaps…

 

 

 

 

 

 

P2P

As with IP cameras, one of the IoT challenges is how to get your controlling device (typically a phone) to talk to the IoT device in a way that does not require opening up inbound ports on your firewall.

All communication is peer to peer, so the term, when used in the context of IoT devices, is perhaps a little misleading, after all, an exposed camera sending a video stream to a phone somewhere is also “peer to peer”. Instead, P2P might be translated to “send data from A to B, even if both A and B are behind firewalls, using a middleman C” (what the hell is up with all the A, B, C these days).

On a technical level, the P2P cameras use something called UDP hole punching, which sounds a bit onymous, but there’s really nothing sneaky about it. What happens is that A connects to C, so that C now knows the external IP address of A. Likewise, B also connects to C, and now C knows the external IP address of both A and B.

This middleman, now passes the IP address of A to B, and B to A. Next step is for A to fire a volley of UDP packets towards B, while B does the same towards A.

The firewall on A’s side sees a bunch of packets travel to B’s address, and when B’s packets arrive, the firewall thinks that the UDP packets are replies to the packets that were sent from A and let’s them through.

You could accomplish the same thing by having A go to “whatsmyip.com” and email it to B, B would then do the same. Then run scripts that send UDP packets over the network, but a STUN server automates this process.

But who controls this “middle man”? Ideally, you’d be in charge of it; you’d be able to specify your own STUN-type server in the camera interface, so that you have full control of all links in the chain. In time, perhaps the camera vendors will release a protocol description and open source modules so that you can host your own middle-man.

The problem might be that you bought a nice cheap camera in the grey market. The camera is intended for the Chinese market, but comes with a “modded” firmware that enables English menus and so on. This is obviously risky. Updating a modded firmware may be impossible and brick the camera, and the manufacturer may be less inclined to support devices that have been modded. You get what you pay for, so to speak (and this blog is free!)

The modder is selling the cameras in the western markets, but the STUN server is still pointing to a server in China. This makes sense if you are a Chinese user, but it may seem very strange that your camera “calls home” to a server in China. A non-modded camera might do the same, simply because running a STUN service is cheaper, and allows the government to eavesdrop on the traffic. If you are Chinese (I am not), you could argue that you don’t trust Amazon, Microsoft or Google because they might work with the NSA. Therefore, using your own server would be preferred.

Apart from the STUN functionality, the camera may follow direction that are sent from B to C to A. This puts a lot of responsibility in the hands of the guys maintaining this server. If it is breached, a lot of cameras will then be vulnerable.

Depending on the end user, P2P may not be appropriate at all. To some users, the cost of a breach is small, compared to the hassle of installing a fully secure system it might be worth it.

While yours truly has abandoned all attempts to appear professional over the years, the truth is that most big installations have their shit together. Unfortunately the volume of DIYers and amateurish installers who don’t really know what they are doing is much bigger (in terms of headcount, not commercial volume), and if there’s one thing we all want to do, it’s to blame someone else.

Caveat Emptor.

this-is-fine.0

 

(Dis)honesty

It’s very hard to differentiate between stupidity and intentional dishonesty. Yesterday, I made a mistake of posting an inaccurate portrayal of statements made by another blogger. I removed it from my blog almost immediately. But subscribers to my blog had received the post, and one reader quickly accused me of “defamation”, and told me that I was “on notice”.

Mistakes are made, I attempt to correct them as quickly and as best I can, but sometimes people just crave confrontation.

I’d like to take the opportunity to talk about true dishonesty. I’m a big fan of Dan Ariely, who studies behavioural economics. When Salty Features wanted to crowdfund a movie about dishonesty, centered around Dan’s work, I did not hesitate to sign up, and I can only recommend that you go watch (Dis)honesty (it’s on Netflix in Denmark and most other places I would imagine).

Dan and his team tries to establish a baseline for how dishonest people are, in general. The experiment is quite simple. They hand out math tasks. The more tasks are solved, them higher the reward (a few dollars). But, instead of handing the papers over for verification, they are to send them straight to a paper shredder. Then proceed to the controller’s desk, and simply state how many tasks were solved. No chance of verification.

The rigid financial theory says that there is 0% chance of being caught, so they might as well say that they solved all the tasks, and reap all the reward.

But that’s not what happens. A lot of people cheat a little bit, some not at all, and some go all in (in accordance with the rigid theory). The completely honest and completely dishonest people are outliers.

How did they know that the students cheated? The shredder was rigged. Some people might have suspected this, but even if they did, there would be no repercussions to go all in on the cheating, so some people are just naturally honest, while some use every opportunity to the fullest.

The experiment was then changed, so that instead of direct payment, the students would receive tokens, that would then be exchanged to real cash. Most people would probably assume that this made no difference at all, but as it turns out, this added abstraction of cheating = cash, increased the cheating substantially. The end result was the same, but one step was interjected that really should have mattered, but it did.

So, what if we look at the business world? Dan argues that the abstraction of money in high finance might be a contributing factor in what he describes as cheating, but it’s not just banks. As Dan’s research shows, a little bit of cheating is common – natural – even.

We can also illustrate the principle in another way:

Here’s an example of something that everyone will agree is problematic:

Company A pays B (who claims “independence”) to write about competitors of Company A.

Anyone in their right mind would say that B is hardly independent, and who would ever trust anything B has to say regarding companies in A’s domain? I know that a few would. There’s always a few. But most wouldn’t.

How about this, then:

Company A pays Company C to sign up to a service from Company D that is owned by B.

The end result is the same, but we’ve inserted a few layers of abstraction between A and B. And as Dan’s research show, there’s always a few that exploit this as far as it can go.

I highly recommend Dan’s books. And may I suggest you visit his sites too.

 

NSFW: Let’s talk about 2009

This motion picture article is protected under the copyright laws of the United States and other countries throughout the world. Country of first publication: United States of America. Any unauthorized exhibition, distribution, or copying of this film article or any part thereof (including soundtrack?) may result in civil liability and criminal prosecution. The story, all names, characters, and incidents portrayed in this production are fictitious. No identification with actual persons (living or deceased), places, buildings, and products is intended or should be inferred.

2009, to some people, seem like long ago. I remember it vividly. It was the nadir of the financial crisis (the great recession) and the central banks were in panic mode. Their attempts to calm the waters with lies, omissions and cover-ups had failed. Jim Cramer had had a meltdown on live TV and Jon Stewart later took him through hell, in what was a rare glimpse of honest journalism on mainstream TV. Incidentally, the show was taped down the street from where I used to live.

Today the market are at an all time high, and to most of the bankers, the year 2009 probably seems like a distant, vague memory. Not something to consider anymore. We’ve moved on etc. Other people have other reasons to distance themselves from that anno horribilis.

I don’t remember the exact date, but I was a little hung over and I had stumbled into some sort of marketing-integration-pep talk-show. At first I thought I was at a church, as I saw a man with a squeaky voice perform a very strange sermon. As the fog of yesterdays gin/tonics lifted, and I could see more clearly, I discovered that the man on stage was just getting started. What followed next, made me question my sanity.

As the man rambled on about how “we can squeeze out cost and squeeze out Verint”, he seemed to drive himself into a state of trance; as the man heard his own voice, it triggered a sort of feedback loop, that in turn caused the mouth to make even more outrageous statements. Tourrettes causes involuntary expressions of vulgarity and noises, but this was not Tourrettes. The steady flow of depravity and vulgarities was entirely voluntary and the man seemed anxious to drive himself into ever higher states of madness.

In my recollection (admittedly rather faint due to over-consumption of stimulants), I stumbled back to my hotel room, Thinking that I had just witnessed a male version of “heavy splash” in Japanese. I showered for 45 minutes. I submerged myself in scalding hot water, but I still felt dirty. I assumed that the man had received the king’s ransom in order to put on that kind of show.

While the memories from that fateful day had started to fade, they were always present in the back of my mind. I had to remind myself that we had just gone through a financial crisis that had driven people to desperation. Many years of over consumption, and debt fueled spending had come back to roost. As the tide went out, some people, it turned out, were naked.

When I recently sat outside my local watering hole, sipping a cup of detoxicating green tea, and nibbling on some gluten-free, fully organic biscuits from a farm just a few miles from Copenhagen, I ran into the same man.

It was a shocking sight. The man had gathered a following that were carrying him around town. He was wearing thick makeup, heavy rouge and pink lipstick. His disciples placed a wooden crate on the ground, and placed the man atop. He locked eyes with me, and there was a long awkward pause.

He prodded one of his lackeys with a stick, and whispered something inaudible in his ears. The lackey then cleared his throat, and proceeded to address me.

“We are not happy with how you portray things” he said. He then pulled out a scroll, and handed it to me. It was hardly legible, it had either been written by a 5 year old child, or a very old man. It stated that they were going to tell their herd about me and my evil deeds.

I was confused. In front of me stood, what would appear to be an adult male, wearing clown makeup, surrounded by a group of escapees from an asylum, announcing that he would “tell on me”.

This was the same man I had seen perform unspeakable acts of depravity just a few years ago.

I asked if his flock knew about his past, thinking that the desire bow down to their high priest would be somewhat diminished. But I also suspected that most people in the flock had never heard of the internet or google, so I genuinely wasn’t sure.

To my surprise, the clown decided to speak. He cleared his throat. His eyes rolled to the skies, and when I once again heard that squeaky voice again it all came back to me. It’s still surprising so vividly you remember things, under the right circumstances. Like hearing a long forgotten pop-song from the 80’s and remembering every detail from that summer in Spain.

But this was no summer in Spain, and the man announced that he was fed up with me bringing up his performance many years ago. He straightened and lifted his arms, palms facing the sky and proclaimed “I received no payment, I did it all for free”. There was a pause. Everyone waiting, breath baited, and then, with a deep voice (well, not that deep) he ceremonially announced: “I am clean”.

As he said those words, my biscuit fell to the ground and broke into 1 large and 4 smaller pieces and some crumbs. My jaw dropped. He had done it all for free!?! It was hard to fathom. My mind started racing. This was unexpected. Why on earth would you do what I had witnessed a few years ago, for free?!!! This was not a man who did anything for free, unless there was something to be gained later.

I later realized that the guy published some sort of periodical that people had to pay to read. It mostly contained self praise, and descriptions of what happens when completely inept people attempt to use high tech equipment. I suppose it could be thought of as a mildly entertaining break from the daily humdrum at the office and you can always call it “working”, because it is kinda, sorta, related to what you do.

I suspect that direct payment, would probably be considered prostitution and therefore illegal. Instead, as payment for his performance on that fateful day, the host of the show would instead purchase a lot of “licenses” to read the “news”.

And that, ladies and gentlemen, how you stay “clean”.

 

I Am Myself

Well, well, well…

This weekend I posted a piece on IPVMs crusade against Hikvision which seemed to suggest a lack of technical comprehension and perhaps – general assholery.

1 minute after posting, I receive a visit from Ghana. I have also had visitors from Mali and other nations in Africa that seemingly have a keen interest in what I have to say. Another option is that someone thinks they need to use Tor (or some other anonymizing browser) to read my blog.

This morning, I woke up to an email, asking me to ensure that the folks from some obscure blog understands that this blog is in no way, shape or form affiliated with OnSSI. A strange coincidence that writing about a sensationalist blog fucking things up, triggers a request for clarification about the independence of this one.

So let me make that absolutely clear, so that even sensationalist bloggers running fake universities, and his “associates” can understand it.

This blog, has nothing to do with OnSSI.

While I have written specifically about the mobile app OnSSI released a while ago, other people in the software development industry (not IP video), have the exact same experience. Next generation apps face an uphill battle as loyal users of the old app discover that things may have changed, and they are much more likely to post a very negative “review”, than people who will eventually benefit from the improvements. Since posts that are anchored in real experiences are dangerous to my livelihood (the blogger is using them as a vector to try and shut me down), I will remove that type of content from the blog (but I am confident that the Ghanaian visitor made a copy before reaching out to protect the innocent, so just ask him for a copy).

So, just to be clear, what you read in these posts, is the opinions and thoughts of the person Morten Tor Nielsen. I submit ideas and thoughts that are founded in a general understanding of the world as I see it.

I suppose that if you are consumed with deranged ideas about infiltration of corner shops and jiffy-lubes by the Chinese government, and your every living hour is spent on thinking about how to attract more subscribers to your rumour-mill, then this might be hard to fathom, but I work on a wide range of things (including overhauling my old Suzuki Bandit 600), and so among the exposure of incompetent asshats (from my humble computer here in Copenhagen), jot about a lot of things.

If you follow my blog, you’ll know that I have been working on micro-PC‘s, I have set up Axis cameras to provide health state information, I have done a lot of GPU work (yes, for OnSSI) and many other things. I have called out BS here, and here, and here  and many other places. I have mused over how companies can improve and what danger signs to look for. I have critiqued buzz-word-driven development (as a response to VR goggles being passed out at a convention). The list goes on…

You have to be senile, demented or sociopathic to think that this blog would somehow reflect the “thoughts” of a company. So if you suffer from any of the 3, and that’s the reason you contact OnSSI rather than writing a comment refuting claims, then you’re excused.

If not, you’re just a sad, over-extended sphincter.

But I think you (and everyone else) know that already.