Category Archives: idiocy

Hikvision Feeds a Troll

It’s possible to turn someone towards the light, and eventually lead them to salvation.

A prominent member of the Westboro Baptist Church, Megan Phelps-Roper made a TED speech about it. What saved Megan was not someone yelling in her face. She was conditioned to expect exactly that from the misguided heathens of the world. Instead, someone approached her with curiosity, warmth and civility and lead her out of the congregations grasp. The “enemy” is rarely a mindless drone out to do evil. Although, our leaders would prefer we see things that way.

Our industry has a variant of the WBC, and Hikvision has chosen a different approach to liberate the members of the sect.

a site that has always trafficked in nefarious insults and innuendo. Hiding behind a keyboard, the tabloid’s staff takes unfounded potshots at our entire industry, bullying one company at a time.

and

Instead, he chooses to distract manufacturers with his pursuit of financial gain and efforts to fulfill his delusions of grandeur.

The problem with this sort of message is that the hardcore members are expecting exactly this sort of rhetoric, thereby further entrenching them in their beliefs. Ultimately the blogger will surely capitalize of the increased attention being paid. I thought it was common knowledge that trolls have an insatiable appetite for the kind of copy Hikvision just released.

jh

More please!

Members of the sect can attend a “university” (not at all like this one) and even make the “dean’s list“. This is impressive stuff, and these people are not going to be swayed by a manufacturer having a breakdown in their public relations department. Furthermore, I suspect Hikvision has several active subscriptions, thereby directly funding the site.

I think Hikvision is correct in calling it cyber-bullying. It has all the traits of schoolyard terrorism; the ring-leader points out an arbitrary enemy, then manipulates the enemy to react. Steps in to protect the flock from the aggressor. If it gets too hot, the ring-leader can count on his 3 or 4 lackeys to do the dirty work.

In this case, the sin of the “enemy” is that the company is partially owned by the Chinese government. Therefore, every vulnerability found in a Hikvision camera is proof positive that the Chinese government is spying on us. I don’t buy that. Governments don’t have to own a company to assert influence over it.

You might remember Stuxnet a vulnerability in SCADA equipment was exploitable by governments and for-lulz hackers alike. Vulnerabilities will continue to exists as long a fallible humans write the code. As long as fallible humans install and (fail to) maintain the equipment we will continue to see flaws and problems. Unfortunately, a lot of companies have deployed small time-bombs with terrible security in place, not just Hikvision.

When I was in the army, we had padlocks on our lockers. On the first day, we were instructed to get a hair-cut “to not look like faggots” (I kid you not, that’s what he said), and then to make sure our lockers were safely locked. The reasoning (for locking up) is that you can’t really trust anyone, and giving the bad apples the opportunity to steal was almost as bad as the guy stealing. At a company I worked for a long time ago (starts with an M), someone shat on the toilet seat in the offices restroom. Someone we had lunch with, talked about code, movies, politics and music with, went to the bathroom, and shat on the seat, leaving it there for some poor soul to find.

Same thing goes for your IP camera. Sticking that thing on the internet, REGARDLESS of manufacturer ownership is like leaving your locker unlocked. You are tempting the swines of the world to mess around, and when they do, we all lose.

madness

Tagged , ,

Listening to Customers

In 2011, BlackBerry peaked with a little more than 50 million devices sold. The trajectory had an impressive ~50% CAGR from 2007 where the sales were around 10 million devices. I am sure the board and chiefs were pleased and expected this trend to continue. One might expect that ~250 million devices were to be sold in 2016 if the CAGR could be sustained. Even linear growth would be fairly impressive.

Today, in 2017, BlackBerry commands a somewhat unimpressive 0.0% of the smartphone market.

There was also Nokia. The Finnish toilet-paper manufacturer pretty much shared the market with Ericsson in Scandinavia and was incredibly popular in many other regions. If I recall correctly, they sold more devices than any other manufacturer in the world. But they were the McDonalds of mobile phones: Cheap and simple (nothing wrong with that per se). They did have some premium phones, but perhaps they were just too expensive, too clumsy or maybe too nerdy?

ngage

Talking on a Nokia N-Gage phone

Nokia cleverly tricked Microsoft into buying their phone business, and soon after the Microsoft gave up on that too (having been a contender in the early years with Windows CE/Mobile).

I am confident that BlackBerry was “listening to their customers”. But perhaps they didn’t listen to the market. Every single customer at BlackBerry would state that they preferred the physical keyboard and the naive UI that BlackBerry offered. So why do things differently? Listen to your customers!

If BlackBerry was a consulting agency, then sure, do whatever the customer asks you to. If you’re selling hot-dogs, and the customer asks for more sauerkraut, then add more sauerkraut, even if it seems revolting to you. But BlackBerry is not selling hotdogs or tailoring each device to each customer. They are making a commodity that goes in a box and is pulled off a shelf by someone in a nice shirt.

As the marginally attached customers are exposed to better choices (for them), they will opt for those, and in time, as the user base dwindles, you’re left with “fans”. Fans love the way you do things, but unless your fan base is growing, you’re faced with the very challenging task of adding things your fans may not like. Employees that may be prostrate bowed but not believing, will leave and eventually you’ll have a group of flat-earth preachers evangelizing to their dwindling flock.

It might work as a small, cooky company that makes an outsider device, but it sure cannot sustain the amount of junk that you tag on over the years. Eventually that junk will drag the company under.

Or, perhaps BlackBerry was a popular hotdog stand, in a town where people just lost the appetite for hotdogs and had a craving for juicy burgers and pizza (or strange hotdogs)

Tagged , ,

Magical “GPU” Based Video Decoder

I was recently alerted to an article that described a magical video decoding engine. The site has a history of making odd conclusions based on their observations, so naturally, I was a bit skeptical about the claims that were relayed to me by a colleague. Basically, the CPU load dropped dramatically, and the GPU load stayed the same. This sounded almost too good to be true, so I did some casual tests here (again).

EouEzI5bBR8uk.gif

Test setup

I am not thrilled about downloading a 2 GB installer that messes up my PC when I uninstall it, and running things in a VM would not be an honest test. Nor am I about to buy a new Intel PC to test this out (my next PC will be a Ryzen based system), so all tests are done with readily available tools: FFMpeg and GPU-Z. I believe that Intel wrote the QSV version of the h264 decoder, so I guess it’s as good as it gets.

Tests were done on an old 3770K, 32 GB RAM, Windows 7 with a GeForce 670 dedicated GPU. The 3770K comes with the Intel HD Graphics 4000 integrated graphics solution that supports Quick Sync.

Terminology

In the nerd-world, a GPU usually means a discrete GPU; a NVidia GeForce or AMD Radeon dedicated graphics card. Using the term “GPU support” is too vague, because different vendors have different support for different things. E.g. NVidia has CUDA and their NVEC codecs, and some things can be done with pixel shaders that work on all GPUs. (our decoding pipeline uses this approach and works on integrated as well as discrete GPU, so that’s why I use the term GPU accelerated decoding without embarrassment).

However, when you rely on (or are testing) something very specific, like Intel Quick Sync, then that’s the term you should use. If you say GPU support then the reader might be lead to believe that a faster NVidia card will get a performance boost (since the NVidia card is much, much faster than the integrated GPU that hosts Quick Sync). This would not be the case. A newer generation of Intel CPU would offer better performance, and it would not work at all on AMD chips with a dedicated GPU (or AMD’s APU solution). Same if you use CUDA in OpenCV, then say “CUDA support” to avoid confusion.

Results

Usually, when I benchmark stuff, I run the item under test at full capacity. E.g. if I want to test, say the CPU based H264 decoder in FFMpeg against the Intel Quick Sync based decoder, I will ask the system to decode the exact same clip as fast as possible.

So, let’s decode a 720p clip using the CPU only, and see what we get.

CPU

The clip only takes a few seconds to decode, but if you look at the task manager, you can see that the CPU went to 100%. That means that we are pushing the 3770K to it’s capacity.

CPU_FPS

Now, let’s test Quick Sync

QSV

Not as fast as the CPU only, but we could run CPU decoding at the same time, and in aggregate get more…. but we got ~580 fps

QSV_FPS

So we are getting ~200 fps less than the CPU-only method. Fortunately, the CPU is not being taxed to 100% anymore. We’re only at 10% CPU use when the QSV decoder is doing its thing:

CPU_QSV

Magic!!!

But surprisingly, neither is the GPU. In fact, the GPU load is at 0%

GPU_QSV

However, if you look at the GPU Power, you can see that there is an increased power-draw on the GPU at a few places (it’s drawing 2.6W at those spikes). Those are the places where the test is being run. You can also see that the GPU clock increases to meet the demand for processing power.

If there is no load on the GPU, why does it “only” deliver ~600 fps? Why is the load not at 100%? I think the reason is that the GPU load in GPU-Z does not show the stress on the dedicated Quick Sync circuitry that is running at full capacity. I can make the GPU graph increase, by moving a window onto the screen that is driven by the Intel HD Graphics 4000 “GPU”, so the GPU-Z tool is working as intended.

I should say that I was able to increase performance by running 2 concurrent decoding sessions, getting to ~800 fps, but from then on, more sessions just lowers the frame rate, and eventually, the CPU is saturated as well.

Grief

To enable Quick Sync on my workstation which has a dedicated NVidia GeForce 670 card on Windows 7, I have to enable a “virtual” screen and allow windows to extend the display to this screen (that I can’t see because I only have one 4K monitor). I also had to enable it in the BIOS, so it was not exactly plug and play.

Conclusion

I stand by my persuasion: yes, add GPU decoding to the mix, but the user should rely on edge-based detection combined with dedicated sensors (any integrator worth their salt will be able to install a PIR detector and hook it up in just a few minutes). This allows you to run your VMS on extremely low-end hardware and the scalability is much better than moving a bottleneck to a place where it’s harder to see.

Tagged , ,

Marketing Technology

I recently saw a fun post on LinkedIn. Milestone Systems was bragging about how they have added GPU acceleration to their VMS, but the accompanying picture was from a different VMS vendor. My curiosity had the better of me, and I decided to look for the original press release. The image was gone, but the text is bad enough.

Let’s examine :

Pioneering Hardware Acceleration
In the latest XProtect releases, Milestone has harvested the advantages of the close relationships with Intel and Microsoft by implementing hardware acceleration. The processor-intensive task of decoding (rendering) video is offloaded to the dedicated graphics system (GPU) inside the processer [sic], leaving the main processor free to take on other tasks. The GPU is optimized to handle computer graphics and video, meaning these tasks will be greatly accelerated. Using the technology in servers can save even more expensive computer muscle.

“Pioneering” means that you do something before other people. OnSSI did GPU acceleration in version 1.0 of Ocularis, which is now 8 or 9 years old. Even the very old NetSwitcher app used DirectX for fast YUV conversion. Avigilon has been doing GPU acceleration for a while too, and I suspect others have as well. The only “pioneering” here is how far you can stretch the bullshit.

Furthermore, Milestone apparently needs a “close relationship” with Microsoft and Intel to use standard and publicly available quick sync tech. They could also have used FFMpeg.

We have experimented with CUDA on a high end nVidia card years ago, but came to the conclusion that the scalability was problematic, and while the CPU would show 5%, the GPU was being saturated causing stuttering video when we pushed for a lot of frames.

Using Quick sync is the right thing to do, but trying to pass it off as “pioneering” and suggesting that you have some special access to Microsoft and Intel to do trivial things is taking marketing too far.

The takeaway is that I need to remind myself to make my first gen client slow as hell, so that I can claim 100% performance improvement in v2.0.

keep-calm-and-ignore-bullshit-7-257x300

Tagged , ,

VR and Surveillance

Nauseating and sweaty I remove my VR goggles. I feel sick, and I need to lie down. Resident Evil 7 works great in VR because things can sneak up on you from behind. You have to actually turn your head to see what was making that noise behind you.

On a monitor I can do a full panoramic dewarp from several cameras at once, and the only nausea I experience is from eating too many donuts too fast. There’s no “behind” and I have a superhuman ability to see in every direction, from several locations, at once. A friend of mine who played computer games competitively (before it was a thing), used the maximum fov available to give him an advantage to mere humans.

panorama

One feature that might be of some use is the virtual video wall. It’s very reminiscent of the virtual desktop apps that are already available.

And I am not even sure about the gaming aspect of VR. In the gaming world, people are already wondering if VR is dead or dying. Steam stats seem to suggest that it is the case, and when I went to try the Vive in the local electronics supermarket, the booth was deserted and surrounded by boxes and gaming chairs. Apparently you could book a trial run, but the website to do so was slow, convoluted and filled with ads.

Time will tell if this takes off. I am not buying into it yet.

 

Armageddon

Oh, video surveillance industry, I have failed ye. And I apologize. I did my best.

The false prophet is constantly preaching to his obedient and subservient flock. Tail wagging, eyes wide open, listening to the dog-whistle playing tunes of fear, uncertainty, and doubt.

All we can do is sit back and watch as the industry gets destroyed by consuming the vile soup consisting of equal parts arrogance and ignorance, served up by his highness.

I shall never forget the time, about 13 years ago, when a store manager asked why the hell it had to be so advanced. He fondly remembered his VCR that had a nice red button and it just worked. Plug in the camera, and you had video. It was that simple.

Pretty much anyone could install these systems. Video quality was shit and tapes wore out, but it was simple and most people could operate it. Once we moved to IP we fucked it all up. It became a nightmare to install and operate, and you had to have a degree in engineering to make sense of any of it.

In this complex world, some people are now shitting their pants over the ownership of a technology company by a government entity. Perhaps I am wrong. Maybe the encopresis is not related to the new gospel, but is a more permanent state of affairs, who knows? But I am starting to notice the smell.

We’re past reasoning here. We’re past the point where the accuser delivers the proof, instead, the accused now has to prove his innocence. Apparently, The Court of Oyer and Terminer has been established, and our present day version of  Thomas Newton presenting his evidence for all to see – “The coat is cut or torn in two ways”.

There’s a reason why, in civilized societies, the accused is not carrying the burden of proving their innocence – it’s damn near impossible to do so. Proving guilt, on the other hand, provided there is any, may be hard, but certainly not impossible. So far, I have yet to see more compelling evidence than oddly torn coats.

Perhaps the leap from analog and coax cables to IP and CAT5 is a leap too far for some people, and in the whirlwind of technobabble, they desperately grasp for something to hold on to. Perhaps in time they will find out that they are clinging to the branches of an old, but potent, poison ivy that has spread all over the garden.

I’m not willing to pass judgment on any camera manufacturer right now. I am willing to accept that people make mistakes. NASA burned up the Mars Climate Orbiter because someone at Lockheed Martin used imperial units! People “carelessly” installed software that contained OpenSSL, which in turn was vulnerable to the Heartbleed bug, and I could go on.

Maybe I am the ignorant one. Maybe I am not “connecting the dots”. I do see the dots, and I do see how someone is trying to make you connect them. But without evidence, I am not going to draw that line. I do have ample evidence that “the flock” are ignorant fools, so I am judging members of that flock by association (fairly or not 🙂 )

Sony IPELA Backdoor

Numerous sites now report that a backdoor has been found in several Sony IPELA cameras. 

You can update the firmware, but as self-proclaimed Messiah of IP video says: “Firmware is updated all the time, just like on a PC, and a backdoor could be injected at any point during this process” (I am still not sure if this is an attempt at humor or evidence of gross incompetence).

From the reddit post on the backdoor, you can find a link to a site that lists a lot of decrypted firmware files. These decrypted files are scanned for vulnerabilities just like sec-consult did.

 

My Hikvision Story

Got this…

cables

added this…

hikvision

next thing you know…

van

solution…

tinfoil

 

Penny Pincher

It’s been 2 years since I built my current workstation, and it’s still a very capable machine. It has an i7 3770K, 32GB RAM and a nVidia GTX 670. There really isn’t a rational reason to upgrade. While full recompilation of the source code takes 10 minutes, I rarely need to do so, and so most of the time, the limitation is really on how quickly I can type and move the mouse around.

I suppose it’s like getting a new car. How often do you really need a new car? And do you really need a car of that size, with that acceleration? In most cases, the answer is no. Yet people buy new cars they don’t need all the time.

So I might be able to rationalize that getting a trophy workstation is irrational, but normal, and so, therefore, it is OK for me to indulge. But then I look at the cost/performance of the high-end gear and then the predicament returns. Do I buy the high-end gear, that I want, but is too expensive compared the performance it offers, or do I go for the sweet spot? For example, the i7 6700K quad-core offers pretty good performance vs the more expensive 6800K hex-core CPU, but the 6700K will not deliver a noticeable performance boost compared to the 3770K…

In many ways it is similar to getting a fast car; you have seen people drive these cars fast or on winding roads mountain roads, and you might do so too, once in awhile, from time to time, but nowhere near as often as you make yourself believe when you get it. Same with the PC, I see people running GTA V and Battlefield 1 at a level of fidelity that is just mind blowing, but I know, in my heart, that I won’t spend more than an hour per month playing these games. Perhaps I am paying for the privilege of knowing that if I wanted to, I too could play Titanfall 2 at the ultra setting.

Perhaps I will just buy some DIY IoT gear and have fun with that…

 

Mirai, mirai on the wall

Update: Mirai now has an evil sister 

Mirai seems to be the talk of the town, so I can’t resist, I have to chime in.

Let me start by stating that last week’s Dyn attack was predictable.

But I think it is a mistake to single out one manufacturer when others demonstrably have the same issues. Both Axis and NUUO recently had full disclosure documents released that both included the wonderful vulnerability “remote code execution” – which is exactly what you need to create the type of DDOS attack that hit Dyn last week.

There’s not much we can do right now. The damage has been done, and the botnet army is out there, probably growing every day, and any script-kiddie worth their salt will be able to direct the wrath of the botnet towards one or more internet servers that that dislike for one reason or other. When they do, it won’t make any headlines, it will just be another site that goes offline for a few days.

The takeaway (if you are naive) is that you just need a decent password and HW from reputable sources. While I would agree on both on the principle -the truth is that in many cases it is useless advice. For example. if you look at the Axis POC you’ll see that the attacker doesn’t need to know the password at all. (I did some probing, and I am not sure about that now).

The impact?

Impact
++++++
The impact of this vulnerability is that taking into account the busybox that runs behind (and with root privileges everywhere. in all the binaries and scripts) is possible to execute arbitrary commands, create backdoors, performing a reverse connection to the machine attacker, use this devices as botnets and DDoS amplification methods… the limit is the creativity of the attacker.

And look, the affected cameras/encoders are based on BusyBox. A popular platform, and therefore a juicy target (see BashLite and ShellShock)

In other words:

Zombie

I am not suggesting that Axis is better or worse than anyone else, the point is that even the best manufacturers trip up from time to time. It can’t be avoided, but thinking that it’s just a matter of picking a decent password is not helping things.

My recommendation is to not expose anything to the internet unless you have a very, very good reason to do so. If you have a reason, then you should wrap your entire network in a VPN, so that you are only able to connect to the cameras via the VPN, and not via the public Internet.

My expectation is that no-one is going to do that, but instead they will change the password from “admin” to something else, and then go back to sleep.