Net Neutrality

You can’t be against net neutrality, and, at the same time, understand how the Internet works.

There is no additional cost to the IPS to offer access to obscure sites; it’s not like a cable package where the cable provider pays a fee to carry some niche channel that no-one watches.

Basically, net neutrality means that the ISP has to keep the queues fair; there are no VIP lanes on the Internet. Everyone gets in the same line, and are processed on a first come, first served basis. This is fundamentally fair. The business class traveler may be angered by the inability to buy his way to the front of the line (at the expense of everyone else), but that’s just tough titties.

It’s clear that not everyone has the same speed on the Internet; I live in an area where the owners association decided against having fiber installed, so I have a shitty (but sufficient) 20/2Mbit ADSL connection. My friend across the bridge, in Sweden, has a 100/100Mbit at half the cost. But that has nothing to do with net neutrality.

If my friend wants to access my server, my upstream channel is limited to 2 Mbit per second. This is by my choice, I can choose to host my server somewhere else, I could try to get a better link and so on, but basically, I decide for myself who, and how much I want to offer. There are sites that will flat out refuse to serve data to certain visitors, and that’s their prerogative.

However, with net neutrality removed, my site may get throttled or artificially bottlenecked to the point where people just quit visiting my site. I would have to deal with several ISP’s and possibly have to pay them a fee to remove the cap. If the site is not commercial* I may not have the funds to do that. I may not be aware that an ISP is throttling my site into oblivion, or even be offered an option to remove the cap.

Clearly, ending net neutrality is not the end of the world. Guatemala and Morroco are two examples of countries w/o net neutrality. In Morroco, the ISPs decided to block Skype, since it was competing with their (more profitable) voice service, so that might give you a hint of what’s to come. They did complain to the King when the ISPs went too far though.

Naturally, fast access to Facebook LinkedIn and Snapchat might be cheaper, and probably all you care about if you’re against NN.

With cloud-based IP video surveillance starting to become viable, this might prove to be another, unpredictable cost of the system. Some ISPs already take issue with you hosting a web server via your retail connection. And they go out of their way to make it difficult for you to do so: Changing your IP address every 4 hours and so on. This is to push you into a more expensive “business plan”, where they simply disable the script that changes your IP. I think it is safe to assume that if you’re streaming 30 MBit/s 24/7 to an Amazon data center, the ISP will eventually find a way to make you pay. And pay dearly. Once you’ve hooked your entire IP video surveillance system into the cloud, what are you going to do? Switch to another ISP? #yeahright

I guess the problem is that the ISP business model used to be to sell the same bandwidth 100 times over. Now that people are actually using the bandwidth, that model falls apart, and the ISPs need other means to make sweet sweet moolah. And that’s their nature and duty. But why cheer them on?

*In the early days, commercial activity on the Internet was banned.

 

Advertisements

HomeKit Flaw

https://9to5mac.com/2017/12/07/homekit-vulnerability/

Does this vulnerability shipping mean you shouldn’t trust HomeKit or smart home products going forward? The reality is bugs in software happen. They always have and pending any breakthrough in software development methods, they likely always will. The same is true for physical hardware which can be flawed and need to be recalled. The difference is software can be fixed over-the-air without a full recall.*

*Unless it’s a Chinese IP camera, then all “mistakes” are deliberate backdoors put in place by the government.

In Defence of Hikvision

Look at this nonsense!

Brian Karas reported on March 2 that he was hearing from multiple Hikvision security camera and DVR users who suddenly were locked out of their devices and had new “system” user accounts added without their permission.

Karas said the devices in question all were set up to be remotely accessible over the Internet, and were running with the default credentials (12345). Karas noted that there don’t appear to be any Hikvision devices sought out by the Mirai worm — the now open-source malware that is being used to enslave IoT devices in a botnet for launching crippling online attacks (in contrast, Dahua’s products are hugely represented in the list of systems being sought out by the Mirai worm.)

[I cut out some text from here (I’ll tell you why)]

According to Karas, Hikvision has not acknowledged an unpatched backdoor or any other equivalent weakness in its product. But on Mar. 2, the company issued a reminder to its integrator partners about the need to be updated to the latest firmware.

OK, so Brian hears that people who a) expose their IP cameras directly to the internet, and b) are using default admin credentials “suddenly were locked out of their devices”. My God, what kind of evil genius hacker is behind this, and there were new “system” user accounts!!?!

This must be the Chinese government’s work. Only a government organisation would be able to crack into IP devices with default passwords that are directly exposed to the internet.

When people got their shit “hacked”…. actually, let’s not call it hacked. Someone logged in, as admin, and changed things, so, not hacking. Someone had done something similar to mirai (which will take any script kiddie 30 minutes to write up, but Karas and Krebs pretend to not understand that). Hikvision sees this, and then reminds people to update their firmware, and as the new firmware does not allow default passwords (as far as I can tell), it seems prudent advice, and what you ought to do.

Krebs seems to want to play a part in all this “dangerous Hikvision camera” bullshit, so instead of posting a meaningful timeline, he spices things up, and injects this little tidbit (which I removed above to ensure a comprehensible timeline).

In addition, a programmer who has long written and distributed custom firmware for Hikvision devices claims he’s found a backdoor in “many popular Hikvision products that makes it possible to gain full admin access to the device,” wrote the user “Montecrypto” on the IoT forum IPcamtalk on Mar. 5. “Hikvision gets two weeks to come forward, acknowledge, and explain why the backdoor is there and when it is going to be removed. I sent them an email. If nothing changes, I will publish all details on March 20th, along with the firmware that disables the backdoor.”

OK, so on the 2nd the n00bs at IPVM and their subscribers are “hacked” by a genius hacker, who is able to guess the password and add new accounts, and then on the 5th, a guy who re-compiles the hikvision firmware discovers a vulnerability. In fact, he tells John Honovich that Hikvision has been very responsive in fixing the issue!! This seems to get lost somewhere between the sensationalist blogs (I think, because I am banned from IPVM).

How the hell do you make a connection between morons who exposes their cameras with default admin credentials, and someone discovering a bug in the validation of a reset packet (I guess that is the vulnerability, because I don’t know the details). You make that connection, if you think it will bring in more subscribers, and by extension, more filthy lucre.

Full disclosure: I am not paid in any way shape or form by Hikvision or any camera manufacturer for that matter. I receive no payment from this blog either, the ads you might see are put there by wordpress that hosts the blog, as compensation for hosting and traffic cost (and profit I guess), but I receive exactly $0.

Influencers

When Netflix* just started rolling out their Internet streaming service I used a VPN service, and streamed a wonderful movie called “Human Centipede”. It’s a surprising accurate portrayal of how things work in my industry, except the labia tua to sphincter connection is entirely voluntary, whereas the victims in the movie were not given an option.

I wonder if the toilet paper industry celebrates “influencers“?

Because, frankly, most people need IP video like they need toilet paper, and it should be just as simple to use. I’ve been to many places on God’s green earth and not once did I encounter toilet paper that wasn’t compatible, or was complicated to work with.

Yet, we “celebrate” leaders that take something that ought to be just as simple, and turn it into a nightmarish ordeal to install and maintain. If you are influential, then you must share some of the blame for the pile of shit that this industry is serving up again and again.

* It’s hard to fathom that a mail-a-DVD-business would be a major driver of the entire US economy, but there you go)

Hikvision Feeds a Troll

It’s possible to turn someone towards the light, and eventually lead them to salvation.

A prominent member of the Westboro Baptist Church, Megan Phelps-Roper made a TED speech about it. What saved Megan was not someone yelling in her face. She was conditioned to expect exactly that from the misguided heathens of the world. Instead, someone approached her with curiosity, warmth and civility and lead her out of the congregations grasp. The “enemy” is rarely a mindless drone out to do evil. Although, our leaders would prefer we see things that way.

Our industry has a variant of the WBC, and Hikvision has chosen a different approach to liberate the members of the sect.

a site that has always trafficked in nefarious insults and innuendo. Hiding behind a keyboard, the tabloid’s staff takes unfounded potshots at our entire industry, bullying one company at a time.

and

Instead, he chooses to distract manufacturers with his pursuit of financial gain and efforts to fulfill his delusions of grandeur.

The problem with this sort of message is that the hardcore members are expecting exactly this sort of rhetoric, thereby further entrenching them in their beliefs. Ultimately the blogger will surely capitalize of the increased attention being paid. I thought it was common knowledge that trolls have an insatiable appetite for the kind of copy Hikvision just released.

jh
More please!

Members of the sect can attend a “university” (not at all like this one) and even make the “dean’s list“. This is impressive stuff, and these people are not going to be swayed by a manufacturer having a breakdown in their public relations department. Furthermore, I suspect Hikvision has several active subscriptions, thereby directly funding the site.

I think Hikvision is correct in calling it cyber-bullying. It has all the traits of schoolyard terrorism; the ring-leader points out an arbitrary enemy, then manipulates the enemy to react. Steps in to protect the flock from the aggressor. If it gets too hot, the ring-leader can count on his 3 or 4 lackeys to do the dirty work.

In this case, the sin of the “enemy” is that the company is partially owned by the Chinese government. Therefore, every vulnerability found in a Hikvision camera is proof positive that the Chinese government is spying on us. I don’t buy that. Governments don’t have to own a company to assert influence over it.

You might remember Stuxnet a vulnerability in SCADA equipment was exploitable by governments and for-lulz hackers alike. Vulnerabilities will continue to exists as long a fallible humans write the code. As long as fallible humans install and (fail to) maintain the equipment we will continue to see flaws and problems. Unfortunately, a lot of companies have deployed small time-bombs with terrible security in place, not just Hikvision.

When I was in the army, we had padlocks on our lockers. On the first day, we were instructed to get a hair-cut “to not look like faggots” (I kid you not, that’s what he said), and then to make sure our lockers were safely locked. The reasoning (for locking up) is that you can’t really trust anyone, and giving the bad apples the opportunity to steal was almost as bad as the guy stealing. At a company I worked for a long time ago (starts with an M), someone shat on the toilet seat in the offices restroom. Someone we had lunch with, talked about code, movies, politics and music with, went to the bathroom, and shat on the seat, leaving it there for some poor soul to find.

Same thing goes for your IP camera. Sticking that thing on the internet, REGARDLESS of manufacturer ownership is like leaving your locker unlocked. You are tempting the swines of the world to mess around, and when they do, we all lose.

madness

Clickbaiting Copycat Caught

It’s pretty damn hard to make secure software. Years ago I commented on Shodan and worried that the IP video industry was next.

Run of the mill ignorance, carelessness, greed what have you, is so common that we scarcely care to click the link. Recently (or not) and old bug was discovered in Intel products that allowed remote control.

Now if you are commercial blogger (or “analyst” if you prefer), you’re not going to try to shed light on the issue. That just doesn’t trigger enough clicks and drama. It’s better to make some unsubstantiated claim that an “Intel backdoor is confirmed”.

ipvm

I can guarantee that someone is now looking up the word “backdoor”, I’ll save you the trouble (it’s in the link above too)

A backdoor is a method, often secret, of bypassing normal authentication in a product, computer system, cryptosystem or algorithm etc. Backdoors are often used for securing unauthorized remote access to a computer, or obtaining access to plaintext in cryptographic systems.

Wikipedia

So, yes, it is probably not a lie to use the word “backdoor”, but it sure is manipulative, something people with a certain mental defect excel at.

For l33t hackers, finding back-doors is sometimes a fun pastime. The purpose can be to cause extensive damage for lulz or filthy lucre, sometimes for companies, sometimes for governments. Usually, it’s a challenge to find vulnerabilities and defects that let’s you crawl into systems that should be locked down. But to the n00b, a backdoor might suggests that it was intentionally put there. After all, you don’t “accidentally” install a backdoor in your house.

Backdoors in code, however, come in various flavours,

  • Deliberate backdoor intended to give an unknown user remote access after the user has deployed the device/software, thereby granting the attacker access. These can be baked into the device, or installed later as a trojan.
  • Accidental backdoor caused by unexpected side-effects of the code. In the olden days, you could mess around IIS servers by using unicode strings in the URL.
  • Accidental backdoor caused by gross negligence/incompetence on the manufacturers side. Hardcoded credentials is an example of such foolishness.

Today you are not going to get away with #1 and #3 for very long. The hackers at blackhat are not like mortal programmers, they understand assembly code, and will locate a hardcoded password or a backdoor in a few days.

But it’s a gradual scale from #2 to #3. For example, HTTP used to have something called “basic authentication“. It used Base64 encoding to hide the credentials in flight, and plenty of cameras and VMSs would use it. 15 years ago, basic authentication would probably have been considered a #2 issue, but today it’s clearly a #3 (a certain unmentionable blog used it not long ago).

You can make up your own mind if CWE-287 is a #1, #2 or #3. It could, conceivably, be a #1. But it will be very difficult to prove, unless you have network captures showing malicious activity initiated by someone associated to the manufacturer (US tech companies and NSA for example).

Another company was notified of a vulnerability on March 5th 2017, on the 12th a security bulletin is released, and the hacker then states :

“I have been communicating with Hikvision since I notified them and they have actually been been quite responsive.”

Quite responsive indeed.

Eventually we will have software in IP cameras that is safe enough that you can expose it to the internet. But for now, I would be extremely careful about opening my CCTV system to the internet.

In Hikvisions case, I think one of the issues is that to reset the cameras password you need to send a specially crafted payload to the device. This causes a lot of issues for lots of users and it strikes me as a potential attack vector. And rest assured that this is not the only issue in the cameras.

As time passes hackers find ways into older cameras that have long been discontinued, but have been deployed and are still operational, they may get more sophisticated in their attacks and find more complex ways of breaching the software.

I guess this was not as exciting a post as you had expected. I’m sorry. You will have to go somewhere else for BREAKING NEWS about the evil Chinese shell companies set up only to spy on you.

 

 

 

Magical “GPU” Based Video Decoder

I was recently alerted to an article that described a magical video decoding engine. The site has a history of making odd conclusions based on their observations, so naturally, I was a bit skeptical about the claims that were relayed to me by a colleague. Basically, the CPU load dropped dramatically, and the GPU load stayed the same. This sounded almost too good to be true, so I did some casual tests here (again).

EouEzI5bBR8uk.gif

Test setup

I am not thrilled about downloading a 2 GB installer that messes up my PC when I uninstall it, and running things in a VM would not be an honest test. Nor am I about to buy a new Intel PC to test this out (my next PC will be a Ryzen based system), so all tests are done with readily available tools: FFMpeg and GPU-Z. I believe that Intel wrote the QSV version of the h264 decoder, so I guess it’s as good as it gets.

Tests were done on an old 3770K, 32 GB RAM, Windows 7 with a GeForce 670 dedicated GPU. The 3770K comes with the Intel HD Graphics 4000 integrated graphics solution that supports Quick Sync.

Terminology

In the nerd-world, a GPU usually means a discrete GPU; a NVidia GeForce or AMD Radeon dedicated graphics card. Using the term “GPU support” is too vague, because different vendors have different support for different things. E.g. NVidia has CUDA and their NVEC codecs, and some things can be done with pixel shaders that work on all GPUs. (our decoding pipeline uses this approach and works on integrated as well as discrete GPU, so that’s why I use the term GPU accelerated decoding without embarrassment).

However, when you rely on (or are testing) something very specific, like Intel Quick Sync, then that’s the term you should use. If you say GPU support then the reader might be lead to believe that a faster NVidia card will get a performance boost (since the NVidia card is much, much faster than the integrated GPU that hosts Quick Sync). This would not be the case. A newer generation of Intel CPU would offer better performance, and it would not work at all on AMD chips with a dedicated GPU (or AMD’s APU solution). Same if you use CUDA in OpenCV, then say “CUDA support” to avoid confusion.

Results

Usually, when I benchmark stuff, I run the item under test at full capacity. E.g. if I want to test, say the CPU based H264 decoder in FFMpeg against the Intel Quick Sync based decoder, I will ask the system to decode the exact same clip as fast as possible.

So, let’s decode a 720p clip using the CPU only, and see what we get.

CPU

The clip only takes a few seconds to decode, but if you look at the task manager, you can see that the CPU went to 100%. That means that we are pushing the 3770K to it’s capacity.

CPU_FPS

Now, let’s test Quick Sync

QSV

Not as fast as the CPU only, but we could run CPU decoding at the same time, and in aggregate get more…. but we got ~580 fps

QSV_FPS

So we are getting ~200 fps less than the CPU-only method. Fortunately, the CPU is not being taxed to 100% anymore. We’re only at 10% CPU use when the QSV decoder is doing its thing:

CPU_QSV

Magic!!!

But surprisingly, neither is the GPU. In fact, the GPU load is at 0%

GPU_QSV

However, if you look at the GPU Power, you can see that there is an increased power-draw on the GPU at a few places (it’s drawing 2.6W at those spikes). Those are the places where the test is being run. You can also see that the GPU clock increases to meet the demand for processing power.

If there is no load on the GPU, why does it “only” deliver ~600 fps? Why is the load not at 100%? I think the reason is that the GPU load in GPU-Z does not show the stress on the dedicated Quick Sync circuitry that is running at full capacity. I can make the GPU graph increase, by moving a window onto the screen that is driven by the Intel HD Graphics 4000 “GPU”, so the GPU-Z tool is working as intended.

I should say that I was able to increase performance by running 2 concurrent decoding sessions, getting to ~800 fps, but from then on, more sessions just lowers the frame rate, and eventually, the CPU is saturated as well.

Grief

To enable Quick Sync on my workstation which has a dedicated NVidia GeForce 670 card on Windows 7, I have to enable a “virtual” screen and allow windows to extend the display to this screen (that I can’t see because I only have one 4K monitor). I also had to enable it in the BIOS, so it was not exactly plug and play.

Conclusion

I stand by my persuasion: yes, add GPU decoding to the mix, but the user should rely on edge-based detection combined with dedicated sensors (any integrator worth their salt will be able to install a PIR detector and hook it up in just a few minutes). This allows you to run your VMS on extremely low-end hardware and the scalability is much better than moving a bottleneck to a place where it’s harder to see.