Category: Coolness

InfluxDB and Grafana

InfluxDBGrafana

When buzzards are picking at your eyes, maybe it’s time to move a little. Do a little meandering, and you might discover that the world is larger, and more fun, than you imagined. Perhaps you realize that what was once a thriving oasis has now turned into a putrid cesspool riddled with parasites.

InfluxDB is what’s knowns as a “streaming database”. The idea is that it’s a database that collects samples over time. Once the sample is collected, it doesn’t change. Eventually the sample gets too old, and is discarded. This is different from traditional databases where the values may change over time, and the deletion of records is not normally based on age.

This sounds familiar doesn’t it?

Now, you probably want to draw some sort of timeline, or graph, that represents the values you popped into InfluxDB. Enter, Grafana. It’s a dashboard designer that can interface with InfluxDB (and other databases too) and show pretty graphs and tables in a web page w/o requiring any HTML/Javascript coding.

If you want to test this wonderful combination of software, you’ll probably want to run Docker, and visit this link.

Now, I’ve already abandoned the idea of using InfluxDB/Grafana for the kind of stuff I mess around with. InfluxDB’s strength is that it can return a condensed dataset over a potentially large time-range. And it can make fast and semi-complex computations over the samples it returns (usually of the statistical kind). But the kind of timeline information I usually record is not complex at all, and there aren’t really any additional calculations I can do over the data. E.g. what’s the average of “failed to connect” and “retention policy set to 10 days”.

InfluxDB is also schema-less. You don’t need to do any pre-configuration (other than creating your database), so if you suddenly feel the urge to create a table called “dunning” then you just insert some data into “dunning”. You don’t need to define columns or their types etc. you just insert data.

And you can do this via a standard HTTP call, so you can use curl on the command line, or use libcurl in your c++ app (which is what I did).

The idea that you can issue a single command to do a full install of InfluxDB and Grafana, and then have it consume data from your own little app in about the time it takes to ingest a cup of coffee says a lot about where we’re headed.

Contrast the “open platforms” that require you to sign an NDA, download SDKs, compile DLLs, test on 7 different versions of the server and still have to nurse it every time there’s a new version. Those systems will be around for a long time, but I think it’s safe to say they’re way past their prime.

 

 

HDR and Low Light

In the early days, the only thing that mattered was pixel count. The more pixels you could cram onto a sensor, the better. Some people noticed that the higher pixels count would decrease the performance in low light scenes (for the same sensor size), and we got to a point where you’d have ridiculously high resolutions on extremely small sensors, but you’d need to be a few miles from the sun in order to get good, sharp footage.,

There are all sorts of software tricks that you can employ to improve the appearance of the image to the casual observer, but you can’t conjure up data that just aren’t there. One trick is to use very long exposure, but that causes moving objects to get blurry, or you can do noise reduction, but that also removes actual details.

So what happened that caused the cameras to suddenly improve quite dramatically over the last couple of years? The sensor guys came up with back-illuminated sensors. Basically, the sensors got a hell of a lot better, and now we’re reaping the benefits.

bi_sesnor

Xda-Developers has a great article on the Sony IMX378 explaining BI sensors and how HDR is achieved. And of course, there’s always Wikipedia.

Competition is a wonderful thing.

 

 

Radars

 

A while back I got fed up with people parking their cars right in front of my driveway, and I decided to find a solution.

A camera could work, but since I am cheap, I decided to look for something a bit more… economical. A PIR sensor wouldn’t work because it triggers when there is motion, and cars and people pass by all the time, so I looked into ultrasonic sensors and eventually radars. If the distance measured drops below a pre-defined threshold and stays there, I know to run into the street, yelling and screaming.

The inspiration came from Adafruit and Andreas Spiess who has a great YouTube channel where you can get more information about ultrasonic sensors and radars (and about 1000 other things).

 

Basically, you can get an Arduino capable board. I hope the WiFi capable ESP8266 will work (since I have one lying around). Then get some cheap sensors from China via Alibaba and you’re ready to experiment. At the very least, it should give you some idea of the base cost of such a device.

Both Axis and Avigilon have launched commercial versions of miniature radars that can interface to your favorite VMS. Combined with a PTZ camera this might be a very interesting combination that offers a bit more smartness than the good old PIR/PTZ combo.

 

(Dis)honesty

It’s very hard to differentiate between stupidity and intentional dishonesty. Yesterday, I made a mistake of posting an inaccurate portrayal of statements made by another blogger. I removed it from my blog almost immediately. But subscribers to my blog had received the post, and one reader quickly accused me of “defamation”, and told me that I was “on notice”.

Mistakes are made, I attempt to correct them as quickly and as best I can, but sometimes people just crave confrontation.

I’d like to take the opportunity to talk about true dishonesty. I’m a big fan of Dan Ariely, who studies behavioural economics. When Salty Features wanted to crowdfund a movie about dishonesty, centered around Dan’s work, I did not hesitate to sign up, and I can only recommend that you go watch (Dis)honesty (it’s on Netflix in Denmark and most other places I would imagine).

Dan and his team tries to establish a baseline for how dishonest people are, in general. The experiment is quite simple. They hand out math tasks. The more tasks are solved, them higher the reward (a few dollars). But, instead of handing the papers over for verification, they are to send them straight to a paper shredder. Then proceed to the controller’s desk, and simply state how many tasks were solved. No chance of verification.

The rigid financial theory says that there is 0% chance of being caught, so they might as well say that they solved all the tasks, and reap all the reward.

But that’s not what happens. A lot of people cheat a little bit, some not at all, and some go all in (in accordance with the rigid theory). The completely honest and completely dishonest people are outliers.

How did they know that the students cheated? The shredder was rigged. Some people might have suspected this, but even if they did, there would be no repercussions to go all in on the cheating, so some people are just naturally honest, while some use every opportunity to the fullest.

The experiment was then changed, so that instead of direct payment, the students would receive tokens, that would then be exchanged to real cash. Most people would probably assume that this made no difference at all, but as it turns out, this added abstraction of cheating = cash, increased the cheating substantially. The end result was the same, but one step was interjected that really should have mattered, but it did.

So, what if we look at the business world? Dan argues that the abstraction of money in high finance might be a contributing factor in what he describes as cheating, but it’s not just banks. As Dan’s research shows, a little bit of cheating is common – natural – even.

We can also illustrate the principle in another way:

Here’s an example of something that everyone will agree is problematic:

Company A pays B (who claims “independence”) to write about competitors of Company A.

Anyone in their right mind would say that B is hardly independent, and who would ever trust anything B has to say regarding companies in A’s domain? I know that a few would. There’s always a few. But most wouldn’t.

How about this, then:

Company A pays Company C to sign up to a service from Company D that is owned by B.

The end result is the same, but we’ve inserted a few layers of abstraction between A and B. And as Dan’s research show, there’s always a few that exploit this as far as it can go.

I highly recommend Dan’s books. And may I suggest you visit his sites too.