IDEA: How to improve the detection reliabiliy

Tips, tricks and frequently asked questions
Post Reply
openMind
Posts: 17
Joined: 23 Feb 2017, 09:31

IDEA: How to improve the detection reliabiliy

Post by openMind » 11 Jul 2017, 13:54

Hi dear Netatmo Team.

I noticed that the detection algorithm of the presence cam seems "just" to analyse the camera stream picture by picture and does not take care (or understand) the context of the event relative to a previous detection.
Often I see event where the camera first detects i.e. is an animal and in the next second(s) at nearly the same position is a person. I nearly all cases one of them is wrong. Often it is because the object which was causing the detection is changing it's shape significant (like a cat. 1st detection is from the side and detected as a animal, than the cat turns around and the 2nd detection is from its back... looks much more like a person and ... bam...), or when the object becomes partially covered by surroundings (like a cat which is going around a corner like in picture 1 and 2).

Here a some examples (with firmware 101 I have a lot of them right now):
Screenshot_20170711-131928.jpg
Screenshot_20170711-131853.jpg
Screenshot_20170711-131832.jpg
The marked preview is the one which was wrongly detected as person.

From a developer point of view, this should by avoidable as the detections are very close to each other (in terms of position and in terms of time).
As I guess that the detection algorithm has also some kind of prediction value (how likely the detection is true) this together should allow to avoid the majority of false detection that I expirience with my two presence cam's.

Best regards

Subzero
Posts: 249
Joined: 08 Oct 2016, 14:03

Re: IDEA: How to improve the detection reliabiliy

Post by Subzero » 13 Jul 2017, 12:57

Interesting point, that does make sense. And maybe this is why lots trigger my floodlight, when I only have it set to person, and the recording still says car for instance? It must at some point in the frame think person, trigger the floodlight, and not see it in context as you say?

guillaumeM_Netatmo
Posts: 4
Joined: 02 Jan 2017, 11:40

Re: IDEA: How to improve the detection reliabiliy

Post by guillaumeM_Netatmo » 13 Jul 2017, 15:45

Hi,
Thanks for your interest in our products.

We already have strategies to deduplicate events in time and space for individual trackers but we think that this not the problem in your case.

The main issue is that our classifier did not predict the right category (it made a mistake) and we cannot neglect potential human (it is a security camera). Your only option at this point is to improve detection and correct the misclassification by hand (please make sure your participate to improve our product). This action will be taken care in a subsequent release.

The general rule of thumb is that you cannot do that much if the classifier did a mistake in the first place and this should be corrected first.

Hope it helps

openMind
Posts: 17
Joined: 23 Feb 2017, 09:31

Re: IDEA: How to improve the detection reliabiliy

Post by openMind » 14 Jul 2017, 10:50

Thanks for your answer.

I do participate to the "Improvement program" by allowing it in the app settings. But I click those buttons to "improve the detection" since months several times the day. But the overall rating of false detection seems not changed so far. Especially this black cat seems to by an issue for your classification algorithm.

I understand your point that it is a security camera and changes to the classification have the potential risk that humans are less likely detected, but this is also a question of balancing. If I receive a false detection several times a day, this does decrease the awareness of those notifications more and more. If I receive those alerts several times during the night, I am forced to turn it off... this way it is hard to see the benefit of a security camera like this.

As the camera seems to analyse each picture individually (and does not take detection's done 1 second before into account), there are a lot of avoidable mistakes in the detection. For example 3 times it detects a cat walking (or at least moving) totally correct. And suddenly it is just using a crop of it (a crop of the cat) and this part is detected as a human...

Let me know if I should upload some example videos of those false detection to help your developers to improve there classification algorithm.

In the middle term maybe add a layer using stuff like TensorFlow (https://research.googleblog.com/2017/06 ... odels.html) could by a game changer...

openMind
Posts: 17
Joined: 23 Feb 2017, 09:31

Re: IDEA: How to improve the detection reliabiliy

Post by openMind » 19 Jul 2017, 07:37

Just to show that firmware 101 has serious improvement needs. Especially when it comes to "black cats" :? :roll: :lol:

This was just this morning...
today.PNG
Once again.

Let my know if you need footage to improve your classification algorithm.

Best regards

fsf976
Posts: 63
Joined: 03 Mar 2017, 10:31

Re: IDEA: How to improve the detection reliabiliy

Post by fsf976 » 10 Aug 2017, 16:09

I have a large black dog and see a similar issue.

Post Reply

Return to “General questions”