Options for Enhancing Video of a Burglary?

[jes]

My home was burglarized last week. My neighbor's security camera captured two 30s video clips of the individual, one at my front door prior to forcing it, and one with him leaving with a cloth grocery bag containing property he was taking.

The video shows his face, but it's not as clear as I would like. I don't know how to quantify its quality except to say its "not bad" but "not great either."

I would welcome any suggestions for extracting and improving still images from the two clips. I'd prefer to outsource this to someone that has experience in this kind of work.

I don't have a lot of confidence that the local police are going to invest a lot of time and effort in this, only because they are overloaded.

Any recommendations for providers or other actions to take would be appreciated.

[brk]

I work in the video surveillance space. The short version is that you are unlikely to be able to do anything worthwhile with those videos.

If you want to contact me privately I can take a look at them and maybe offer some specific suggestions.

[jes]

Thank you.

I would welcome any suggestions you might have. If you'll contact me at [email protected] I'll send you the two clips. I don't see how to find your email and I'm reluctant to ask you to disclose it publicly.

[fortran77]

Sometimes, if you have multiple shots where the subject hasn't moved much, you can combine them to get accurate subpixel resolution (i.e., resolution higher than the camera has). However, this works best to subjects that are "fixed" like license plates.

Here's a "practical" way to do it:

https://www.dpreview.com/articles/0727694641/here-s-how-to-p...

And here's an example of a research paper about it:

https://arxiv.org/pdf/1801.04590.pdf

The local police will spend no time and effort on it. And any video you produce/enhance will not be able to be used as evidence by them.

[ttul]

This is what I was thinking as well. Superresolution techniques aren't going to surface information that was not already in the image. However, combining multiple sequential images of the same subject can have this effect.

The problem is that I'm sure the subject moves considerably from one image to the next. You will have to isolate his face in each frame and transform the face grabs so that they line up perfectly before the images can be combined by averaging.

It's worth a shot!

[jes]

Thank you for this.

I don't know much about this topic, but it seems reasonable to think that multiple frames ought to be useful in estimating subpixel values.

I agree with your take on the suggestion that that police won't spend time on it, and that any altered images wouldn't be admissible as evidence. They did spend considerable time taking fingerprints in my home. And even the original video is damning. I am hopeful that better quality images can be used to get the name and possible location of the suspect. I'm offering a reward of $1,000 to the first person that gives Kirkland PD information that leads to them interviewing a credible suspect.

I doubt that any of our property will be recovered. I'd like to see this individual off the streets. Perhaps someday, he will get treatment that would help him.

[himinlomax]

> And any video you produce/enhance will not be able to be used as evidence by them.

It may not be good evidence in itself, but if the person is recognized, they could be investigated and more evidence found. It's not unlikely they'd be in possession stolen property, that their clothes match the video, that they left cell phone records in the vicinity and so on.

[CharlesW]

A company called MotionDSP makes a tool called "Forensic" that costs $250/month: https://www.youtube.com/watch?v=u_B5PUYaIIA

[jes]

Wow. That seems like a pretty compelling tool. I may license for a month and see what it can do.

Thank you for taking the time to help me with this.

[behringer]

I was able to save a group photo with this technique once: https://www.gimp.org/tutorials/Smart_Sharpening/

[samstave]

It would be interesting to save a frame to a modern iPhone, edit it in the photos app AND then apply this technique to the output of the iPhone edited pic. I have been able to get a lot more detail from a photo in a dark setting on the photos editing app on iPhone.

[sansnomme]

You might want to consider reaching out to ClearView AI. Ethics aside, they probably can help you identify the burglars. With regards to the comments suggesting super-resolution, just remember that the Second Law of Thermodynamics prevents recovery of information that does not exist. Neural networks and ML can give you a good guess at what the face may look like, but you won't necessarily get more information out of it and if done incorrectly, might even get a false positive. What neural networks are useful for here is to extract the feature vectors from the face and perform facial recognition on them.

[tpmx]

The key breakthrough here (for the typically crappy footage) will probably involve

a) using many frames to ID the perp

and perhaps

b) working at a native H.264 I/P frame level with the neural networks, rather than at a decoded framebuffer level.

[nlh]

This is a totally random shot, but I wonder if Pixelmator Pro's new ML Super Resolution could be of help here:

https://www.pixelmator.com/pro/machine-learning/

You could grab individual frames of the video and run them through this and see what the results look like?

(no connection to Pixelmator - just played with the software and it looked pretty darn cool)

[drcode]

There is a reason Google's initial DNN project was called "deep dream"- It's because it essentially uses "dreaming" to generate new content based on a source database of unrelated images.

It is making up new image details (highly convincing and realistic new image details) and hence this tech is completely inappropriate for handling criminal evidence.

[jes]

Thank you for this. I'll take a look at Pixelmator Pro and their "ML Super Resolution." It seems credible to me that some kind of ML approach ought to be useful here.

[hackpert]

You might want to try out some stacking based resolution improvement methods, like the ones used in astronomy. They take data from multiple "sources" and combine them to produce a high resolution picture.

For example, check out https://www.astronomie.be/registax/

[throwaway_tech]

Beyond the video...Depending on what was stolen, if it is likely to be sold, then monitor craigslist, offer up and FB garage/yard sale groups in your area.

[pintxo]

Second that. Actually got 4 tires/rims back that way (found them on ebay)

[wizzerking]

OpenCV has a bunch of image processing algorithms that I searched for, but don't see mentioned CLAHE Contrast Limited Histogram Equalization _ I have used to good effect to get better contrast Neural Network DeNoising https://papers.nips.cc/paper/4686-image-denoising-and-inpain...

https://arxiv.org/pdf/1807.08176.pdf

I'm sure github has some repos on this

[piracy1]

You can enhance the overall image quality using: https://github.com/alexjc/neural-enhance on every frame. https://letsenhance.io/

For faces: https://www.geoffreylitt.com/2017/06/04/enhance-upscaling-im...

[solveit]

I would strongly recommend against neural enhancing for identifying faces. There's a good chance the face will get "enhanced" into somebody in the training set that kinda looks similar to the blurry photo.

[jerome-jh]

With a video player/editor, locate the I-frames (or simply the best looking ones, playing frame by frame), extract them, then use _any_ photo editor to sharpen, adjust contrast/luminosity, enlarge, etc, and print them. It is not rocket science and will give you something to show to the police/whoever, in about an hour of work.

Before investing real money on this, be aware of the legal value of these videos: do we really see the burglar forcing the front door? Taking your property? Is your neighbour entitled to film that area? The answer strongly depends on your local laws.

[m3kw9]

If you have the time stamp you could also reference other areas where he could have walked and get more footage. But the faster you go and get these cottage the better as some are not kept after a few days

[cj]

I've always assumed those scenes in TV shows where they pull up blurry photos / video and "enhance" them 10-fold was just hollywood.

Can this actually be done to any degree of usefulness?

[s_kilk]

Correct, if the information simply isn't present no amount of trickery will conjure it up.

[jaclaz]

Not as much as you see on TV, of course, particularly the "zoom in" that you see there, but there are specific enhancers that sometimes do what appear like "miracles".

If you want to test one, for "motion blur" (let's say the licence of a car driving away), get this (last freeware version):

https://github.com/Y-Vladimir/SmartDeblur/downloads

And test it on the images here (actual test images from a reknowned Commercial vendor of a suite that is also for videos):

https://articles.forensicfocus.com/2014/10/08/can-you-get-th...

[jes]

Here are links to the two videos:

https://www.dropbox.com/s/jkyxlkh0aa5d24m/IMG_5402.mp4?dl=0

https://www.dropbox.com/s/oxgwpe6bn39p0re/IMG_5404.mp4?dl=0

[rs23296008n1]

First video shows he touched a few things. Perhaps fingerprints off those? Those clothes would mark that guy depending on locale. Perhaps other cameras saw him as well?

Also, now you know your camera isn’t doing the best, add other camera(s) where it would have helped. Eg to capture face-on angle as you approach door. Consider upping the resolution of the cameras you add.

On a more general note, rethink about how this guy found your place as a suitable target. Mitigate what you can. He knew he had plenty of time. And he seemed quite comfortable being there.

[throwaway_rock1]

One observation is that he had a different way of walking, particularly the way his foot landed. Combine this with the fact that he was comfortable being there means that he is from around your place

[lisk1]

Topaz Labs have some amazing photography enhancing tools that can help. They are claiming they use AI for the enhance. Topaz Studio have this feature ClearAI for me even on standard settings is doing an amazing job to clear the photos. For batch processing you have to use it as a Photoshop plugin , the frames of interest from the videos have to be extracted as separate jpeg photos.

[ubercow13]

Using AI to enhance CCTV footage for the purpose of identifying criminals seems somewhat unfair to the individuals whose photos were used to train those AI models.

[andrewtbham]

This is not a practical recommendation by any means... but fast ai has a lecture on image restoration with deep learning.

https://www.fast.ai/

https://youtu.be/9spwoDYwW_I?t=2913

[mackman]

Oh god this is a terrible idea when talking about identifying a suspect. You’re basically making an educated guess to construct evidence that could then be used to falsely accuse someone. Please don’t do this.

[akeck]

It may or may not work, but try applying the below project to a few frames to see what you get. Make sure to preserve the original footage on the original media, though.

https://github.com/alexjc/neural-enhance

[c3534l]

You can't get information from a photograph that isn't in that photograph in the first place. Any software technique that purports to fill in those blanks, like a neural network, is drawing a face in based on what it knows about faces in general. The face you get may look realistically like a face, but it won't look like the face of the person who robbed you, except perhaps by pure coincidence.

[theincredulousk]

Video frames can be composited to improve clarity the same way it is done in Astrophotography.

And your assertion that "any software technique...won't look like the face" is incorrect. Again using multiple frames, it's entirely possible to infer what clearer underlying detail would result in the less-sharp pixel values in the video, given the training data establishing the relationship. It's a specific technique for reconstruction, not just drawing in a realistic but synthetic replacement like Photoshop's content-aware fill.

[jolmg]

I think the idea is to merge the multiple bad images available from the videos to make one good image. They're not asking to improve the quality of the image using just one image.

[katyusha]

https://www.ghacks.net/2019/10/23/videocleaner-is-an-open-so...

[ravenstine]

The tools provided by Topaz Labs are incredible for enhancement and have free trials.