Picking the best photo and Motion detection.

When the camera records a sequence of images, some of images are going to show the approach and one image are going to the vehicle large and centered. I want to pick the  image where the vehicle is best seen so that 1) I can use it to best represent the sequence on an overview page 2) can feed a few good images to a potential License plate recognition software.

Purple box around picked image.

In the sequence to the left, the images around the middle of the sequence gives us what we want. So why not just pick the middle image.

Picking the middle image in the sequence will produce the right image in 70% of the cases.

Problem 1 with picking a photo at a certain position in the sequence is that these photos might work well for vehicles travelling in one direction, but will be completely wrong in the other direction. (Neighborhood recording one-way streets are in luck)

Problem 2: Changing the Motion detection sensitivity will completely change any setting based on position in the sequence.

Problem 3: middle image in a sequences that contains 2 evenly spaced vehicles will show the empty road between the two vehicles.

The question: is it possible to programatically a) select the right photo every time and b) determine when there is more than 1 vehicle in the sequence and split the sequence into two or more sequences, one for each vehicle.

To do this we have to not only look at motion detection but to determine where the moving object is in the photo and then pick the photo where the object is the largest without being partially out of the frame.

The first internet search took me to this fine page, where I found this algorithm

	h1 = Image.open("image1").histogram()
	h2 = Image.open("image2").histogram()

	rms = math.sqrt(reduce(operator.add,
		map(lambda a,b: (a-b)**2, h1, h2))/len(h1))

The algorithm is quite popular and quoted many times on the internet when there is talk about comparing images.

What is does is comparing image histogram and returning a single number showing how different the images are, with 0 meaning the images are identical.

Hey piece of Cake! I just have set a threshold value and when the difference is biggest the car is biggest in the photo….

Lets see how that worked in real-life:

  1. The difference between the 1st (empty road) and 2nd image is low  (<10) – so far so good
  2. As the car approaches the difference to the 1st photo increases – (50, 65, 85) hey I am onto something
  3. As the car is right in the middle of the picture the difference between the empty road and the photo is 535  – This might actually work
  4. As the car is leaving the picture stage right the difference drops to 462 – still on track
  5. Now we have empty road again and the difference is 437 – hey wait this is wrong. How can two pieces of empty road be as different as if there had been a minivan in one photo?

The RMS Difference between these two images is as much as if there had been a big minivan in one of the images

.If you look carefully you can see a slight shade difference between the two images. the Sun moved …. (it has a tendency to do that).
Loading the images into Photoshop and examining the histograms, I can see that the histogram is almost identical but shifted between the images.

After hours of playing with the above algorithm, I can conclude that the algorithm is really good at detecting changes to a photo such as adding text or manipulating a small block of the photo, but it is useless at detecting differences between two scenery photos, because the scenery is constantly changing.

The next algorithm search took me to examining the difference function in the Python Image library (PIL). Again the the entire picture is different ;-(, but reducing the number of colors, and the resolution caused me to generate this image of a minivan, with all the changes contained within the box.

Box outlining the changes in the photo.

 

 

 

 

 

 

 

From there I made the current algorithm which is pretty good for a first attempt.

#compare current image with Master and make a box around the change
diff_image = ImageOps.posterize(ImageOps.grayscale(ImageChops.difference(master_image, cropped_img)),1)
rect = diff_image.getbbox()
if rect != None:
	ImageDraw.Draw(cropped_img).rectangle(rect, outline="yellow", fill=None)

I highlight the detected area of change on the original thumbnail to easily determine effectiveness.

Yellow box highlights the are of the detected vehicle

Hmm,I detect the roof, the front bumper, then larger and larger parts of the vehicle, there is definitely room for improvements here.

This is as far as I currently am on detecting the vehicle inside the image, but check back for my next attempt at cracking this algorithm.

 

4 thoughts on “Picking the best photo and Motion detection.

  1. Do you have an estimate or any experience that would guide us on post installation maintenance costs?

    • Allan.

      We are hosting the images and application on the web, and are still fine-tuning the setup. Most hosting providers with “unlimited plans” actually won’t allow you to store images in the volume that we need on their inexpensive plans, so we are currently trying to balance our usage to stay under the limits, but might have to go with a bigger plan.
      We have estimated our yearly maintenance cost to be between $200 and $1600 / year, it is quite a range, but I am confident that we can stay below $1600/year while hoping to get closer to $200/year. We are currently experimenting with Dreamhost, but this might not be the final choice. Full article to be made when we have an answer. Dreamhost is currently supporting our efforts with a $10 off Coupon called “NEIGHBORHOOD” – use it.

      Jesper

  2. Hi – our neighborhood is also trying to set some cameras up for license plate/facial recognition but were thinking of doing it on a more neighborhood-wide basis (a few host houses only). thanks so much for this blog – great info!!! (too much technical for me to follow at the moment, but I will go back to reread). Did you guys talk to Logitech? I’d love to talk via phone a bit (my preferred method of communication).

    • Silvia,

      It is great that your neighborhood is getting organized, Cameras that can capture license plates is a real advantage to a neighborhood, I just got a a call from the police that one of our photos was instrumental in solving a case.

      This page is very technical (and not really relevant to you), check out the other pages, I think they will be more to your liking, remember to subscribe to be informed about new posts.

      We did check out Logitec, the Logitec cameras are good for indoor and short-range outdoor use. The Logitec cameras are not designed for capturing license plates on a street. I will write you offline with my phone number.

      Jesper

Leave a Reply

Your email address will not be published. Required fields are marked *