Picking the best photo and Motion detection.

When the camera records a sequence of images, some of images are going to show the approach and one image are going to the vehicle large and centered. I want to pick the  image where the vehicle is best seen so that 1) I can use it to best represent the sequence on an overview page 2) can feed a few good images to a potential License plate recognition software.

Purple box around picked image.

In the sequence to the left, the images around the middle of the sequence gives us what we want. So why not just pick the middle image.

Picking the middle image in the sequence will produce the right image in 70% of the cases.

Problem 1 with picking a photo at a certain position in the sequence is that these photos might work well for vehicles travelling in one direction, but will be completely wrong in the other direction. (Neighborhood recording one-way streets are in luck)

Problem 2: Changing the Motion detection sensitivity will completely change any setting based on position in the sequence.

Problem 3: middle image in a sequences that contains 2 evenly spaced vehicles will show the empty road between the two vehicles.

The question: is it possible to programatically a) select the right photo every time and b) determine when there is more than 1 vehicle in the sequence and split the sequence into two or more sequences, one for each vehicle.

To do this we have to not only look at motion detection but to determine where the moving object is in the photo and then pick the photo where the object is the largest without being partially out of the frame.

The first internet search took me to this fine page, where I found this algorithm

	h1 = Image.open("image1").histogram()
	h2 = Image.open("image2").histogram()

	rms = math.sqrt(reduce(operator.add,
		map(lambda a,b: (a-b)**2, h1, h2))/len(h1))

The algorithm is quite popular and quoted many times on the internet when there is talk about comparing images.

What is does is comparing image histogram and returning a single number showing how different the images are, with 0 meaning the images are identical.

Hey piece of Cake! I just have set a threshold value and when the difference is biggest the car is biggest in the photo….

Lets see how that worked in real-life:

  1. The difference between the 1st (empty road) and 2nd image is low  (<10) – so far so good
  2. As the car approaches the difference to the 1st photo increases – (50, 65, 85) hey I am onto something
  3. As the car is right in the middle of the picture the difference between the empty road and the photo is 535  – This might actually work
  4. As the car is leaving the picture stage right the difference drops to 462 – still on track
  5. Now we have empty road again and the difference is 437 – hey wait this is wrong. How can two pieces of empty road be as different as if there had been a minivan in one photo?

The RMS Difference between these two images is as much as if there had been a big minivan in one of the images

.If you look carefully you can see a slight shade difference between the two images. the Sun moved …. (it has a tendency to do that).
Loading the images into Photoshop and examining the histograms, I can see that the histogram is almost identical but shifted between the images.

After hours of playing with the above algorithm, I can conclude that the algorithm is really good at detecting changes to a photo such as adding text or manipulating a small block of the photo, but it is useless at detecting differences between two scenery photos, because the scenery is constantly changing.

The next algorithm search took me to examining the difference function in the Python Image library (PIL). Again the the entire picture is different ;-(, but reducing the number of colors, and the resolution caused me to generate this image of a minivan, with all the changes contained within the box.

Box outlining the changes in the photo.

 

 

 

 

 

 

 

From there I made the current algorithm which is pretty good for a first attempt.

#compare current image with Master and make a box around the change
diff_image = ImageOps.posterize(ImageOps.grayscale(ImageChops.difference(master_image, cropped_img)),1)
rect = diff_image.getbbox()
if rect != None:
	ImageDraw.Draw(cropped_img).rectangle(rect, outline="yellow", fill=None)

I highlight the detected area of change on the original thumbnail to easily determine effectiveness.

Yellow box highlights the are of the detected vehicle

Hmm,I detect the roof, the front bumper, then larger and larger parts of the vehicle, there is definitely room for improvements here.

This is as far as I currently am on detecting the vehicle inside the image, but check back for my next attempt at cracking this algorithm.

 

Alarm + Community Security Cameras is the best thing since sliced bread.

During my work on the Neighborhood Security Camera work I had an epiphany that I need to share:

Having an Alarm makes your Neighborhood Security Cameras 10 times more effective.
AND
Having a Neighborhood Security Camera Program makes your Alarm 10 more effective.

Let me explain:

Alarm alone: it will have a slight deterrent effect on Crime, and with a police response time in our area measured in hours, absolutely no effect on apprehension.

Neighborhood Cameras alone: if advertised well it will have a deterrence, but with a regular burglary on a property without alarm the time window of the crime will be too great to be useful to the police. “I left early this morning and when I came back tonight we had been burglarized, so it was one of the 200 cars that passed the Camera between 7:00 am and 6:00 pm. The local police will put this case at the bottom of the pile, with a solveability factor of next to nothing.

Neighborhood Cameras and Alarm:
Alarm was triggered at 1:43 pm, camera has 1 unknown car going into the neighborhood at 1:38 pm and leaving at 2:02 pm. we have the license plate. Police solveability is high, and it will go to the top of the pile, hopefully leading to recovery of personal property.

It really makes perfect sense once you think about it 😉

 

Collecting Trusted License Plates

One of the problems with having a Neighborhood video surveillance system is the number of unknown vehicles that show up on any recording.

For any such system to be valuable, the neighborhood group managing the system has to filter out the trusted vehicles in order to locate the suspicious vehicles.

Inherently trusted vehicles

The Firetruck is inherently trusted

The Firetruck is inherently trusted

Some vehicle comes with a certain level of trust based solely their apperance:

  • Police cars
  • Ambulances
  • Firetrucks
  • Garbage Trucks
  • Cement Trucks
  • Fedex
  • UPS
  • USPS

Not so trusted vehicles.

At the other end of the scale are vehicles that are suspicious just because they don’t fit the standard of the neighborhood: This could be because that they are too expenses, too flashy, to loud, too fast, too slow, too colorful, too dilapidated, too old. etc.

The bland majority

Between these two poles are a large number of vehicles that are just ordinary: these are the cars that the bad guys choose when they want to blend in.

On a typical day we have more than 100 unique such cars passing our camera. The only way to be able to distinguish the good cars from the bad is by having a list of trusted license plates.

If you live in an area without garages or with a shared parking structure, the task can be accomplished by just walking with a camera phone a couple of times.

However if you, like us, live in an area were many people park their cars inside garages you will have to engage the owners. The good news is that this actually increases security awareness and the feeling of community, core features of a good Neighborhood security group

  1. Start by sending out emails asking people for License plates
  2. then send out 1 or 2 more nagging emails to get some more addresses.
  3. Finally make a list of all the addresses where you are still missing the licenses and go door-to-door asking people to open the garages.

Where to record the vehicle information.

Some people will find that putting car model/make and license plate on your neighborhood roster makes them uncomfortable.
Most people find that having a trusted vehicle list that is independent of the Roster and is only accessible by a small group of trusted neighbors is the way to go.
Many people will volunteer color, Make and Model in addition to the License plate, some will only give you the license plate. Some will volunteer extra information like year, accessories (like roof-rack and stickers). A good system will have a way to capture this information.

For most group the list is so simple that it can be kept in a single spreadsheet, however if you have more than one person viewing and updating a file based sheet, you will have a hard time making sure everyone has an updated copy

We found that using a hosted shared spreadsheet was the best solution, and are currently using the free Google Docs spreadsheet.

 

Traffic Count and Statistics.

Cute little garbage truck captured in the fog, one of 8 passings of garbage trucks that day.

Our Neighborhood Security Surveillance Camera records a lot of traffic but how many passers-by do we really record:

I decided to pick 1 day and do a detailed count, this happened to coincide with a theft from a car that night which allowed me to combine the work.

in 24 hours, the motion detector recorded 491 sequences of events on 1 camera covering an entrance to our neighborhood.
The count below records objects passing the camera, not unique cars passing, e.g. the same car coming and going will be counted as 2.
1 Bicycle
2 Motorcycles
3 Utility trucks
7 Delivery vans
8 Contractors
8 Garbage Trucks (garbage day, but I was surprised it was that many)
14 Pedestrians (with 6 dogs)
238 Passenger Cars
and
1 Suspicious Person (Photo and license plate submitted to Oakland Police)

Total of 281 relevant events.

The rest was false alarms on the motion sensor, e.g.  moving shadows from clouds, fog, changing light. (it was a foggy day)  e.g.

License plate readability
Of the 266 vehicles, 7 did not have a License plate. (2.6%)
Out of 259 vehicles with License plates 255 could be read by human(*) (98.5%)

(8) The human is me, method included selecting the best photo for each vehicle and using different zoom levels, sometimes part of the license plate could be read in one photo and the rest of the plate in another photo. No image manipulation was needed (such as Photoshop)

Room for Improvement
A motion detector False Positive rate of 42% – is too much.

While I would rather have too many recordings than missing 1 critical, here is room for improvements.
I am currently adjusting the sensitivity levels in order to try reduce the number of false recordings.

 

Result of Neighborhood Security Camera Pilot

We have just finished the pilot installation of our Neighborhood Security Camera solution.

Pilot Requirements
Pilot had to uncover if it would be possible to make a solution that could satisfy the following requirements:

  1. Be Legal
  2. Be affordable (both Startup and Maintenance)
  3. Installed by the neighborhood members themselves not professionals
  4. Store the recording in the ‘Cloud’ to allow distributed access
  5. Evidence quality recordings that would allow manual reading of license plates Day and Night.
  6. Determine which software should be used for review of the recordings.

The Process
We have evaluated a number of cameras, low cost, medium cost and high cost. The cameras were tested in live setup in our neighborhoods under a number of conditions both day and night, rain and shine  and the famous Oakland hills fog.

The Surveillance of public space presented a number of legal and practical challenges that we had to address.

The distributed nature of a residential neighborhood compared to most home and commercial installations meant that we had to solve a number of logistical challenges that had not  been solved before on a limited budget.

Conclusion

 

  • Cheap cameras don’t work at all.
  • Medium priced cameras  work during the day but not at night.
  • Only Expensive cameras work both at day and night,
  • Infrared Illuminators are required for night recordings.
  • Legal issues can be solved.
  • It is possible to coordinate recordings from multiple residences on a budget.
  • We could not find any suitable Commercial or Open Source software.
  • Cloud Storage provider is found, but not tested (due to lack of software)

Readable License plate in Thick fog. Click to enlarge. Test car - not a Criminal!

Night+Rain+Fog image of readable License plate. Click to enlarge. Actual photo

Real car driving by Camera at Night with Rain and Fog. Click to enlarge. Test car - not a criminal!