f2 Image Recognition Suite

Partial Face Recognition

Until now, face recognition systems have treated a face as a homogeneous entity. f2’s Partial Face Recognition technology is unique in that it takes just part of a face and matches it against the same parts in a database of complete faces. This has real value in forensic and investigative environments where only part of a face may be available. For example, surveillance images, body parts after an explosion, burn victims, a photograph where only part of a face is visible, etc.

Partial face recognition is a module within the Face Forensics face recognition application. All the functionality of full-face recognition is available to recognize and match the partial face. Selecting “Partial face” on a pull-down on the Face Screen displays a facial

For more information CLICK on the button below to download the Partial Face Product Sheet.

Dead Body Identification

Dead body identification is made easier with Corpse-ID, including instances where you find someone who is unconscious, i.e. where the eyes are closed. It is a separate but fully integrated capability within the f2 Image Recognition Suite. It can work even if the face is significantly damaged. Closed eyes are an issue because all face recognition systems place much value on the area around the eyes due to the clear edges and high contrast of the features there. If both eyeballs are not visible the recognition performance of almost all face recognition systems can be severely degraded or not work at all, as the eyeballs are the key anchor points around which the characteristics of numerous facial features are measured.

The original design of f2 included the unique ability to match a selected part of a face, and to apply different weightings to different areas, making it possible to match only the selected part, and thus work with closed eyes and damaged faces. In partial mode Corpse-ID can work with two eyes (i.e. mailbox or balaclava view), one eye, and sometimes with no eyes at all.

In the above example the closed-eye face to be searched is copied into the box on the right, the eye centers are set manually, and the top matches are displayed as thumbnail images on the left.

As with all biometrics, face recognition systems require a database of images to match against. In the case of corpses this database will generally be specially created following a particular disaster such as a boat sinking, tsunami, earthquake, etc, with pictures provided by concerned family members and friends. Multiple photos of an individual can be held in the database. (Watch the video on the left to see a demo of Corpse ID.)

When a victim is found an investigator will take several photos of their face. If the eyes are both visible and open the face can be matched immediately against the database. If the eyes are closed or the face is damaged it will be positioned over a generic facial template and the eye locations will be set manually.

Each of the shots will be encoded and matched against the encodings of all the faces in the database. Thumbnail images of matches above a user-defined threshold will be displayed for investigators to compare. Text filters, e.g. on gender, can be used to narrow down the number of potential matches where appropriate, for example where a face is heavily damaged and a number of distinguishing features are not visible.

CLICK on the button below to download the Corpse ID Product Information Sheet.

Tattoo Recognition

The f2 Tattoo Recognition Module is designed to match an unknown tattoo against a database of tattoos in order to identify its owner. It will also match a partial shot of a tattoo, e.g. where it’s partially concealed by clothing, against the full tattoos in the database. Results are displayed as thumbnails of potentially matching tattoos, in descending order of Match%. Any of these can be selected to display the face and name of the owner. Tattoo recognition is similar to biometrics like fingerprints, face, and iris recognition in the way the image is encoded and then matched.

In the example above, the partial probe image is on the right and top matches are displayed on the left. f2 finds the single correct match in the 5700-tattoo database. The non-match below it is visually different to the probe but rates highly because it includes leaves of similar shape.

However it differs in that a tattoo can be of anything, and be of any shape and size in the image. f2 determines the unique characteristics of each, e.g. lines, curves, colors, etc, and their relative positioning. This means that the matchlist can contain tattoos which appear totally different to the probe image. This happens because the characteristics being searched for by f2 will often be present in other images that visually look quite different to the probe. However the correct match (assuming it’s in the database) should almost always be at or near the top of the list. f2 Tattoo is designed to work with images captured under controlled conditions and that meet NIST Tattoo Capture Guidelines.

Features:

• Matches complete tattoos and also those partially obscured
• Can also match scars and marks subject to sufficient detail
• Accesses images in existing external databases in read-only mode. (User can opt to store images.)
• Links a matched image back to its associated face and details in the original database
• Text (if included in the original record) can be used to help filter out non-relevant matches
• Automatic enrollment of all images in a database. No limit to database size
• Import and automatic enrollment of images in a folder
• Automatic detection and enrollment of new images added to an external database
• Probe and database images can match at up to 45° rotational difference
• Searches any database of enrolled tattoos on a VPN under appropriate controls
• Side-by-side comparison of “probe” image and potential match for visual inspection, including magnification
• Easy to install/use f2 Tattoo is an independent but fully integrated module within the f2 Suite.

CLICK on the button below to download the Tattoo Recognition Product Sheet.

Scene/Object Recognition

Scene/Object Recognition (SOR) is a module within f2’s Image Recognition Suite. It’s designed to match virtually any solid object with distinguishing features against a database containing such objects and return the closest matches. It works similar to f2’s face and tattoo matching, i.e. a probe image is copied in and encoded using f2’s algorithms to produce a unique digital array which is then compared with all available images in the database. A key feature of SOR is its ability to identify objects which are a part of a larger object, often due to something else in front of it partly concealing it.

f2 has included a scene recognition module for some time now. However this was designed to match relatively large images, for example – rooms. Yet, there remained a clear need for object matching and a prototype module was developed for this. The experience gained emphasized that with minor modifications, such as image enhancement, the two modules could be combined into one.

SOR can process images from any source, including photos stored on PCs, and in the Cloud. It can process very large volumes of images even if they’re significantly obscured).

There’s no practical limit to the size of database that SOR can work with, subject to an appropriate hardware configuration.

1. Ability to match objects in static images, video files, Cloud, and website images
2. Ability to identify partially concealed objects
3. Being able to link scenes with the same object in them
4. Being able to optionally filter searches by dominant color, and text description if present

SOR does not actually match an image against other images; it matches a numeric string called an encode array describing the characteristics of an image against a database of encode arrays describing images. This increases speed, accuracy and minimizes the space required.

Within SOR the primary entity is the crime scene, i.e. a still shot or video frame. SOR analyzes the image to find multiple areas with strong contrasting features and transforms the characteristics of these mathematically into an encode array that uniquely identifies the image. The encode arrays for all the images in the database are in a master database in server memory.

(SOR comes with small sample databases of images to match against, so it can be tested immediately after installation).

Multiple Record Detection

f2 is a highly-advanced face detection and recognition system – the result of many years experience in the forefront of face recognition development. It will:

• Search a database to identify an unknown face
• Verify that an individual is who they claim to be
• Detect and recognize faces in a video stream
• Identify part of a face in a forensics or investigations environment
• Check an entire database, or multiple databases, for multiple records of the same person using different names, i.e. N:N matching

A major problem with almost any database containing records of people is that the same individual can have made multiple applications (e.g. for driver’s licenses, entitlement cards, etc), or have been arrested for a number of different crimes, using a different name each time. Face Forensics is specifically designed to enable instances of this to be easily identified. This applies to multiple records in the same database, and across different databases. The process involves first automatically encoding all the facial images in one of the databases. Then each image in the other database is encoded and matched against each in the first. While this can take some time (depending upon the size of the databases) the process is fully automatic and can be left to run until the task is complete. All possible matches, i.e. above a user-defined threshold, are sent to a separate folder for subsequent review.

Face Forensics is available as an application or .NET SDK. It runs under 64-bit Windows 10. It will access SQL Server databases.  An evaluation version is available.

For more information contact: Contact@SketchCop.com

CLICK on the button below to download the Multiple Person Search Product Sheet.

f2 FAQ

Do you have questions about f2? Read below to learn more about the features, benefits and system requirements.

Q: What kind of hardware and operating system do I need?
A: Face Forensics (f2) runs on 64-bit Windows 10  If you do not have the .NET Framework 4.7.2 or higher installed, the f2 installer will inform you that you must install it first and tell you to go to Download the .NET Framework 4.7.2 and install it.

Q: Do I need to have a database of images before I can use f2?
A: No, f2 comes with a default Enrollment database that you can use to add your images. An Enrollment DB is also sometimes referred to as a Watch List or Missing Persons List.

Q: Can f2 work with my current database that contains images?
A: Yes, as long as your database is an SQL Server database. f2 can attach to your database and enroll the images to create an Enrollment DB.

Q: Can f2 be networked?
A: Yes, multiple f2 machines can connect to the same Enrollment DB. More generally, each f2 runs its own matching service by default, so you can select one (or more) of them to act a central matching service for all f2 machines. There is also a Web Service installer that allows you to set up a face matching service that can be accessed from anywhere on the internet.

Q: What type of camera do I need?
A: f2 captures video using the Windows DirectX v9.0 drivers or higher. Therefore any USB digital camera that is DirectX compatible should be able to work with f2. f2 does not use color information for faces, so it is not necessary to have a color camera. Near infra-red cameras can also be used.

Q: How does f2 Search for missing children?
A: f2 can analyze images on suspicious web sites and associated links while automatically matching against an Enrollment DB of missing children photos. Several f2 machines can be set up to continually search the internet while automatically sending the match results to a shared folder that can be viewed by investigations officers.

Q: How does f2 notify roaming policing/security staff with an alert?
A: f2 can continually monitor an area while detecting and matching faces. Detection and/or recognition results can be automatically e-mailed to recipients carrying hand held devices. Detected and matched images are sent as attachments in the e-mail.

Q: What types of video files can f2 play?
A: f2 can play most AVI, MPEG, MP4, and WMV video files.

Q: Do I need any special hardware to use partial face recognition?
A: You need a mouse with a scroll wheel to adjust the size of the partial image when positioning it on the template.