NiFi Ingest 24.4
NiFi Ingest

# FaceDetection

Runs face detection on the file(s) associated with an IDOL document FlowFile, and adds information about any detected faces to the IDOL document.

After detecting a face, the processor can run face recognition and clothing analysis. Clothing analysis returns a region containing the clothing on a person's upper body, and a region containing the clothing on a person's lower body. It also analyzes the color of the clothing in each of these regions.

Supported file formats

The processor can handle video files.

The processor can handle the following image formats:

- TIFF
- JPEG
- JPEG 2000
- PNG
- GIF
- BMP (compressed BMP files are not supported) and ICO
- PBM, PGM, and PPM
- WebP

Additionally, if you configure your MediaServiceImpl controller service to use a KeyView Export Service, the processor can handle document formats, including:

- Adobe PDF
- Microsoft Word Document (.DOC and .DOCX)
- Microsoft Excel Sheet (.XLS and .XLSX)
- Microsoft PowerPoint Presentation (.PPT and .PPTX)
- OpenDocument Text (.ODT)
- OpenDocument Spreadsheet (.ODS)
- OpenDocument Presentation (.ODP)
- Rich Text (RTF)

## Properties

Name | Default Value | Description
--- | --- | ---
IDOL License Service |  | An IdolLicenseServiceImpl that provides a way to communicate with an IDOL License Server.
Media Service |  | A MediaServiceImpl that manages media analysis resources.
Video Sample Interval | 100 | The interval between video frames that are selected to be analyzed, in milliseconds.
Any Orientation | false | A Boolean value that specifies whether to detect faces in any orientation, rather than just upright.
Clothing | false | A Boolean value that specifies whether to run clothing analysis.
Portrait Faces Only | false | A Boolean value that specifies whether to detect faces looking at the camera (true), or looking in any direction (false).
Minimum Face Size | 48 | The minimum size of a face. NiFi Ingest does not attempt to detect faces that are smaller than the specified size. Increasing the minimum size can increase processing speed.
Minimum Face Size Unit | pixels | The units for "Minimum Face Size". You can use pixels or a percentage of the smallest image dimension (width or height).
Recognition Results | 0 | The maximum number of recognition results to return per face.
Recognition Threshold | 50 | The minimum confidence score required to recognize a face.
Database File |  | The path of a file that contains a face recognition database. Set this property to use a database that you exported from Media Server, using the action `ExportFaceDatabase`.
Shared Database |  | The name of the database to use for face recognition. Set this property to use a database that is stored in the external database specified by the Media Service (see the "Media Service" property).
Aligned Face Image Height |  | When set to a non-zero value, for example "300", the processor saves a JPEG image of the detected face. The image is rotated so that the face is upright and is cropped around the person's head.
Image Output Format |  | When set, the processor saves the full image or best video frame for each detected face, in the specified format (BMP, GIF, JPEG, PDF, PNG, PPM, TIFF or WEBP).
Video Clip Quality |  | When set to "low", "medium", or "high", the processor saves a video clip of the detected face at the specified quality level.
Video Clip Height | 300 | The height of the video clip that is generated when you set Video Clip Quality, in pixels.

## Relationships

Name | Description
--- | ---
success | Processing was successful.
failure | Processing failed.

## Example Output

The following examples show the metadata that is added to an IDOL document by face detection.

Image or document input

    <idol_media>
      <faces>
        <face page="1">
          <identity provider="Micro Focus">
            <database>people</database>
            <identifier>person1</identifier>
            <confidence>93.72</confidence>
          </identity>
          ...
          <bestRegion>
            <region height="225" left="44" page="1" top="65" width="225">
              <angle>-1.19</angle>
              <verticalAngle>12.3</verticalAngle>
              <viewAngle>5.73</viewAngle>
              <eye radius="13.11" x="191.79" y="149.04"/>
              <eye radius="13.42" x="117.64" y="150.59"/>
            </region>
            <alignedFile>/opt/nifi/idol_repository/Files/FaceDetection_03916629-0180-1000-5cab-27ba2fa5b4d0/8d9e7847-18f4-49b3-af7f-332eb26d452d.jpeg</alignedFile>
            <imagefile>/opt/nifi/idol_repository/Files/FaceDetection_03916629-0180-1000-5cab-27ba2fa5b4d0/f304ace9-841f-44f9-b58d-1637e378100a.bmp</imagefile>
          </bestRegion>
          <clothing>
            <lower>
              ...
            </lower>
            <upper>
              <color b="224" g="224" proportion="50.6" r="224">white</color>
              <color b="64" g="64" proportion="46.22" r="64">black</color>
              ...
              <region height="282" left="0" page="1" top="212" width="330"/>
            </upper>
          </clothing>
          <region height="225" left="44" page="1" top="65" width="225">
            <angle>-1.19</angle>
            <verticalAngle>12.3</verticalAngle>
            <viewAngle>5.73</viewAngle>
            <eye radius="13.11" x="191.79" y="149.04"/>
            <eye radius="13.42" x="117.64" y="150.59"/>
          </region>
        </face>
      </faces>
    </idol_media>

Video input

    <idol_media>
      <faces>
        <face duration="1.36" start="1.44">
          <identity provider="Micro Focus">
            <database>people</database>
            <identifier>person2</identifier>
            <confidence>80.63</confidence>
          </identity>
          <bestRegion>
            <region height="130" left="452" timestamp="2.64" top="123" width="130">
              <angle>4.17</angle>
              <verticalAngle>-0.53</verticalAngle>
              <viewAngle>-18.54</viewAngle>
              <eye radius="9.04" x="539.52" y="170.35"/>
              <eye radius="10.09" x="496.09" y="167.19"/>
            </region>
            <alignedFile>/opt/nifi/idol_repository/Files/FaceDetection_03916629-0180-1000-5cab-27ba2fa5b4d0/3e9220a9-9142-4f2b-b7a1-a38ce76fdcb9.jpeg</alignedFile>
            <imagefile>/opt/nifi/idol_repository/Files/FaceDetection_03916629-0180-1000-5cab-27ba2fa5b4d0/bdc37ced-2cb9-4b06-b9c2-df58cf81f4ef.bmp</imagefile>
          </bestRegion>
          <clothing>
            <lower>
              ...
            </lower>
            <upper>
              <color b="224" g="224" proportion="61.53" r="224">white</color>
              <color b="128" g="128" proportion="6.37" r="128">gray</color>
              ...
              <region duration="0.04" height="429" left="261" start="1.44" top="147" width="516"/>
              <region duration="0.04" height="423" left="255" start="1.48" top="153" width="537"/>
              ...
            </upper>
          </clothing>
          <region duration="0.04" height="124" left="459" start="1.44" top="35" width="124">
            <angle>0.31</angle>
            <verticalAngle>-5.96</verticalAngle>
            <viewAngle>-20.71</viewAngle>
            <eye radius="8.06" x="541.85" y="78.87"/>
            <eye radius="9.08" x="499.83" y="78.64"/>
          </region>
          <region duration="0.04" height="128" left="459" start="1.48" top="40" width="128">
            <angle>0.31</angle>
            <verticalAngle>-5.96</verticalAngle>
            <viewAngle>-20.71</viewAngle>
            <eye radius="8.31" x="543.99" y="85.37"/>
            <eye radius="9.35" x="500.68" y="85.14"/>
          </region>
          ...
          <videoFile>...</videoFile>
        </face>
      </faces>
    </idol_media>

The XML contains a `face` element for each detected face.

- The `region` element describes the region in which the face was detected.

  The `left`, `top`, `width`, and `height` attributes provide the position and size of the region in pixels (`left` specifies the distance from the left side of the image to the left side of the region, and `top` specifies the distance from the top of the image to the top of the region).

  When you analyze an image or document the `page` attribute specifies the page on which the face was detected. When you analyze a video and a face is tracked across multiple frames, there can be multiple `region` elements with `start` and `duration` attributes that provide video timestamps, in seconds.

  The `region` element includes the following further elements:

  - `angle` - describes the orientation of the face in the image or video, for example if the face is upright the angle is `0` and if the face is upside-down the angle is `180`.
  - `verticalAngle` - indicates how far the person is looking up or down.
  - `viewAngle` - indicates how far the person is looking to the left or right.
  - `eye` - describes the position and radius of the person's eyes.
- The `identity` element describes the identity and confidence score for a recognized face. There can be multiple identities for each detected face, depending on the value you choose for the Recognition Results property.

- The `clothing` element is present if you run clothing analysis. This element can contain information about clothing worn on the upper body (the `upper` element) and the lower body (the `lower` element). The `upper` and `lower` elements contain further elements:

  - `color` - describes the color of the clothing (as a name, and as RGB values), and the `proportion` of the clothing that is the specified color (as a percentage).
  - `region` - the location of the clothing in the image or video frame. Results from a video can include multiple regions, each with a `start` time and `duration` (in seconds) for which the region is valid.

- The `bestRegion` element represents the best video frame in which the face was visible. It includes the `region` element for that frame, but can also include:

  - `alignedFile` - The path of the image that was generated when you set the property Aligned Face Image Height.
  - `imageFile` - The path of the image that was generated when you set the property Image Output Format.

- The `videoFile` element contains the path of the video file that is generated when you set the property Video Clip Quality.

Copyright 2018-2024 Open Text
Version 24.4 | Last updated September 2024

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL NiFi Ingest 24.4

Topic Title: FaceDetection