The first step in this example ingestion pipeline is KeyView Extraction. Extraction extracts individual files from containers (such as zip files or other types of archives) so that they can be processed by other processors.
To add KeyView Extraction
Add a processor, by dragging the processor icon from the components toolbar to the canvas.
The Add Processor dialog box opens.
In the Source list, click idol.nifi.
The list of processors is filtered to those that are provided in the NiFi Ingest distribution.
Select the KeyViewExtractFiles processor and click ADD.
The processor is added to the canvas.
Create a connection between the input port and the KeyViewExtractFiles processor. Hover the mouse over the input port until you see the connection icon - - and then drag the icon to the KeyViewExtractFiles processor.
The Create Connection dialog box opens.
Click Add.
The connection appears on the canvas. NiFi automatically adds a queue between the input port and the KeyViewExtractFiles processor.
In its default configuration, the KeyViewExtractFiles processor creates new documents for extracted subfiles but does not extract subfiles recursively. This means that if a zip file contains another zip file, the second is not automatically extracted. The new (extracted) documents are routed to a relationship named subfile, so to extract subfiles recursively route this relationship back to the processor as described in the next step.
Create a connection from the KeyViewExtractFiles processor to itself. Hover the mouse over the processor until you see the connection icon - - and then drag the icon away from and back to the processor.
The Create Connection dialog box opens.
In the For Relationships area, select the subfile check box and click ADD.
The connection appears on the canvas. By default, the name above the queue (in this case "subfile") matches the relationship(s) that you selected.
Right-click the KeyViewExtractFiles processor and click Configure.
The Configure Processor dialog box opens.
In the properties list, find the KeyView Filter Service property. Edit the value of the property by clicking the value field. Choose the name of the service that you created in Add a KeyView Service, and then click OK.
To find out more about this processor and the properties that you can set, see the documentation for the KeyViewExtractFiles processor.
|