Reporting Tasks are used to monitor things such as disk utilization and memory utilization of the NiFi instance. A reporting task runs in the background and provides various statistics of the NiFi instance.
Reporting Tasks run in the background to provide statistical reports about what is happening in the NiFi instance. The DataFlow Manager (DFM) adds and configures Reporting Tasks in the User Interface as desired.
Common reporting tasks include:
- ControllerStatusReportingTask - Logs the 5-minute stats that are shown in the NiFi Summary Page for Processors and Connections, as well optionally logging the deltas.
- MonitorDiskUsage - Checks the amount of storage space available for the specified directory and warns (via a log message and a System-Level Bulletin) if the partition on which it lives exceeds some configurable threshold of storage space.
- MonitorMemory - Checks the amount of Java Heap available in the JVM for a particular JVM Memory Pool. If the amount of space used exceeds some configurable threshold, will warn (via a log message and System-Level Bulletin) that the memory pool is exceeding this threshold.
- StandardGangliaReporter - Logs the 5-minute stats that are shown in the NiFi Summary Page for Processors and Connections, as well optionally logging the deltas between the previous iteration and the current iteration
Adding a Reporting Task
Adding a new reporting task is a very simple process within the Fuusion Architect. From the Global Menu, located at top right corner of the Fuusion Architect, once you select it, the pull-down menu is displayed. Select Controller Settings which will display the NiFi Setting windows.
From the Global Menu in the top right corner, select Controller Settings.
From within Controller Settings, select the Reporting Tasks tab. Click the plus icon to add a Reporting Task.
Here we can see numerous available reporting tasks, along with their tags hinting at their purpose.
NOTE: When you click on a specific Type listed, it will display details about that Reporting Task including it version (1.8.0) along with other details to assist you in determining which Reporting Tasks best fits you specific needs.
As an example, we'll select two of these tasks, MonitorDiskUsage and MonitorMemory. Each of these reporting tasks are self-explanatory - they will monitor disk utilization and memory utilization and provide alerts when they reach a set threshold. Select each in turn and click Add.
Click the Edit (pencil) icon on MonitorMemory
Under the Properties tab, set the memory pool. As an example, we'll use G1 Old Gen.
Set the Usage threshold - here it is set to 1%, but this is only as a sample scenario, to generate a near immediate monitoring alert. Typically, you would set this to a much higher threshold.
Once you have completed the edits you want, you need to select “Apply” to commit the change.
Next, we will click the Edit icon on the MonitorDiskUsage reporting task.
Again, as an example, set the usage threshold to 1%.
Next, set the directory location to somewhere easily found.
NOTE: The above example is using the “windows” form (i.e. \User\NiFiTest ) for the “Directory Location”. If you are using a Linux system, the form would be (i.e. /User/NiFiTest).
After applying the changes on both, click the Play (Start) icon on each of the tasks.
Once back to the Fuusion Canvas, right-click and select Refresh, and you should immediately see bulletin alerts because the threshold was set so low for this lesson.
Note that you can edit the reporting tasks at any moment by returning to Controller Settings in the Global Menu, selecting the Reporting Tasks tab, stopping the tasks, and clicking the edit icon.
Here you can not only change the thresholds or the location of the log files for the tasks, but also set them to run on a schedule. This is found under the Settings tab. Again, we see the default of 5 minutes observed for component reporting.
Note as well that the Usage threshold provided does not have to be a percentage, but can be a defined storage value, such as 8 GB.
Site to Site Reporting Tasks
There are also reporting tasks that can be used to monitor multiple clustered instances of Fuusion. These are configured in nearly the exact same way as the previous reporting tasks, save that connection properties need to be applied. Here you see a sample flow with an input port and exit port configured.
Just as before, on one of the instances, you will use the Global Menu in the top right to select Controller Settings and then select the Reporting Task tab. Click the plus sign ( + ) and select a SitetoSite reporting task.
Once the reporting task is selected, you will need to define the connection properties. For site to site the key properties are:
- the Destination URL
- the Input Port Name
Note that the Input port must be defined at the root level on the secondary instance. Once set up, it can simply be referenced by name in the primary. For information on how to configure an input port, see the Connecting Site-To-Site NiFi Documentation.