Managing data completeness tests

The Data Completeness test and analysis section are data analysis features. Regular users and administrators can run the test to find and identify late or missing data across the entire organization. You can use the data completeness analysis section to track and monitor data entry tasks, by activity over time to ensure that all the required data for the reporting period has been obtained and input into the system. For further information, go to Viewing and managing data completeness tests.

For the data completeness function to run correctly, you must apply some settings to each activity such as who is responsible for the data entry, how many days late would be considered overdue, or when to escalate to management. You can configure these settings when you are creating the activity or meter, or afterwards.

Data Completeness scanning is available for Automatic Meter Reading (AMR) or interval data activities and non-AMR or non-interval data activities.

Interval data

Interval data activities are meter only activities that receive interval data, data every 15 to 30 mins. This test scans for missing intervals within the meter upload data sheets. You can only configure interval scanning on half hourly or interval data meters for electricity, water, or gas. For further information, go to Running a data completeness test on interval data.

Non-interval data

For all other activities, you can configure non-interval tests. Non-interval data activities include all other types of data, such as invoices or supplier statements, and is the main scan used. For further information, go to Running a data completeness test on non-interval data.

Running a data completeness test on interval data

For all other activities select non-interval data and go to to Running a data completeness test on non-interval data.

Note

You can also run an interval data completeness scan on multiple activities at once. For further information, go to Running data completeness tests on interval data for multiple activities.

In Diligent ESG, select Data Management and select GHG Activities.
Locate the activity that you want to test in the table and select Select in the activity row.
In the test details page, select Config to manage the test settings.
Select the Data completeness test tab and expand Interval data – Data completeness test.
Select the Scan interval data for incomplete data entries checkbox.
Select Run test. The test runs immediately.

The system now scans the interval upload data files, and you can view the results in the Analysis>Data Completeness page. For further information about this page, go to Viewing and managing data completeness tests.

To configure emails to alert users to the data completeness of the data, go to Assigning alerts and reminders to activity owners and managers.

Running a data completeness test on non-interval data

Non-interval data activities include all other types of data, such as invoices or supplier statements, not included in an interval test and is the main scan used. You can schedule a non-interval data completeness test when you are creating the activity or at a later date. You can also update and change these settings at any time.

Note

You can also run a non-interval data completeness scan on multiple activities at once. For further information, go to Running data completeness tests on non-interval data for multiple activities.

In Diligent ESG, select Data Management and select GHG Activities.
Locate the activity that you want to test in the table and select Select in the activity row.
In the test details page, select Config to manage the test settings.
Select the Data completeness test tab, expand Non-interval data – Data completeness test, and select the Scan non-interval data for incomplete data entries checkbox.
In the Frequency of data dropdown, select how often you expect at least one item of data to be entered for the selected period. For example, you expect at least one record to be added to an activity every three months.
In the Number of days after the scan when alerts appear field, enter the number of days after a scan occurs before a user is alerted to incomplete data.
In the At least one data point in each dropdown, select how you expect at least one item of data to be entered for the selected period.
From the At least one data point in each day options, select Yes if you expect at least one data point for every day in the period selected in Step 7. If you select No, then the test only checks for one data record in the selected period.
In the First scan date field, enter or select the date for the first scan.
From the Scan historic data options, select Yes if you would like to scan data from a specific data, and in the First historical scan date field, enter or select the date.
Select Run test. The test runs immediately.

The system now scans the non-interval upload data files, and you can view the results in the Analysis>Data Completeness page. For further information about this page, go to Viewing and managing data completeness tests.

To configure emails to alert users to the data completeness of the data, go to Assigning alerts and reminders to activity owners and managers.

Viewing and managing data completeness tests

You can use the Data Completeness analysis section to track and monitor data entry tasks, by activity, over time. The Data Completeness screen has two sections; the chart and the data table sections. You can filter results in both the chart and table using the Organisation or Geographic navigation tree, Data Filters, or table column filters. You can view the status of the tests, which can be one of the following:

Up-to-date The activity data is up to date for the current update period.
Pending This is the default status when Activity completeness settings are not configured.
Waiting The activity is waiting for activity data to be entered for the current update period.
Missing Data Some activity data is missing for prior update periods.
Overdue The activity data is overdue for the current update period.
Not configured Activity setup is not complete.

In Diligent ESG, select Analysis and select Data Completeness.
In the Organisation or Geographic navigation tree, select a company, business, or location to filter the tests shown in the table.
In the Data Filters section, expand Scope, Source Category, Activity Type, and Data Streams, and select or clear the checkboxes to filter the tests shown in the table.
In the data table section, in the fields below the column headings, enter a term or figure to filter the test results shown, such as Missing Data in the Status column.
(Optional) At the top of the data table section, select the Export to Excel icon or Export to CSV icon to export the table.
Select the Select Columns dropdown, select the checkboxes beside other columns that you want to add, and select Show/Hide.
In a test entry row, select Details to view a test and its results in more detail.
In the test details page, select Config to manage the test settings or alerts.

Assigning alerts and reminders to activity owners and managers

If you are an Administrator or SiteAdministrator, you can configure data completeness and data anomaly scans with email alerts and escalations that are sent to Activity Owners and Activity Managers. You can configure the email reminders and alerts to be sent after a certain period of time has passed.

An Activity Owner is a user who is responsible for uploading data to a particular activity. You can assign multiple Activity Owners to one activity, and you can assign a user as an Activity Owner to multiple activities in the system. However, you can only assign users as an Activity Owner for an activity if they have been provided with access to the site where the activity is located.

If configured, these users will receive email reminders for late or missing data entries within the activities they have been assigned to. These email reminders appear after the specified time period configured in the data completeness test.

An Activity Manager is assigned to manage the data at particular activities. You can assign multiple Activity Managers to one activity, and you can assign a user as an Activity Manager to multiple activities in the system. However, you can only assign users as an Activity Manager for an activity if they have been provided with access to the site where the activity is located.

If configured, Activity Managers can receive escalation email alerts when users do not input data within a specified time period. These are usually Divisional Managers or Administrators of the account.

In Diligent ESG, select Data Management and select GHG Activities.
Locate the activity in the table that you want to assign owners and managers to and select Select in the activity row.
In the test details page, select Config to manage the test reminders and alerts.
Select the Email Reminders & Alerts tab and select the Enable e-mail alerts for this activity checkbox.
In the Reminder/escalation emails field, enter the number of days after an original alert to send an escalation email alert.
Select the Data Completeness (Missing, and late data) checkbox to enable email alerts for the data completeness test.
Select the Activity Owners tab, select one or more names from the left field, and select the Arrow icon > to assign that person as an activity owner. Assigned names move to the right field.
Select the Activity Managers tab, select one or more names from the left field, and select the Arrow icon > to assign that person as an activity manager. Assigned names move to the right field.
Select the Anomalies (Duplicates, Overlapping and Inconsistent Data) checkbox to enable email alerts for the data anomaly test.
Repeat Steps 7 and 8 for the anomaly owners and managers.
Select Run test.