Assessment scoring for third-party assets
Criticality levels and risk assessments are an integral part of the TPRM framework. These scores can help you categorize and quantify whether a third-party is fit to partner with. This topic explains how criticality levels and risk scores are calculated from assessment responses.
Note
Because response weightings and impact level boundaries are configurable, the examples in this section may not match the calculations you see in your organization. For more information, see Viewing your organization's calculations in Robots.
Criticality levels
Most organizations have their own metrics to assess criticality levels of third-parties. This can depend on various factors, such as business impact, regulatory requirements, security compliance, and financial responsibility. Identifying criticality level serves for an initial categorization of third-parties.
If you use the categorization assessment to determine criticality levels, Third Party Risk Management applies a rating calculation to determine criticality levels for third-party assets, based on corresponding assessment responses.
What is a categorization assessment?
A categorization assessment comprises several multiple choice questions that a Business Owner or Third-party Owner answers. Third Party Risk Management uses it to calculate the overall criticality level for each asset.
Calculating criticality levels
After a respondent submits their responses, the Third-party Categorization Workflow robot uses the individual scores to calculate a percentage value, which it then compares with ranges specified for each criticality level. It then assigns the third-party a criticality level that corresponds with the range the value falls in.
1. Calculating categorization scores
Each response in the criticality assessment questionnaire has a weighting assigned to it. System Admins with Professional User subscriptions can customize each response weighting in the Third-party Categorization Workflow robot.
This robot calculates a score for the questionnaire based on the highest possible score for each question:
- For questions that only allow a single response, the highest possible score is the highest weighting of all possible responses to the question.
- For questions that allow multiple responses, the highest possible score is the sum of all weightings of all possible responses to the question.
Show me an example of a question's effect on a categorization score
Example
The respondent is answering this question: What type of data will be stored, processed, or transmitted by this third party? The question accepts multiple responses.
The available responses and their respective weightings are as follows:
| Response | Weighting |
|---|---|
| Employee information (PII) | 3 |
| Customer information (PII) | 3 |
| Financial information | 1 |
| Proprietary information | 1 |
| IT Infrastructure information - Confidential | 3 |
| IT Infrastructure information - Encrypted | 1 |
| None of these data types apply | 0 |
The respondent selects Financial information and IT Infrastructure information - Confidential.
- The total score for this question is 4 ( 1 + 3 = 4 ).
- The highest possible response weighting for this question is 12 ( 3 + 3 + 1 + 1 + 3 + 1 + 0 = 12 ).
2. Assigning an impact level
Third Party Risk Management calculates the final questionnaire score as follows: categorization score = ( sum of all response weights / sum of all highest possible question scores ) * 100%.
Lastly, the Workflow robot takes that percentage score and compares it with the impact level boundaries defined for your organization, which you can customize in the robot. The impact level corresponding with the range the percentage score falls into appears in your asset.
Show me an example of assigning an impact level
Example
After completing the questionnaire:
- The sum of the response weightings the respondent selected is 41.
- The sum of the highest possible scores for all questions is 110.
The final questionnaire score is 37% ( ( 41 / 110 ) * 100 = 37.27 ).
In your organization, scores between 20% and 40% are assigned the Low criticality level. Therefore, Low appears in the Criticality Level field for the third-party asset.
Risk scores
What is a risk assessment?
A risk assessment comprises several multiple choice questions that a Third-party Owner answers. You can choose between the SIG Lite and CAIQ Lite questionnaires to categorize your assets. Typically, a Business Owner can assign it to the Third-party Owner or any other external user who can answer all the questions.
Determining risk score and level
As with categorization questionnaires, each response in the risk assessment is assigned a score. Once a respondent submits the responses, Diligent One uses the individual scores to calculate a percentage value. This percentage value is populated in the risk score field.
Diligent One determines a risk level after comparing the risk score with ranges specified for the different risk levels defined.
1. Calculating risk scores
Each response in the risk assessment questionnaire has a weighting assigned to it. System Admins with Professional User subscriptions can customize each response weighting in the SIG Lite or CAIQ Lite Workflow robots.
Depending on the assessment you selected, a score for the questionnaire based on the highest possible score for each question: By default, risk score questions all have the following available responses and corresponding weightings:
| Response | Weighting |
|---|---|
| Yes | 0 |
| No | 1 |
|
N/A - Please provide an explanation Note When you select this option, the explanation you provide is not taken into account in the response weighting. |
0 |
Because you can only choose one of the above responses, the highest possible score is the highest response weight of all possible responses to the question (in this case, 1).
These questionnaires may also include informational questions that have different available responses, but those questions don't affect the risk score calculations.
Show me an example of a question's effect on a risk score
Example
The respondent is answering this question in the CAIQ Lite questionnaire: Do you use an automated source code analysis tool to detect security defects in code prior to production?
The respondent selects Yes. The response weighting for this question is 0.
The highest possible response weighting for this question is 1.
2. Assigning a risk level
Third Party Risk Management calculates the final questionnaire score as follows: categorization score = (sum of all response weights / sum of all highest possible question scores ) * 100%.
Lastly, the Workflow robot takes that percentage score and compares it with the risk level boundaries defined for your organization, which you can customize in the robot. The risk level corresponding with the range the percentage score falls into appears in your asset.
Show me an example of assigning a risk level
Example
After completing the CAIQ Lite questionnaire:
- The sum of the response weightings the respondent selected is 22.
- The sum of the highest possible scores for all questions is 73.
The final questionnaire score is 30% ( 22 / 73 ) * 100 = 30.14 ).
In your organization, scores between 0% and 40% are assigned the Low criticality level. Therefore, Low appears in the CAIQ Lite Risk Level field and 30 appears in the CAIQ Lite Risk Score field for the third-party asset.
Viewing your organization's calculations in Robots
System Admins with Professional User subscriptions can view the response weightings and impact level boundaries your organization uses to calculate the above scores using these steps:
- Open the Robots app.
- From the dashboard in Robots, select Workflow Robots.
- Navigate to the robot that contains the script you want to view, and select the robot to open it.
- In the upper-right corner of the robot, click Development to switch to development mode.
- In the Script versions tab, select the version of the script that you want to view.
-
Click Edit script.
Result The Robots script editor opens with the script. For more information, see Python and HCL scripting in Robots.
If you need assistance finding the robots that contain your customizations, or customizing your organization's calculations to meet your needs, contact support or your Diligent representative.
