DIRT Report

Data Quality Index (DQI) - Impact of Data Quality

• DIRT fields with high percentages of unknown data lead to greater uncertainty about the accuracy of analysis.
• Higher quality data across all stakeholder groups is critical to identifying and focusing efforts on factors that have an impact on reducing damages.  

Data Quality Index (DQI) is a feature within DIRT to score data quality. It was developed to provide submitters with confidential feedback based on measures of the completeness and/or quality of the reports they submit. For each DIRT field, points are assigned based on the importance or value of that question to overall data analysis. If fields are left blank or if “unknown” is selected, the submitter receives zero DQI points for that particular field. The intent is to identify opportunities for DIRT users to improve their score by completing as many optional DIRT fields as possible.

When a DIRT submission is entered, the user is shown a chart as shown below. The middle column (weighting %) shows the maximum value of that “part.”  The first column (Score (/100)) shows the percent of the weighting achieved for the report (or the average of all reports in a bulk file upload).

For example, Part D^[1] has a weighting of 20%, and includes these questions (as answered for example above).

Figure 24 provides a depiction of the number of companies entering DIRT data, their number of reports, and average DQI scores for 2021 reports.

Figure 24

Most exhibits in DIRT Reports are based on “known” data, meaning that “unknown” selections or blank fields are filtered out. This requires an assumption that the data masked by the “unknowns” is proportional to the known slices. For example, for type-of-excavator, the full 2021 dataset has 55.27% contractor and 29.85% “unknown.” When we remove “unknown”[1] from the total denominator, contractor jumps to 78.78%^[2] –  a difference of more than 20%. This is the best assumption we can make given the data, but the reality may differ. Occupants and/or utility might really be higher, and contractors could be lower. The no locate request root cause increases from 17% to 26% respectively with “unknowns” included versus excluded in the denominator. Reducing unknowns, especially in these high-DQI-value questions, reduces the margin for error and makes analysis more accurate.

Table 18 shows the number of reports by DQI range. It is encouraging that the largest band in terms of report count – nearly half – are in the “good” DQI range. However, nearly 35% of reports are below 60, indicating room for improvement.

Table 18—Reports by DQI Ranges of 20

Table 19 shows the average DQI by event source for all 2021 reports.

Table 19—Average DQI by Event Sources

Many reports attributed to excavators as the event source are initially reported to an 811 center (one call center), which then enters the DIRT report. Table 19 shows a DQI of 60.94 for excavators. This is a blend of reports submitted through 811 centers (56.46) and reports submitted directly by excavators (84.51). 

Because the volume of reports submitted through 811 centers is much higher, the blended result is skewed in the lower direction. There actually is a wide range of DQI scores for excavator reports submitted through 811 centers. Of the ten 811 centers that gather information from excavators, six have a DQI of 75 or above, three are below 40, and one is in the low 50s. Colorado 811, one of the high volume/high DQI submitters spotlighted in this section, demonstrates the potential for high DQI utilizing the 811-center-entered data submission model.

The 811 centers with DQI below 40 occasionally enter “known” values for the high-DQI-value DIRT fields, but mostly enter “unknown.” These reports provide us with a count of damages and indicators of overall trends, but provide limited additional information such as identifying root causes and types of excavator, equipment, work performed, etc.  811 centers have an opportunity to provide more complete data and greater insight into the excavator submitted damages.