The American Trucking Associations today issued a white paper calling into question the reliability of scores from the Federal Motor Carrier Safety Administration’s (FMCSA) Compliance, Safety, Accountability (CSA) program in evaluating the safety of individual trucking companies.
CSA is the administration’s safety monitoring and measurement system used to identify unsafe carriers and prioritize them for future interventions or audits. The agency also encourages third parties to use CSA Safety Measurement System (SMS) scores as a tool for making safety-based business decisions. The FMCSA hopes to leverage the power of the marketplace to make judgments about carriers and, as a result, compel them to improve their safety performance.
SMS scores also have the potential to be used by plaintiffs’ attorneys and prosecutors in the context of post-crash litigation.
“ATA continues to support the objectives of CSA and to call for improvements to the program,” said ATA President and CEO Bill Graves. “However, data and methodology problems continue to plague the system and the accuracy and reliability of companies’ scores.”
In its paper, ATA examined data and research on the connection between CSA scores and crash risk, as well as how problems with the data and methodology produce an imperfect and unreliable measure of a carrier’s safety record.
“It may make sense for FMCSA to use scores in those categories that correlate positively with crash risk to prioritize companies for enforcement review,” Graves said. “In the process, FMCSA can verify whether or not the scores paint an accurate picture. But third parties need to know that for the purposes of drawing conclusions about individual carriers, the scores are unreliable.”
According to research cited by ATA, scores in at least three of the system’s measurement categories don’t bear a positive correlation to crash risk. Even in those categories that generally have a positive correlation to crash risk, the paper points out that there are tens of thousands of real-world “exceptions,” carriers with high scores and low crash rates and vice-versa.
The relationship between scores and crash risk is impacted by a number of data and methodology problems that plague the system, according to the ATA study. These include a substantial lack of data — particularly on small carriers that make up the bulk of the industry. It also includes regional enforcement disparities; the questionable assignment of severity weights to individual violations; the underreporting of crashes by states; the inclusion of crashes that were not caused by motor carriers; and the increased exposure to crashes experienced by carriers operating in urban environments.