To its critics, Compliance, Safety, and Accountability 2010 (CSA 2010), the Federal Motor Carrier Safety Administration's (FMCSA) ambitious effort to remove unsafe drivers from the roads, has skated on thin ice throughout its four-year life. In the wake of a federal government study released this week raising concerns about the program's methodology, the ice may be thinning some more.
On Monday, the Government Accountability Office (GAO), the investigative arm of Congress, issued a 111-page report analyzing the program's effectiveness in measuring the safety fitness of the more than 500,000 truckers operating on U.S. roads each year. On one level, the report gave a thumbs-up to the program, better known as CSA 2010, and a controversial system called Safety Measurement System (SMS), which tries to identify high-risk truckers by gathering carrier performance data from accident investigations or roadside inspections and then grading the carriers by calculating violation rates for those analyzed and comparing them to similar carriers over a matrix of seven categories.
According to the GAO report, SMS has improved public safety by broadening the number of potentially high-risk truckers subject to the agency's "intervention." This often involves the issuance of warning letters but can also take the form of unannounced inspections of a carrier's facilities. In a period of five fiscal years ending with fiscal 2012, FMCSA more than doubled its number of annual interventions, GAO noted.
However, the report said the flaws in the system's methodology make it difficult for FMCSA to reliably assess the safety risks of most carriers. GAO said that in order for SMS to effectively identify carriers most likely to be involved in accidents, the violations that the agency uses to calculate SMS scores should have a "strong predictive relationship" with crashes. But the federal regulations used to compute SMS scores are not violated often enough to strongly associate them with the crash risk of individual carriers, according to GAO.
The report also found that most truckers lack sufficient safety data to ensure that their performance can be reliably compared to other carriers. About 95 percent of the nation's fleets operate less than 20 vehicles, and FMCSA lacks the funding to inspect such a broad universe frequently enough to collect even the minimum amount of data to generate a reliable SMS score. The grading system is the agency's way of squeezing the most inspection productivity from scarce resources. Critics allege this "grading on the curve" approach lumps together high-risk and low-risk carriers, making otherwise fit carriers guilty by association and casting a cloud over the efficacy of the program.
The report urged FMCSA to revise its SMS methodology to better reflect the limitations it has in gathering safety information and for using it to compare carrier performance. Those limitations should also be taken into account when FMCSA determines a carrier's fitness to operate, the report said. GAO said the Department of Transportation, the FMCSA's umbrella agency, agreed to consider the recommendations.
The GAO report was requested by the House Transportation & Infrastructure Committee. Failing to take any steps in response to the findings would be tantamount to FMCSA, in the words of an individual close to the issue, "thumbing its nose" at the committee.
FMCSA plans to use CSA data when it opens a safety fitness rulemaking scheduled for later in 2014. In addition, it publishes the SMS scores on its website. Neither the use of CSA in a rulemaking or the public disclosure of SMS scores sits well with truckers, brokers, or shippers. For example, by making scores publicly available FMCSA invites erroneous carrier-selection judgments based on inaccurate data, they argued. This, in turn, exposes shippers and brokers to significant liability risk in the event of a crash-related fatality or injury if a plaintiffs' lawyer argues they chose a carrier that they thought was in good safety stead but actually wasn't, or that they declined to rely on CSA data that showed a history of one or two safety infractions, they said.
The best near-term remedy is "removing the scores from public view," said Dave Osiecki, executive vice president and chief of national advocacy for the American Trucking Associations (ATA), a trade group of large carriers. The ATA has long supported CSA's objectives but, over the years, has grown increasingly uncomfortable with the validity of the process. The association said the GAO report confirms that the SMS scores don't present an accurate assessment of the safety of many truckers. The group urged FMCSA to make immediate changes to the program.
In a statement on its website, FMCSA said the GAO report shows that the SMS program has been more effective in identifying carriers for "targeted enforcement" than the old safety measurement program, known as "SafeStat," which was replaced in 2010. According to the agency, researchers analyzed the links between historical safety data and future crash involvement by taking two years of pre-SMS safety information for a subset of carriers, running it through the SMS algorithms, and then following those companies' crash records for eighteen months. The results showed that the companies that SMS would have identified for intervention had a future crash rate of twice the national average, according to FMCSA.
In addition, 79 percent of the carriers that SMS would have ranked as high risk in at least one of the seven categories had higher future crash rates compared to those the system would not have identified, according to the FMCSA statement.
Editor's note: An earlier version of this article incorrectly stated that FMCSA's intervention could take the form of announced inspections instead of unannounced inspections.
