Flexible Facts

Flexible Facts, it’s in the data

By Alan Perry

February 15, 2014

Data has become increasingly more important both in the practice of Fire & EMS as well as the management of these vital public safety institutions. Data should never be subjective, yet frequently the way in which it is vetted, gathered, analyzed and used is just that, and in some cases downright deceitful. It is incumbent on anyone trying to make a good decision based on data to do their homework; making certain that the data is accurate, relevant and free from bias and other influences that may corrupt it. Good decisions also require the good judgment of the evaluator and his/her ability to recognize their own biases in the process. It is not uncommon for poor or incomplete data to be used and/or manipulated to make some predetermined point. Don’t let the numbers fool you, especially if you know an agenda already exist, the numbers don’t always tell the whole story and at best are only a small part of the information needed to make a good objective decision.

If I were a Fire Chief trying to justify the number of engine companies in light of declining call volumes I might simply move my ALS personnel to the engines, requiring the engine companies to respond to more EMS calls and increase their call volumes. If that resulting statistic is looked at by itself the point will have been made, however looking at the total circumstance I might discover that it was a simple card trick. This also compounds the problem because of the additional expense involved with running full engine companies on EMS calls that they are not needed on and takes resources away from the agency that could be applied elsewhere in the taxpayer’s interest.

An EMS Chief can play this game too; when faced with the possibility of losing a medic company he/she might “voluntarily” take the busiest medic company out of service, the resulting increase in response times across the whole system could be used to suggest that losing that company has produced catastrophic results and that staffing must be restored if not increased. By looking only at the response times as an indicator of system staffing needs one could be misled by the data. Playing games with people’s lives is a certain way to lose credibility and the confidence of the public.

An EMS system with poor data discipline might deduce from data acquired with poor documentation practices that its mid-level EMS providers are not performing their skills and recommend doing away with that skill level to save the department money on their training and recertification. Looking a little deeper one would discover that these providers are the backbone of the organization extending their skills on both BLS & ALS calls. The source of the data error is that only one provider writes the report and frequently does not give other providers credit for those skills because it is simply too difficult to keep track of them in the reporting system. It is likely this reporting error was known and the data simply used as an excuse to reduce training expenses at a time when expanding EMS education is even more necessary.

Data always deserves investigation; know its source, its accuracy and the bias of the parties involved. Do not view it in isolation; if it seems questionable assume there might be a problem with it and look for the flaw(s). Do not build research around data and do not build data around research, if you focus your efforts on a specific outcome you will probably find it while excluding the important facts. Data should be viewed in a scientific light, in science things must be proven and able to be duplicated before they are considered fact. The same standard should apply to data, for data to be sound it should be free of defect and produce the same results consistently without external manipulation. If your data cannot hold up to this standard it should not be used, especially when the lives and careers of others are at stake.

Be Diligent,

Alan