Standard-Examiner Editorial: OUR VIEW: UTA's Secrecy Bug

Final calculations are useless unless the initial assumptions built into such calculations can be independently reviewed and validated

Just to get the conversation rolling this morning, we'll cast the spotlight upon yesterday's strong Standard-Examiner editorial, hammering the UTA's stonewalling of raw data which ostensibly support a consultant's findings that a proposed mass transit route -- along Washington Boulevard, 36th Street and Harrison Boulevard -- to Weber State University and McKay-Dee Hospital makes the most economic sense":

• OUR VIEW: UTA's secrecy bug

And whatever you do, don't miss Cal Grondahl's most excellent companion cartoon:

• Grondahl 10/17/10 Cartoon

Here's the key principle, as earlier set forth in Dan Schroeder's 9/20/10 letter:

...it is clear that we cannot trust the bottom-line results when the consultant is unable to show how it obtained them. It is also clear that UTA was negligent in not asking to see the detailed calculation itself, months ago. It is unprofessional and inexcusable for UTA to ask stakeholders to make a $150 million decision on the basis of data that has not been carefully scrutinized by its own professionals. (However, it is human nature to accept results without scrutiny when they tell us what we already wanted to hear.)

As an added bonus we'll refer to an article posted 10/10/10 on Dan Schroeder's blog, Dan's Diary, in which he emphasises the importance of "[u]nderstanding uncertainties, and learning to live with uncertainties, and stubbornly insisting on knowing the uncertainties when making predictions" based on mathematical data:

• Uncertainties in Science and Politics

Here are Dan's money paragraphs:

I’ll end (once again) with a local political example. A respected economic consultant recently predicted that the middle segment of Ogden’s proposed streetcar system will stimulate $8.5 million of investment if it follows one proposed alignment, but only $1.5 million if it follows an alternate alignment. Ten days ago I asked her what the uncertainty range is on those numbers, and she replied, “Well, you can see that we rounded them to the nearest half million.” I’m afraid I laughed at that point, and tried unsuccessfully to convince her that the uncertainties were many times larger. I knew the numbers had been calculated from property value assessments, and that these assessments can be systematically off by 50% or even more. Worse, I knew that the lists of properties to be included in the calculations had been compiled through a subjective, undocumented process. After our conversation I looked up some of the property assessments and quickly saw that you could increase the $1.5 million prediction to over $9 million by excluding just two properties (out of several dozen) from the list. A fair estimate of the uncertainty would be much higher still.

But economic consultants apparently aren’t in the habit of thinking about uncertainty. Undoubtedly this is because their clients don’t want to hear about it; they just want simple answers. In this case the client was the Utah Transit Authority—a government agency that supposedly represents the people. Ultimately, it is the citizens at large who need to learn to think like scientists.

Remember back in math class when your instructor insisted that you "show your work" as you performed mathematical calculations? Same basic concept. The results of final calculations are useless unless the initial assumptions built into such calculations can be independently reviewed and validated.

A Weber County Tip O' The Hat to the Standard-Examiner editorial board for keeping the UTA's highly unprofessional conduct in the public limelight... and a well deserved Hat Tip to Dan Schroeder too, for keeping everybody on their toes in re this matter.