Does This Election Violates Everything We Thought We Knew About Data?

I don't know if it's too early to jump on to this thread after yesterday, but I'm Australian so I'll be brave!

I don't think data-driven systems are necessarily broken, I just think they're not the silver bullet we sometimes like to think they are.

We know this from the arts. Using data-driven marketing and development approaches, we can get a better response on our marketing efforts, we can put online advertising in the right places, etc. 

But ...

... at the end of the day, we're at the mercy of how many people in our town like the sound of our theatre / classical music / dance that we are performing. Data is great in helping us broaden our communications, in helping us target the right people. But the ultimate factor is still who or what is on the stage, and what potential audiences think they'll get out of it, right?

Matthew, from my point of view it is never too early to have a thoughtful conversation.  

I completely agree with you that data is not a silver bullet.  

This is particularly true when we are working with historical data and trying to create a predictive model of a future event. The idea of a predictive model is to make one somewhat better than just making a straight random guess. It will never be perfect.  (Remarkably good for the typical cases is about the best we are likely to get.)

Here is what I think I've learned so far about what has happened with the 2016 US Presidential election predictions.

With predictive modeling, one will have incomplete data, so one has to make guesses to fill in the missing elements / features.  From what I'm hearing the issue with the US 2017 election predictions is that folks making the predictions had problems with the assumptions they made about level of engagement among the rural voters based on past voter behavior (or might I say apathy). Apparently, it had been as the cities went so went the states.  In this case that did not happen.  In state after state the rural voters (who in the US tend to be more conservative) voted in larger numbers than expected.

I also like your point, that the connection between the customer, and the product is an important.  Apparently, many folks liked neither of the candidates in this case.  

So, is there anything we can learn from this as Arts and Cultural Administrators?  Is it just that Black Swans exist?  (Be prepared to be surprised?)  That is useful.  However, are there other useful things we can learn?

A related article, and different perspective on the lessons we can learn about predictive analytics as a result of the 2016 Presidential Election and the usefulness of predictive analytics.

https://www.powerpivotpro.com/2016/11/did-analytics-just-get-ahem-trumped/

I've followed Rob Collie for a number of years now, although I don't agree with all that he is saying here. He has some very good points.  And if you want to learn about MS Power Pivot his stuff can be very useful.

The additional comments are also useful in this thread to get some good counterpoint to Rob's perspective.