Ask a Silly Question

Do you remember as a child, when you asked about something you didn't understand? The usual response for me was:

"Ask a silly question, get a silly answer"

That seemed to make sense as a child, and when I received this response I usually thought that I must be asking something that other people understood. The response indicated that my question made me look silly and I quickly learned to ask less questions.

And as I grew up, I threw the saying in the same group as other 'clever sayings', useful for making yourself look smart and others feel a bit more stupid.

"If you don't have anything worth saying, don't say anything at all"

"If you don't know where you're going, you'll end up somewhere else"

"Who is more foolish? The fool or the fool that follows the fool?"

But, the thing is that the saying "Ask a silly question, get a silly answer" causes major problems when it comes to adult life. Because we learn not to ask for clarity, we learn not to query what we have been told, we learn to accept blindly what the 'experts' put in front of us, whether its the media, the government...and yes, even the statistician.

So I'm going to suggest something controversial.

Silly questions are good and should be encouraged.

When I train others I often start by saying that there are no silly questions, that if one person doesn't understand then it is likely that others don't as well but are afraid to say so. How many times have you spoken up in a meeting and started with "This may seem like a silly question but...", only for others to also say that they haven't understood?

I remember being in an important meeting once. There were many senior people around the table, most of whom wouldn't even have known who I was. The meeting had started clearly enough, but very quickly it had developed into a jargon filled discussion that was too difficult to follow. I remember plucking up the courage, and speaking up.

"Excuse me, this may be a silly question, but does anybody understand what is being said here?"

Hushed silence, annoyed glares...and then somebody else piped up

"Yes, I'm not really sure what is being discussed - could someone explain in simple terms?"

There's always a breath of relief when someone else supports you, even more so when they are more senior (and more respected) than you. It was no different here.

But you know the really funny thing, the thing that sticks in my mind years later? Nobody could explain anything. The discussion had gotten so out of hand that nobody knew, or understood, what anyone else was saying. And because nobody had wanted to admit their lack of understanding, the discussion had progressed unintentionally into parody. Even more to my amazement, the person that I had interrupted, who was chairing the meeting, also had to admit that he really didn't know what he was talking about either.

Sheepish smiles, muttered apologies...but it wasn't the last meeting I attended with them where I suspected the same thing was happening.

So what's my point, and how does this relate to data and analysis?

We are in a current culture where we are competitive by nature, where we want to be seen as the "expert". I have blogged previously that everybody seems to think they are an "expert" in data analysis, and how this is a dangerous thing. But within our own professions it is also dangerous to put someone on a pedestal as an expert, if that means that we are no longer able to ask the silly questions...if we are unable to query whether what we are being told is right. Often we decide that it is the more experienced analyst, our manager, our director, the media, the government etc. that know better than us, or shouldn't be challenged. We assume, just as I did with my parents, that being in authority automatically made someone 'the expert' .....and by implication knowing all the answers. While I firmly believe in showing respect to others, this doesn't mean that we can't think for ourselves. And being able to ask silly questions about data is vitally important, because once you do it becomes clear that others, sometimes "the experts", don't have the answers after all.

Let me give you an example.

We have heard a lot recently about unemployment rates, that there are more people in employment than ever before, that fewer people are becoming unemployed. The government uses this to suggest that their policies are working, that austerity has been a success, that changes in benefits have encouraged people back into work. I am not going to say whether I agree with this or not, but I do want to show that there are some 'silly questions' about the data analysis that need asking. Here I am just going to concentrate on one (although there are more).

Is a decreasing unemployment rate a good thing?

This is a silly questions, right? Of course a decreasing unemployment rate is a good thing, isn't it? Fewer people being unemployed surely means more people are remaining in work?

Let's look in a little more detail. Here we're referring to unemployment prevalence, in its simplest terms defined as:.

A decreasing prevalence rate for unemployment may be the result of one, or two things

1. A decrease in the number of unemployed

2. An increase in the number of economically active

Unemployment in the UK is defined by the International Labour Organisation (ILO) - an agency of the United Nations, and falls into two categories

without a job, want a job, have actively sought work in the last 4 weeks and are available to start work in the next 2 weeks or ·
out of work, have found a job and are waiting to start it in the next 2 weeks

In general, anybody who carries out at least one hour’s paid work in a week, or who is temporarily away from a job (e.g. on holiday) is classified as "in employment". Those who are out of work but do not meet the criteria of ILO unemployment are economically inactive (and therefore excluded from all calculations).

So our silly question is actually asking, when is a decrease in the number of unemployed individuals, as defined by ILO, a bad thing? And actually, there are a number of scenarios where a reduction in recorded unemployed cases is a bad thing. The first, and most extreme, is mortality. If mortality rates among the unemployed are consistently higher than those in the 'economically active population', then the unemployment rate will drop. What about those who are severely ill, have mental health issues, and are therefore unable to start work in the next two weeks? What about the increase in those taking on zero hour contracts, of those working more than one hour per week, failing to register at the job centre (and therefore unable to evidence actively seeking work), failing to actively look for work in the last 4 weeks (for whatever reason)...all of these would result in a reduction in the number of 'cases' of unemployment. What about those who have savings who have lost their jobs but cannot claim benefit - and are therefore counted as economically inactive rather than unemployed? As more and more organisations downsize, this type of individual becomes more common. What about those who are ashamed of being out of work and don't want others to know? What about genuine mistakes in employment status by HMRC and others? What about those who were sacked or have resigned without another job to go to? What about those who are doing unpaid work, voluntary work. What about the increasing number of homeless?

So a reduction in numbers of people defined as unemployed may be for a large number of reasons, most of which do not include getting a job that pays a sufficient wage. Are you thinking any differently yet?

But, of course, a reduction in the rate can also be caused by the employable population increasing. Given the change in state pension ages, the failure of many personal pensions, increased cost of living, public health advice etc. more and more people are choosing to carry on working in their current role rather than retire. Others are going part time, or reducing to one day/week but still working sufficiently to be included as economically active. This means that the number of economically active is increasing. There is also migration - legal immigrants boost the economically active population, and also have high levels of employment. Illegal immigrants, or those awaiting a decision (and hence unable to work) are not included in the calculations. Ongoing changes to disability payments, benefits and other assessments have also resulted in more people being classified as "able to work", boosting the number of economically active.

Okay...given the above, our question doesn't seem quite so silly now, does it?

Now I am not saying that any of the above means that we are being wilfully misled over unemployment, and to an extent I picked the measure at random. I am also not saying that all the factors I raised are significant or likely to carry on in the long term. What I am saying is that even what we think of as "simple data" , really isn't simple. And that's problematic. We listen to others, to colleagues, the media, the government and believe what we hear without question. After all, it is easy to rely on someone else's understanding when they are perceived to be the expert.

Of course I needn't have concentrated on unemployment - I could have looked at disease prevalence, or waiting times at hospital etc. The same principles hold true. Always be willing to question your, and others', understanding. Is an increasing disease prevalence really a bad thing? It could be because:

we are getting better at catching cases early due to improved screening methods / techniques
there is less misdiagnosis of the condition
our coding of information via non-ambiguous terminology is getting better
our ability to get hold of recorded data and related data quality is getting better
less people are dying of the condition because of improved treatment

Or it could be a bad thing, not related to the disease in question, for instance

mortality from other conditions is increasing, reducing the population size
people are not being coded as 'recovered', to ensure continuing payments via incentive schemes (known as gaming)
people with this disease are dying of something else and are not being removed from systems
people are being registered with health services more than once

Statistical analysis, visualisations, forecasting, process control and the like, all tell you whether numbers are increasing or decreasing. What they don't tell you, and can never tell you, is why. As the 'data savvy' generation, with so much data at our fingertips we run the risk of forgetting the basics, getting carried away with 'clever analysis' and ending up drawing the wrong conclusions. This could, of course, have major unintended consequences.

So I would challenge you to think a little more as to what data really means. And once you have done so, don't be afraid to ask the silly question - the answer might be more important than you think.

Go to Training

Page updated

Google Sites

Report abuse