Considerate Cycling 46: How Not to Do Survey Research

etc

BBC Bristol News have recently published two surprisingly uncritical reports arising from an on-line survey of attitudes to street parking policy in Bristol.

After a (not-returned) telephone call from me and a bit of Twitter activity by the Mayor, BBC Bristol News made some changes to the text of the most recent article but some dubious statistics given by the research author are still reported. See http://www.bbc.co.uk/news/uk-england-bristol-30512827 (As of the afternoon of 18/12/2014)

Behind the BBC Bristol News item is a long-running story of Bristol’s attempts to tidy up conflict between resident and commuter parking. A previous Council Administration and now the elected City Mayor have pursued the idea of setting up Residential Parking Zones. It’s a very boring topic for everyone else, but for interested parties it seems to arouse strong feelings.

The immediate story is that former Ford employee, now Bristol businessman, Mark Moran wrote and published a web questionnaire on the subject of Bristol’s parking scheme. I have not been able to find a data set or a report of Mark Moran’s findings so the following comments are based on the questionnaire alone. My own interest in the topic is as a postgraduate degree holder in research methodology, as a resident of Clifton with specific problems related to parking in Bristol, and as an advocate of cycling within cycle campaign associations.

I have annotated the full text of the questionnaire with the sorts of observations that I would have been able to share had the BBC News Editor called me back as promised.

Bristol Residential Parking Zones – Does the city want them?

Have your say.

Asking the real questions about RPZ

There has been no full survey done of the whole of Bristol and its surrounding areas with regards to RPZ. Feelings are running very high on this issue and it is the one most often in the news

The intention of this survey is to reach the entire ‘Greater Bristol’ area to establish once and for all the true feelings about these schemes

[a survey about the “Greater Bristol Area” would need to collect information about where respondents lived and worked so that the representativeness of the sample could be established. A publicised web survey will clearly recruit only a self-selecting sample whose general characteristics (age, residential area, car ownership, employment and so on) would need to be tested against the known characteristics of the population of the area as a whole.]

1. Do you agree with the blanket introduction of Residential Parking Zones across Bristol? Yes no

[The unexplained adjective “blanket” makes a clear answer more difficult. Is this a question about the scheme per se, or is it about the scale and pace of the scheme’s implementation? As a type of question it comes under the heading of “portmanteau question” – a question that could be construed as being two or more questions in one. Such questions can be difficult to answer and their later analysis cannot be done with confidence]

2. Do you think the Mayor and Bristol City Council have listened to the concerns of residents and businesses over RPZ? Yes no

[Again, this is two (or more) questions in one]

3. Do you think an efficient cheap and reliable public transport system should be in place before RPZ’s [sic] are introduced? Yes no

[This question contains so many ideas that answers to it could mean almost anything. It’s hard to see how it asks for any kind of answer that would not already have been given to Question 1.]

4. Do you think these schemes will discourage people from travelling to work in Bristol?

Yes no

[A simple categorical question with a yes/no answer? On the face of it yes. In practical terms it is more helpful to respondents and to later analysis to offer a range of options when opinions are sought. However this is an opinion question about other people’s likely behaviour. Why not just ask the respondents about their own behaviour? Even that sort of question is hypothetical, but the answers to it might be more well-grounded and reliable]

5. Do you think these schemes will cause businesses to close? Yes no

[The same problems arise here as we see in Question 4. As with 4 it is a hypothetical question of a type generally used only to test general attitudes or opinions and very rarely with just a yes/no response. My own reading of this question (and some others) is that the author is simply marshalling the beliefs he already holds and is looking for people who agree with him]

6. Do you live in Bristol, Work in Bristol or Visit Bristol? Live Work or Visit

[This one is a mess. Three different questions are rolled into one and a number of interesting or relevant others are omitted, such as “do you work at home?” “are you retired?” “do you have a condition that affects mobility?” or “are you a student?”]

7. Should the residents of the affected areas be forced to pay for the RPZ? Yes no

[The word “forced” is an emotionally charged word that indicates that the questioner does not agree with the idea of resident parking schemes however they are financed. It also seems to rest on confused ideas about how the various elements of consultation, implementation and future running costs have been and will be financed. Offering one rather stark and unlikely option “residents … be forced to pay” makes it hard to imagine anyone ticking Yes .]

8. Would you support raising funds for a judicial review into the RPZ programme? Yes no

[This question suggests that the author is planning to or would like to attempt such a legal challenge and that he is asking for financial support. The possibility of gathering a representative set of responses from the questionnaire when his own views are so close to the surface must be thrown into doubt.]

9. Would you support the cancellation of all planned RPZ’s [sic] and the reversal of those already in place?
Yes
no

[as Q8]

10. Please use this box to add any further comments


[This question is potentially the one that will generate some interesting responses. It will also be the hardest one to analyse.]

Advertisements