Pouring scorn on the Sun has become a bit of a thing on social media after the newspaper splashed on a new opinion poll that apparently showed significant support for “jihadis” among British Muslims.

This was the front page of Monday’s edition (try not to get distracted by Vicky Pattinson from I’m a Celebrity):

The headline says: “1 in 5 Brit Muslims’ sympathy for jihadis”.

The first paragraph of the article reads: “Nearly one in five British Muslims has some sympathy with those who have fled the UK to fight for IS in Syria.”

The story was based on this telephone survey of 1,003 British Muslims carried out by the polling company Survation. As always with opinion polls, we are assuming that the views of a small group of people are representative of the wider population.

One question in this poll is generating all the fuss. Some 20 per cent of people said they had “some sympathy” or “a lot of sympathy” with “young Muslims who leave the UK to join fighters in Syria”.

Numerous commentators have taken issue with the Sun’s take on this statistic, for two main reasons. Let’s look at each of the criticisms and see if they are fair.

The people who took part in the survey may not have been representative of all British Muslims

Much has been made of the fact that Survation called a list of people with Muslim-sounding names to try to narrow down their search for British Muslims.

This doesn’t mean that we don’t know whether the people who took the calls were really Muslims or not: the researchers asked what their religion was before carrying on with the questionnaire.

Is looking for surnames suggestive of ethnic or religious background a strange thing to do when you are trying to identify a sample from a particular minority?

This certainly isn’t the first time it’s been done. Survation used a similar technique when they surveyed British Jews for the Jewish Chronicle newspaper earlier this year, apparently without controversy.

And fellow pollsters Ipsos Mori did something similar recently while polling British Jews for research commissioned by City University.

In a statement today, Survation also said they found Muslims using “geographic targeting” and “our growing telephone opinion panel of British Muslims”.

Political polling expert Professor John Curtice from the University of Strathclyde told us Survation’s methodology was imperfect but understandable, given the practical difficulties of finding people from a small minority to survey.

Focusing on traditionally Muslim surnames would miss converts to Islam with western-sounding names, while concentrating on parts of the country with a large Muslim population means you miss Muslims who live in other areas.

All of these things increase the danger that you are not generating a sample truly representative of Britain’s whole Muslim population, although like all pollsters, Survation have to practise the art of the possible.

Prof Curtice said: “The truth is that anyone who wants to do a survey about the Jewish population, or the Muslim population or any minority group undoubtedly faces a fair old methodological challenge.

“Can you do it perfectly? No. Is it unreasonable to do it? Put it like this… I can think of good reasons as to why understanding the attitudes of Muslims is important.”

The wording of the question was ambiguous

The question doesn’t actually mention the Islamic State group (Isis, Isil, Daesh) by name. It just talks about young Muslims joining “fighters in Syria”.

Conceivably, people could have had other groups in mind when they expressed sympathy for Britons heading to the war-torn country.

The question is vague enough to include people fighting for the Assad regime or Kurdish forces battling against Islamic State.

And the word “sympathy” arguably covers a multitude of sins too: I can have sympathy with someone without endorsing what they do.

The Sun has robustly defended its claim that this question would have been understood to relate to IS, saying in an editorial today: “No one agreeing to the statement ‘I have a lot of sympathy with young Muslims who leave the UK to join fighters in Syria” was in any doubt which fighters we meant.”

Survation say: “The wording of the question… was not chosen by The Sun newspaper but was chosen by Survation in order to be completely comparable with previous work we have done, both among Muslims and non-Muslims and therefore enable meaningful and proper comparisons to be drawn.

“However, there is a distinction between the work we do and how clients chose to present this work. Survation do not support or endorse the way in which this poll’s findings have been interpreted.

“Neither the headline nor the body text of articles published were discussed with or approved by Survation prior to publication.”

Survation is at pains to point out that it asked this question before in a poll commissioned by Sky News in March, and an even greater proportion of Muslims said they had some sympathy with fighters in Syria.

The latest poll actually shows a reduction in sympathy for those who travel to Syria between March and November, as this Survation graphic shows:

But that fact did not trouble the Sun’s headline writers.

In March the company surveyed non-Muslims, asking the same questions, and found that there was some sympathy for young Muslims heading for Syria here too.

Some 13 per cent of non-Muslims had “a lot” or “some” sympathy with the fighters, compared to 28 per cent of Muslims in March, and 20 per cent in November.

So what do Muslims think about IS?

We can knock the Sun’s story down on a number of levels, but the bigger question remains: is there support among British Muslims for the so-called Islamic State?

Pollsters ICM did some research for the Sun’s deadly rivals, the Daily Mirror, on this in July.

Now there was much less ambiguity in the question: responders were asked whether they had “a very favourable, somewhat favourable, somewhat unfavourable or very unfavourable opinion of the Islamic State otherwise known as Isis, or Isil?”

This time – with the Islamic State clearly named – some 9 per cent of people said they had a “very” or “somewhat” favourable view of the terror group.

But this was a poll of a sample representing the whole population of Great Britain, not just British Muslims.

The Mirror’s spin on this was that “around half of Britain’s three million Muslims could be ISIS sympathisers”.

This appears to be based on the assumption that everyone who said they liked Isis was a Muslim, although there is absolutely no evidence in the polling to support this.

This is arguably as outrageous a statistical claim as the one made by the Sun on Monday, but it does not seem to have provoked as much anger, possibly because the headline steered the story in a different direction: “ISIS-supporting Brits may be disenfranchised by Tory cuts”.

Another ICM poll in August found that support for IS among the general British population was dwarfed by enthusiasm for the group in France. This was before the latest Paris attacks but after the Charlie Hebdo massacre in January.

What about other polls that attempt to zoom in on British Muslims alone?

As we noted in a previous FactCheck, right-wingers have tended to spin the results of various opinion polls carried out with the Muslim populations of different countries to paint a simplistic picture of blanket extremism.

But that does not hide the fact that some surveys have shown evidence of troubling, radical or illiberal attitudes among British Muslims.

Some examples include this 2006 study on British Muslim attitudes by GfK NOP Social Research, which found that 30 per cent wanted to live under Sharia law, 28 per cent wanted Britain to be an Islamic State, 22 per cent thought the July 7 2005 London bombings were justified because of British support for the war on terror.

In a 2006 Populus poll, 16 per cent of British Muslims said suicide bombings in the UK against military targets might be justified.

A ComRes poll for the BBC in February this year found that 11 per cent of British Muslims agreed with the statement “I feel sympathetic towards people who want to fight against western interest”, while 27 per cent had “some sympathy for the motives behind the attacks on Charlie Hebdo in Paris”.

Many polls find that younger Muslims are more radical than the older generation. A YouGov survey of Muslim students in 2008 found that 33 per cent supported “a worldwide Caliphate based on sharia law”, among other things.

Frustratingly though, while we have polls that ask how Brits in general feel about IS, and polls that ask British Muslims other searching question about their attitudes, but we can’t find good research that nails the exact question of British Muslim support for IS.

There is of course always an element of doubt as to how far these questionnaires can capture the nuances of people’s views, and how honest people are being.

Some academics have cautioned against reading too much into polls that show Muslim support for terror organisations, arguing that people sometimes declare allegiance to extremists as a kind of “protest vote” against western foreign policy.