Avoiding oversimplification
On The World Tonight this week we've been looking at Darfur and Iraq - both subjects that came up during the new prime minister, Gordon Brown's trip to the US and the UN.
On Monday, in his press conference with President Bush, Mr Brown, said:
- 鈥淒arfur is the greatest humanitarian disaster the world faces today and I've agreed with the president that we step up our pressure to end the violence that has displaced two million people, made four million hungry and reliant on food aid and murdered 200,000 people.鈥
On Tuesday, Sir John 91热爆s, the head of the UN's Office for the Coordination of Humanitarian Affairs, , suggested Iraq may qualify for world's worst humanitarian situation with eight million Iraqis now dependent on humanitarian aid:
- 鈥淥ne of the world's largest and fastest-growing humanitarian crises is also among the least known: Iraq. More than four million people, one out of every seven Iraqis, have fled their homes in what is the largest population displacement in the recent history of the Middle East.鈥
There are also an unknown number of civilian dead with estimates ranging from President Bush's 30,000 to the of more than 600,000.
Covering both these conflicts in broadcast journalism can be difficult, because in a radio or TV programme you have a limited amount of time to explain complex situations to audiences who may not be very familiar with the details. Hence the need to simplify, the art of this is simplify without distorting the picture - something we take pains to try to avoid.
On Darfur, earlier this week, we spoke to the Sudan specialist, Julie Flint, who said the new resolution backed by a threat of further sanctions is unlikely to work unless the rebel groups are brought together and there is a cohesive peacekeeping effort.
Tonight (Friday), we are planning to lead on Darfur because the rebel groups are meeting in Tanzania which gives us an opportunity to assess how likely it is the competing factions will sign up to a peace agreement so the newly agreed UN force will have a peace to keep.
We'll also talk to some veterans of the UN force in Bosnia because there are concerns the mandate of the new force for Darfur may suffer from the same weaknesses and ambiguities as the Bosnian force, which struggled for three years to deliver aid and was unable to stop the killing of thousands of civilians, let alone keep the peace.
We will also analyse how complex the Darfur conflict is - a mixture of competition over increasingly scarce water resources between settled agriculturalists and nomadic herdsmen, as well as among other things, an attempt by the Sudanese government to put down a regional revolt and a conflict between ethnic Arabs and Africans. We'll also ask whether the coverage by some Western journalists and the rhetoric of some Western leaders and pressure groups that say the Sudanese government is carrying out genocide in Darfur is an oversimplification that has impeded attempts to reach a political solution to end the fighting.
On Iraq, we heard on Wednesday from our correspondent in Baghdad, Nicholas Witchell, that there are signs the American-led 'surge' is making progress in military terms. But on the same day the main Sunni party in the governing coalition announced it was leaving so we turned to Yahia Said, an Iraqi specialist at the London School of Economics, who told us there has been no progress on achieving political progress there between the Sunni and Shia political groups.
Again - not a simple picture and difficult to report without, on the one hand confusing the audience, and on the other presenting a misleading picture through oversimplification.
It would be interesting know what you make of our efforts.
Comments
Trying to get through the maze, separating the grain from the chaff and highlighting the dismal situation in Dafur is almost a thankless task. The grim reality coupled with the callous disregard for the magnitude of the problem by countless world leaders baffle arm-chair critics and concerned observers, The United Nations is finally acting and one can only hope and pray the situation changes for the better.
"There are also an unknown number of civilian dead with estimates ranging from President Bush's 30,000 to the Lancet's of more than 600,000. "
I think there are some important things to clear up here. First of all, not all of the figures are estimates and certainly not estimates of the civilian dead. For example, one of the most widely quoted figures is from Iraq Body Count which is not an estimate at all. It is simply a count of violent deaths that happen to make it into the western media. Aside from being a non-estimate, it is most definitely not an estimate of the excess civilian deaths due to the war. Any such estimate would have to take account of deaths due to disease which often rises in a war zone. People who die from disease due to a war are every bit as much victims as those that are killed violently.
Then there is the question of accuracy. When it comes to a media-based count, no-one has any idea about the accuracy and quoting a figure without an error estimate is fairly meaningless. Any journalist in Iraq will tell you just how difficult it is to do any reporting, so any kind of media-based count is bound to be terribly inaccurate.
In contrast the Lancet Report was indeed an estimate of the civilian dead and it was carried out carefully by professionals. Unlike IBC they provide errors estimates as well as the figures, which is crucial and it has widespread academic support. It's strange then that the head of 91热爆 news is on the record as saying the following.
"The Lancet study is a snapshot taken more than 18 months ago and though the methodology has been widely acknowledged as standard, there has been argument about whether the sampling method is the most appropriate for this kind of survey... I have also consulted our specialist in our analysis and research department and we conclude that the best source is probably the Iraq Body Count website."
I'm sure everyone would like to know who this "specialist" is, and why he/she believes that IBC figures are the best when they do not even include excess deaths due to disease (nor can they). Since the Lancet report uses standard statistical sampling methods, will the 91热爆 also be rejecting the results of it's own surveys which use similar methods?
....an attempt by the Sudanese government to put down a regional revolt and a conflict between ethnic Arabs and Africans. We'll also ask whether the coverage by some Western journalists and the rhetoric of some Western leaders and pressure groups that say the Sudanese government is carrying out genocide in Darfur is an oversimplification that has impeded attempts to reach a political solution to end the fighting.
You guys have a real way with words. What's really happening is that Arab Muslims have been carrying out a programme of ethnic cleansing and genocide of Africans in Sudan - of African Christians in the south and more recently of African Muslims in Darfur. Arab Muslims have killed hundreds of thousands of Africans and driven more from their homes. All this directed and controlled, of course, by the Sudanese government.
And what is this implication that Western journalists and leaders are somehow distorting the real picture by speaking of genocide?
The 91热爆's lack of any real effort to report honestly on Sudan over the years has been nothing short of disgraceful. You have failed utterly in your task as journalists to bring us the truth about Sudan, including the role of China in supporting the genocidal Sudanese government because of oil interests, and you have been complicit in obscuring the true nature of the "conflict" and keeping people in ignorance as to what is really going on there.
Believe that you are "avoiding oversimplification" if you like, but what you have really been doing up till now is avoiding your duty.
And now that you can no longer avoid the issue, since worldwide pressure is building up to resolve the crisis, you want us to tell you what we think about a few hours of your present coverage? What have you done in the past to make us believe that your coverage will not simply be more of the same and therefore a complete waste of time?
Burnett writes:
"Covering both these conflicts in broadcast journalism can be difficult, because in a radio or TV programme you have a limited amount of time to explain complex situations to audiences who may not be very familiar with the details."
I can imagine this can be even more difficult when some in that very audience lobby the 91热爆 to report falsehoods about these details, such as in the letter above:
"[IBC] is simply a count of violent deaths that happen to make it into the western media."
But making oneself familiar with the IBC's sources would reveal that it includes quite a lot of non-"western" media sources:
"In contrast the Lancet Report was indeed an estimate of the civilian dead"
But the Lancet was not, and could not be, "an estimate of the civilan dead" at all. The report explicitly states that, "Separation of combatant from non-combatant deaths during interviews was not attempted."
Then it is claimed Lancet "has widespread academic support", but this is also, at best, a half-truth. It's been criticized heavily, and continues to be, by researchers across the spectrum of the academic community.
And the notion that Lancet provides some kind of firm "errors estimates", presumably meaning its confidence intervals, is also rather dubious, as these "error estimates" themselves rest on unclear assumptions and have been built from methods and code that have been kept secret from other researchers. And even then they only account for a portion of the potential error that exists in such a survey, as even a letter in the Lancet journal from a prior co-author of the first Lancet study indicated.
It is indeed hard to explain complex situations to "audiences who may not be very familiar with the details", but this is surely made more difficult when some from that audience demand that the 91热爆 report false information to further confuse and mislead that audience.
It's extremely difficult to report a complex situation in a way which is easily understood but not over-simplified to the point of being misleading. Probably impossible, in fact.
Nonetheless, the fact that you are clearly thinking about it and doing your best is one of the reasons why I love the 91热爆. Your competitors in the world of print media wouldn't even bother with such niceties, and would just think "What makes the most sensational story?"
In order to give more space to convey pertinanent information, I think removing the "infotainment" content would be a wise thing to do. I know that the 91热爆 feels it needs to compete for readers/viewers against the commercial infotainers such as Sky, CNN etc, but on it's present course it is in danger of sinking below the waves and merging itself in the meaninless eye candy of the "shock and hold" tactics employed in their world.
A better strategy is to just relay "dry" accurate information, without opinion and make the assumption that the audience are intelligent, and can form their own opinions.
Yeps - lots of people will switch over - but that'll be their loss. The 91热爆, along with other state broadcasters have the advantage of not relying on corporate advertisments - or corporate whim - to exist. Use this advantage and I think in the longer term trust and valued service will raise the 91热爆 back to where it should be - head and shoulders above the entertainment industry that are the commercial news providers.
In that respect, state broadcasters have a duty to be a banner holder, a standard keeper to help stop the commercial newsproviders from descending too far into the mire.
Our own state broadcaster DR, here in Denmark, has been trusted implicitly, because it does exactly what I've outlined above - if you look at trust rankings, it ranks first in the world, and unsurprisingly so.
It does this by an uncompromising, draconian policy of factual reporting - eg. just recently one of their reporters sent in a piece showing the withdraw of Danish soldiers - three days before it actually happened. The reporter who faked the report, one Jeppe Nybroe, has been suspended for 3 months.
Everybody realises just how difficult it is to record actual events in Iraq, and indeed generally in troublespots around the world, but Jeppe fell victim to his desire to entertain with pictures and to be first, rather than what he should of done and informed with facts.
With the recent "Queen storming out" etc. I think this could be a wise strategy to embrace - inform only - leave entertainment to entertainers.
Time may well be an issue for radio, but when 91热爆 News 24 runs the same four of five stories for hours on end, seven days a week you have to wonder why they can't find the time to explain things in full. Besides, how many people really aren't aware of the situation in Iraq?
Perhaps your viewers are slightly more wised-up than you give them credit for?
Quote martin42:
"But making oneself familiar with the IBC's sources would reveal that it includes quite a lot of non-"western" media sources:
I should have clarified and said "English language sources".
IBC say on their site that...
"We have not made use of Arabic or other non English language sources,
except where these have been published in English. ... It is possible
that our count has excluded some victims as a result. "
They also admit on their site that...
"It is likely that many, if not most civilian casualties will go
unreported by the media. "
Quote martin42:
"But the Lancet was not, and could not be, "an estimate of the civilan dead" at all. The report explicitly states that, "Separation of combatant from non-combatant deaths during interviews was not attempted."
They did not ask since the information would be unreliable and may put the interviewee at risk. However, it's quite clear that most of the deaths have occurred after the main invasion, so we are not talking about Iraqi soldiers. Even if you consider insurgents, do you really think that 650,000 insurgents have been killed?
Quote martin42:
"Then it is claimed Lancet "has widespread academic support", but this is also, at best, a half-truth. It's been criticized heavily, and continues to be, by researchers across the spectrum of the academic community."
Who are the critics? Please provide a list. Here are some of the supporters.
First of all, Ministry of Defence Chief Scientific Adviser, Sir Roy Anderson, stated: "The study design is robust and employs methods that are regarded as close to "best practice" in this area. "
Here are just a few of the academics who support the report:
Professor James A Angus, dean, faculty of medicine, dentistry and health sciences, University of Melbourne.
Professor Bruce Armstrong AM, director of research, Sydney Cancer Centre; professor of public health and medical foundation fellow, University of Sydney
Dr Jim Black, head of epidemiology, Victorian Infectious Diseases Service
Professor Peter Brooks, executive dean, faculty of health sciences, University of Queensland
Professor Jonathan Carapetis, director, Menzies School of Health Research, Darwin
Dr Ben Coghlan, medical epidemiologist, Centre for International Health, Burnet Institute
Professor Mike Daube, professor of health policy, Curtin University
Associate Professor Peter Deutschmann, executive director, Australian International Health Institute, University of Melbourne
Associate Professor Trevor Duke, Centre for International Child Health, department of pediatrics, University of Melbourne
Professor Adele Green AC, deputy director, Queensland Institute of Medical Research
Associate Professor Heath Kelly, head, epidemiology unit, Victorian Infectious Diseases Reference Laboratory
Professor Stephen Leeder AO, co-director, Menzies Centre for Health Policy; professor of public health and community medicine, University of Sydney; chairman, Policy and Advocacy Group, Australasian Faculty of Public Health Medicine
Professor Alan Lopez, head, School of Population Health; professor of medical statistics and population health, University of Queensland
Professor John Mathews AM, professorial fellow, School of Population Health, University of Melbourne
Professor A. J. McMichael, director, National Centre for Epidemiology and Population Health, ANU
Dr Cathy Mead PSM, president, Public Health Association of Australia, Canberra
Professor Rob Moodie, chief executive, Victorian Health Promotion Foundation
Professor Kim Mulholland, infectious disease epidemiology unit, London School of Hygiene and Tropical Medicine, UK
Professor Terry Nolan, head, School of Population Health, Melbourne University
Associate Professor Tilman Ruff, Nossal Institute for Global Health, University of Melbourne; president, Medical Association for Prevention of War
Associate Professor Peter Sainsbury, school of public health, University of Sydney
Dr Tony Stewart, medical epidemiologist, Centre for International Health, Burnet Institute
Professor Richard Taylor, professor of international health; head, division of international and indigenous health, School of Population Health, University of Queensland; director, Australian Centre for International and Tropical Health and Nutrition.
Associate Professor Mike Toole, head, Centre for International Health, Burnet Institute
Associate Professor Paul J. Torzillo AM, University of Sydney; senior respiratory physician, Royal Prince Alfred Hospital, Sydney; clinical director, respiratory and critical care services, Central Sydney Area Health Service.
Dr Sue Wareham OAM, immediate past president, Medical Association for Prevention of War, Canberra
Professor Anthony Zwi, School of Public Health and Community Medicine, associate dean (international), faculty of medicine, NSW University
It's also worth noting that the same methodologies have been used by the same authors (and others) in other war zones for many years.
Quote martin42:
"And the notion that Lancet provides some kind of firm "errors estimates", presumably meaning its confidence intervals, is also rather dubious, as these "error estimates" themselves rest on unclear assumptions and have been built from methods and code that have been kept secret from other researchers."
What assumptions are you talking about? The errors come out the statistics of the cluster analysis. And what do you mean by "methods and code that have been kept secret from other researchers". What information are the authors keeping secret? Please specify.
'I should have clarified and said "English language sources"'
English language sources such as:
Al-Zaman
Al-Alam TV
Al Arabiya TV
Al-Furat
Al-Ittihad
Al Jazeera (Web)
Al Jazeera TV
Al Sharqiyah TV
..etc. etc.
Perhaps in addition to not knowing what is 'Western' you don't really know what is available in English either, and shouldn't say anything.
"However, it's quite clear that most of the deaths have occurred after the main invasion, so we are not talking about Iraqi soldiers."
There continue to be Iraqi soldiers and they continue to engage in fighting and get killed Steve.
"Even if you consider insurgents, do you really think that 650,000 insurgents have been killed?"
No, I don't think 650,000 anything have been killed. But a portion of whatever number the approach used in Lancet spits out is soldiers, militia, insurgents.
"Who are the critics? Please provide a list."
Name-droppings of anyone with some letters afterward who has ever said a kind word about the Lancet study are not very enlightening or useful. Nor would it be for critics. Most that you list refer to one-liners or form letters that stem from the day or week the study was published. Mostly meaningless slogan stuff, not real analysis.
Critics from epidemiologists and demographers to conflict researchers of many fields are named elsewhere (many of them even study the Iraq war, unlike most of the names in your list). You can find them on wiki or google if you want names. But while you're at it, make sure to read past the names to what is actually said. One good piece of criticism is here:
"What assumptions are you talking about? The errors come out the statistics of the cluster analysis."
Different researchers using different methods and programs will build different CI's based on the approach they chose. The same data can produce many different CI's depending on a host of choices by those building them. That you seem oblivious to this makes my point, and confirms the need to say it.
Also, read the letter I mentioned (co-authored by Richard Garfield in the Lancet journal). The CI's in Lancet make no accounting of reporting errors in their data. As such, one huge assumption built into them is that everything reported to them from this field research is 100% accurate. Since nobody has checked, or can check, the claims in this data, that's an assumption, a leap of faith in both their interviewers and their respondents.
Credible studies tend not to take leaps of faith in either, but use means of verifying what their teams report, such as repeat interviews by other parties to check the data. This is pretty standard even with non-controversial studies where everyone involved does not have such obvious incentives to distort the data based on the outcome they hope the estimates will have or to take shortcuts for safety or other reasons. Particularly with such a politically hot type of study it's best not to make it as easy as possible for the participants to take shortcuts or manipulate things. But that is what the authors of the Lancet did. Competent and un-biased researchers would not do so.
"And what do you mean by "methods and code that have been kept secret from other researchers."
Meaning the exact manner of their sampling scheme and the approach they used to build their CI's, among other things, have been kept secret from researchers so that nobody can replicate what they did or verify what they did, only guess and try other assumptions and see if they come close. Sometimes they come close. Other times they do not. Which approach is best is debatable. My guess is that the choices made in Lancet were whichever ones produced the narrowest CI (and which is why they keep them secret).
"What information are the authors keeping secret? Please specify."
Just a few of the things they are keeping secret are:
1) The towns or cities they sampled - preventing other researchers from knowing how they went about picking the locations, whether this was a good approach to begin with or whether the locations they sampled are themselves biased or have any hope of being representative.
2) The streets they sampled - preventing other researchers from finding out how much their "main street" shortcut scheme for picking houses biased the estimates.
3) The demographic data on the households they sampled - preventing other researchers from finding out if their sample is demographically representative of the country, or if it is also biased and unrepresentative. IOW, did they sample Sunni, Shiia, Kurd appropriately? Male/female? Young/old? These can all make big differences, and it's all secret.
4) The households they sampled - preventing other researchers from finding out if households were actually sampled at all, or if sampled, whether they were actually selected randomly as alleged, if the sampling scheme was biased, or worse, if they were cherry-picked because those households were known to have suffered war deaths.
5) The code and assumptions used to build their CI's - preventing other researchers from seeing if these have any credibility and if they can be replicated.
All are unknowns, and all can mean the estimates are entirely wrong. It could be argued that perhaps keeping one or two (but not all) of these things secret is justified for "security reasons", but whether reasonable or not, the inability for anyone to know any of these things severely weaken the reliability of the results, which essentially amount to assertion.
It is quite clever, how you slant your article about the Arab gunman who grabbed an Israeli soldier's gun and shot him and others with it before being shot:
1) First, you title the article, "Gunman dies in Jerusalem shootout", thus putting the FOCUS of the article on the fate of the one Arab ASSAILANT, rather than all the Israeli victims. It would have been too "objective", I suppose, to have titled the article: "9 injured by Arab gunman in Jerusalem".
2) You blithely insert a tidbit of "history" as follows: "East Jerusalem has been occupied by Israel since 1967". How convenient that you completely decontextualize this little fact. Apparently, mentioning that it was taken during an unprovoked war initiated by multiple Arab countries is too space-consuming to mention in your little background.
3) You further write, "Israel's annexation of the city is not recognised by the international community", as if it were simply an incontravertible fact. "The international community views the UK as a racist, colonizing has-been" is equally true, especially if one is free to endorse whichever "international community" one chooses. In fact, the latter opinion is probably held by more people than the former.
4) Your article closes with, "A prominent Palestinian politician, Mustafa Barghouti, condemned the killing as a crime carried out by a Jewish extremist on occupied Palestinian land". That you close your article with such a blatantly one-sided--not to mention IRRATIONAL--quote (the gunman had already been identified as an Arab, not a Jew) is shocking, but not surprising, I suppose, given how you handled this story from the start. Still, it's a nice rhetorical device to end with only half a perspective; it's much more effective technique in evoking uncritical, one-sided opinions.
5) Are your contributors (I won't call them "journalists") trained in journalism?
6) When did Al Jazeera purchase the 91热爆?
Max