PROGRAMME GROUP HUMANS, ENVIRONMENT, TECHNOLOGY (MUT)
  Start
  Works on EMF
 
  Projects
  EMF Guide
(in German)
  Publications
(in German)
  Links
 
ProjectTaskExpert Opinions Results

Analysis 1: Verification of the data base of the expert opinions

Range of issues
The four experts agree that scientific work alone can be the basis for risk assessment. Moreover, the experts only used studies that are published and thus professionally reviewed for compliance with scientific quality standards.
Nevertheless, it should be examined, however, to what extent the experts base their work on the same studies or whether here already significant differences occur.
A first step of the scientific dialogue is therefore the verification of the scientific studies selected. Iif great discrepancies already occur in selecting the studies, the experts' conclusions already rely on a different data base, signifying that for this reason alone they may come to different assessments.

Procedure
The starting point of the evaluation was the literature lists of the expert opinions. The Ecolog-Institut cited 113 studies1, Prof. Glaser (HU Berlin) cited 112 studies, the Öko-Institut 121 studies and Prof. Silny (RWTH Aachen) 123 studies. These literature lists of the experts were transferred into a database (Excel). The different studies (320 in total1, 2) were listed in the lines, and the experts citing them were identified in four columns (see Table 1 illustrating the procedure).

  Ecolog Glaser Öko Institut Silny
Study 1 X X    
Study 2 X X X  
Study ...        
Study n     X X
Table 1: Structure of the database

However, this procedure contains uncertainties. Thus, for example, studies by the same authors with almost identical titles are found in different journals. Without a detailed contents review it cannot be finally ascertained whether the contents are actually the same. The following evaluations should therefore be interpreted with care.

Results
Table 2 shows the agreements between the experts.

Intersection Number of studies Percent
Cited by only one of the four experts 232 73 %
Cited by two of the four experts 43 13 %
Cited by three of the four experts 29 9 %
Cited by all four experts 16 5 %
     
All studies 320 100%
Table 2: Agreements of the experts

The result indicates considerable differences between the scientific studies used: only 5 % or 16 out of 320 studies are found in all four expert opinions. 73 % of all the studies used are only taken into account by one expert.
Even if it is assumed that there are hidden agreements (wrong citations, duplicate publication of identical contents in different journals) and if the common intersection is doubled for reasons of care, the literature used by all experts is only 10 % of the total scientific literature used .
A slightly more positive picture with respect to agreement - which, however, still gives rise to concern - is obtained if the number of studies used in the respective expert opinion is taken as the basis for percentage agreement instead of the total number of scientific studies used (N = 320). In this approach, the percentage of studies jointly used in all four expert opinions is roughly 15 %.
Figure 1 shows the agreements among the expert opinions separately for the individual experts.

Figure 1: Agreement for the individual expert opinions

Reading aid: example Glaser: 49 % of the literature cited by Glaser is only found in his analysis, 16 % is also cited by another expert, 20 % is also cited by two other experts and 15 % is cited by all experts.

Table 3 shows the intersections between two experts each, irrespective of any other agreements with other experts. The paired comparison of conforming citations shows that there is no "formation of camps", for instance, Glaser and Silny versus Ecolog and Öko-Institut. The hypothesis plausible at first sight that scientific policy differences play a part in the selection of studies is to be rejected.
  Cited in
Ecolog-Insitut
Cited in
Glaser
Cited in
Öko-Insitut
Cited in
Silny
Also cited in
Ecolog-Insitut
- 32% (36) 26% (32) 35% (43)*
Also cited in
Glaser
32% (36) - 26% (32) 37% (46)
Also cited in
Öko-Insitut
28% (32) 29% (32) - 30% (37)
Also cited in
Silny
38% (43)* 41% (46) 31% (37) -
Table 3: Paired agreements (absolute frequencies in brackets)

* Reading aid: 38 % of the studies cited by Ecolog are also cited in Silny, 35 % of the studies cited by Silny are also cited in Ecolog.
The different percentages for the same absolute frequencies result from the different long literature lists of the experts.

Figure 2 shows for the four studies what percentages of the sources cited originate from what time period. For all four studies the majority of the sources cited is to be found in the period 1996-1998. Drastic differences in the distribution over time of the sources taken into account cannot be observed for the studies. Only the Glaser study differs from the other three in that its percentages in the periods 1993-1995 and 1996-1998 are below those of the others, whereas it contains more sources from the period 1980-1984 and earlier.

Figure 2: Percentage of sources according to publication years

Summary
In summary, this evaluation shows that - for the same questions - the experts used different scientific studies to a considerable extent. Of course, this does not provide any information about the quality of the individual expert opinions. However, the result indicates a first hurdle for the process of a comparable and transparent risk assessment concerning electromagnetic fields of mobile telephones: the selection of the scientific studies used for risk assessment. Obviously, there are no uniformly used criteria for the selection of these studies.
This result simultaneously demonstrates how important the planned scientific dialogue between the experts is. In view of the problem pointed out, it would have to be clarified in this dialogue what selection criteria can and should be used.

Further procedure
In a next step we will verify the expert opinions in detail. For each class of biological and health effects it is to be examined whether these have been considered by all experts and whether they refer to the same or to different studies. This makes it also easier to avoid uncertainties of the analysis, i.e. wrongly cited or multiply published studies.


1 In this evaluation, only those studies are taken into account which are specified as essential by the authors themselves in Appendix E. The literature list comprises a total of 231 studies.
2In this evaluation, only those studies are taken into account which Prof. Glaser himself specified as essential in his literature list, which comprises a total of 245 studies.

ProjectTaskExpert Opinions Results
Research Center Jülich Imprint Programme Group Humans, Enviroment, Technology (MUT)

Deutsche Version

23.03.2010
Gisela Degen