Analysis
1: Verification of the data base of the expert opinions
Range
of issues
The
four experts agree that scientific work alone can be
the basis for risk assessment. Moreover, the experts
only used studies that are published and thus professionally
reviewed for compliance with scientific quality standards.
Nevertheless, it should be examined, however, to what
extent the experts base their work on the same studies
or whether here already significant differences occur.
A first step of the scientific dialogue is therefore
the verification of the scientific studies selected.
Iif great discrepancies already occur in selecting the
studies, the experts' conclusions already rely on a
different data base, signifying that for this reason
alone they may come to different assessments.
Procedure
The starting point of the evaluation was the literature
lists of the expert opinions. The Ecolog-Institut cited
113 studies1, Prof. Glaser (HU Berlin) cited 112 studies,
the Öko-Institut 121 studies and Prof. Silny (RWTH
Aachen) 123 studies. These literature lists of the experts
were transferred into a database (Excel). The different
studies (320 in total1, 2) were listed in the lines,
and the experts citing them were identified in four
columns (see Table 1 illustrating the procedure).
| |
Ecolog |
Glaser |
Öko Institut |
Silny |
| Study
1 |
X |
X |
|
|
| Study
2 |
X |
X |
X |
|
| Study
... |
|
|
|
|
| Study n |
|
|
X |
X |
Table
1: Structure of the database
However,
this procedure contains uncertainties. Thus, for example,
studies by the same authors with almost identical titles
are found in different journals. Without a detailed
contents review it cannot be finally ascertained whether
the contents are actually the same. The following evaluations
should therefore be interpreted with care.
Results
Table 2 shows the agreements between the experts.
| Intersection |
Number of studies |
Percent |
| Cited
by only one of the four experts |
232 |
73
% |
| Cited
by two of the four experts |
43 |
13
% |
| Cited
by three of the four experts |
29 |
9
% |
| Cited
by all four experts |
16 |
5
% |
| |
|
|
| All
studies |
320 |
100% |
Table
2: Agreements of the experts
The
result indicates considerable differences between the
scientific studies used: only 5 % or 16 out of 320 studies
are found in all four expert opinions. 73 % of all the
studies used are only taken into account by one expert.
Even if it is assumed that there are hidden agreements
(wrong citations, duplicate publication of identical
contents in different journals) and if the common intersection
is doubled for reasons of care, the literature used
by all experts is only 10 % of the total scientific
literature used .
A slightly more positive picture with respect to agreement
- which, however, still gives rise to concern - is obtained
if the number of studies used in the respective expert
opinion is taken as the basis for percentage agreement
instead of the total number of scientific studies used
(N = 320). In this approach, the percentage of studies
jointly used in all four expert opinions is roughly
15 %.
Figure 1 shows the agreements among the expert opinions
separately for the individual experts.
|