The document discusses the impact assessment approach used in the euroFOT project to evaluate the effects of advanced driver assistance systems (ADAS) tested in large-scale field operational tests. The approach analyzes data from 1500 test vehicles across Europe to assess impacts on safety, traffic efficiency, the environment, and user acceptance. A global assessment will scale up the results to estimate impacts across the European Union for different penetration rates of the ADAS technologies. Challenges include integrating multiple assessment methodologies and addressing gaps, as well as handling the large volume of data generated.

1. euroFOT: LARGE SCALE FIELD OPERATIONAL TEST
IMPACT ASSESSMENT
Ahmed Benmimoun1*, Mohamed Benmimoun2, Isabel Wilmink3, Martijn van
Noort3
1. Institut für Kraftfahrfahrzeuge, RWTH Aachen University, Steinbachstr. 7, 52074 Aachen,
Germany, Phone: +49 2418861181, E-Mail: benmimoun@ika.rwth-aachen.de
2. Institut für Kraftfahrfahrzeuge, RWTH Aachen University, Germany
3. TNO Mobility & Logistics, The Netherlands
ABSTRACT
This paper discusses the approach for impact assessment that is applied in the
euroFOT project. The euroFOT project aims to investigate the impacts of advanced
driver assistance systems (ADAS) and to encourage the deployment of these.
Started within the seventh framework programme, the euroFOT project will establish
a comprehensive, technical and socio-economic assessment program for evaluating
the impact of ADAS on safety, environment, traffic efficiency, user acceptance and
other user-related aspects in a real environment. The approach is based on the
FESTA handbook, which provides guidelines for conducting field operational tests
(FOTs). The handbook approach is adapted according to the specific requirements of
the euroFOT project. With this assessment program, the impacts of the ITS equipped
vehicles in the euroFOT test are studied.
KEYWORDS
Field operational test (FOT), impact assessment, FESTA
1

2. INTRODUCTION
euroFOT intends to analyze the impacts of ADAS in a real environment with normal
drivers and for a period of time that enables the collection and processing of data in a
statistically sound way. Extensive field operational tests are being set up to assess
the impacts of ADAS in real traffic, in order to determine in which ways the
effectiveness of ADAS regarding traffic efficiency, safety and environment can be
improved. Altogether 1500 test vehicles from different manufacturers and with
different ADAS take part in the FOT. The project duration is 40 months and will finish
in August 2011.
euroFOT investigates systems that are already on the market or sufficiently mature to
be tested as a commercial application. Based on the recommendations from existing
roadmaps and on the availability of well developed systems, the following group of 9
functions has been selected for euroFOT:
1. Longitudinal functions:
 Adaptive Cruise Control (ACC), Forward Collision Warning (FCW), Speed
Regulation System (SRS).
2. Lateral functions:
 Lane Departure Warning (LDW), Impairment Warning (IW), Blind Spot
Information System (BLIS).
3. Other functions:
 Curve Speed Warning (CSW), Fuel Efficiency Advisor (FEA), Safe Human-
Machine Interaction for Navigation Systems (SafeHMI).
These functions are tested in different vehicles from different European OEMs using
different data acquisition systems for logging different data (CAN-data, video, GPS
etc.). The FOT is carried out via various test sites across 4 countries in Europe,
which are managed by the so called vehicle management centers (VMC).
Challenges of the evaluation
This paper describes the data analysis approach developed within the euroFOT sub-
project “Evaluation, impact assessment, and socio-economic cost benefit analysis”
(SP6). The approach is specified in the “data analysis plan” for each VMC. This plan
covers, besides the assessment of the impacts on traffic efficiency, safety and the
environment, an analysis of user related aspects concerning driver behavior,
2

3. workload and user acceptance. The results of the impact assessment will
subsequently be used in the cost-benefit analysis, which is also part of SP6.
The assessment poses several challenges:
 handling the large amount of data from many different data sources
 designing a complete analysis from data acquisition to the impact
assessment, resulting in input for the cost-benefit analysis. This involves
integrating various existing methodologies plus filling in some remaining
gaps.
This paper focuses on the set up of the impact assessment and illustrates the
proposed approach to deal with the above challenges.
IMPACT ASSESSMENT
The approach will use FESTA as a starting point. Other methods that will be
integrated into the approach were defined in the projects AIDE [1] and eIMPACT [2].
The impact assessment covers the following impacts.
 Impacts on safety: Within the safety assessment subtask, traffic safety
effects, e.g. estimation of expected accident risk reduction, are
investigated, in order to estimate the reduction of crashes, fatalities and
injuries by using the functions.
 Impacts on traffic efficiency: The direct and indirect traffic effects of the
functions, e.g. changes in travel time, speed and the amount of congestion,
are studied. Effects for higher penetration rates can be analysed using
micro-simulation, based on the data from the FOT and the specification of
the functions.
 Impacts on environment: This subtask will determine the effect of the
functions on fuel efficiency and emissions, based on both measurements
and modelling.
The approach of the impact assessment is presented in the following section.
3

4. Approach
The data analysis plan for euroFOT describes the procedural steps involved in
analyzing, in a harmonized way, the massive amount of data to be derived from the
FOT - in order to answer the pre-defined hypotheses and research questions of
interest. Adherence to the data analysis plan will ensure that each partner has a clear
understanding of how to analyze the available data, taking into account all the
common and specific requirements of euroFOT.
The structure of the data analysis plan [3] was derived, to a large extent, from
recommendations made in the EC-funded FESTA project [4]; but for each step of the
data analysis process (e.g. hypothesis formulation, performance indicator
calculation), choices needed to be made to tailor the FESTA method to the specific
needs of euroFOT. The data analysis approach is illustrated in Figure 1. At each
VMC, the data is collected from the test vehicles (e.g. speed and headway
measurements and position at 10 Hz). The relevant data is extracted and stored in
the database. These processes will be managed by the sub-project “Data
management” (SP3). When the data are available in the database, the data analysis
phase can start. At each step, the quality of the data is checked.
Figure 1 – Data analysis approach
4

5. The database contains a huge amount of data – too much to analyse directly.
Therefore, for efficient handling of the data, they are clustered into specific events
(e.g. overtaking manoeuvres, lane changes etc.) and situational variables (e.g.
weather condition, number of lanes, road type etc.). Using the clustered data,
performance indicators (PI), such as average speed or time headways, are
calculated. Each PI is used in the testing of hypotheses that have been specified by
the consortium partners. There are hypotheses relating to the various types of
impacts and user related aspects.
To limit the organisational complexity of the analysis, it has been divided into a VMC
specific and a common part. The common part consists only of the analysis of
functions that are tested by more than one VMC. In this way, the majority of the work
is delegated to the VMCs, but according to a common approach that enables
integration of the results.
The approach described above provides a way to reduce an enormous data set into
a limited number of indicators that enable further analysis. Additionally, in order to
determine unexpected relationships in the large data sets, an exploratory data
analysis (data mining) is conducted. The results are used for the global assessment.
To scale up the impacts of the tested functions the global assessment will be carried
out. Within the quality assessment a review of all performed steps will be applied, in
order to assure that the whole analysis chain process is done properly. Finally the
results will be provided, in order to perform the cost benefit analysis.
By definition, the results of the FOT represent only a small sample of the overall
traffic in Europe. For this reason, simulations of scenarios with higher penetration
rates will be carried out for scaling up locally determined effects within the global
assessment phase. For this, micro-simulation as well as further models will be
applied to estimate direct and indirect traffic and environmental impacts.
The next section describes the global assessment step in more detail.
Global assessment
The objectives of the global assessment are:
 to analyse the effects on the EU level for the functions tested in the FOT on
traffic efficiency, safety and the environment, by considering various
penetration rates (low/medium/high)
5

6.  to provide input for the cost benefit analysis.
The global assessment translates effects found for the equipped fleets and trips
made in the FOT to the EU level. This means scaling up of the effects found in
certain situations, or for certain groups of drivers. This leads to an understanding of
the effects of functions if they would be used in the entire European Union (by a
small or larger part of the drivers). The global assessment needs to provide input for
the cost benefit analysis (CBA). The CBA in euroFOT requires information about the
costs of the functions and the benefits. The benefits are derived from the impact
assessment. The following impacts will be quantified for use in the CBA:
 traffic efficiency:
o direct effects: travel time changes for CBA, but also
homogenisation/reduction of congestion effects for environmental
impact assessment
o indirect effects: changes in the amount of accident related
congestion (based on changes in number of accidents)
 traffic safety:
o changes in the number of accidents with fatalities, injuries (severe,
slight) or material damage only
 environment: changes in fuel consumption and emissions of CO 2
o due to direct traffic efficiency effects
o due to changes in accident related congestion.
There are two major challenges here. One is that previously developed
methodologies need to be combined in a consistent way. The other is that the
resulting methodology still has some gaps. It is worth to address these challenges,
because the FOT data offer a unique opportunity to obtain more reliable results, due
to of the availability of extensive sets of real-world measurements.
The methodologies that will be integrated into the euroFOT approach include:
 the eIMPACT methodologies for safety and traffic impact assessment
 the AIDE methodology for translating changes in certain performance
indicators into changes in accident risk
 data analysis procedures included in the FESTA approach.
6

7. Parts of the eIMPACT and AIDE methodologies are adapted to allow the use of
measured data. The remaining gaps are mainly found in the safety impact
assessment, where a link needs to be created between the effects found in the FOT
data and the change in accident risk. The use of an incident based approach is
among the ideas considered.
Another task is the scaling up of the FOT results. The FOTs are carried out in 4 test
sites across Europe. However, the global assessment looks at the effects on the EU
level. Therefore, the effects found need to be scaled up to the EU level, by taking into
account the different situational variables across Europe, e.g. motorway usage,
prevalence of adverse weather etc.
Furthermore, it has to be considered that the fleet of FOT vehicles constitutes only a
very minor part of the vehicle fleet on the road. For some functions, the effects may
be different for higher penetration rates (and denser traffic) – this is the case, if the
equipped vehicles influence the driving behaviour of other vehicles.
As a consequence of these two issues, the following approaches to obtain the
desired results will be carried out:
 global assessment directly from the FOT-data (scaling up via situational
variables)
 global assessment using models to assess effects for higher penetration
rates (this also includes scaling up via situational variables)
Table 1 shows some properties of these two approaches. An example of a function,
that is analysed via both the direct and the modelling route, is ACC; for a system like
IW however, the modelling route offers no additional insights and this function will
therefore only be analysed via the direct route.
Table 1 - Approaches for the global assessment
Direct route: Modelling Route:
 uses FOT data directly for global  uses FOT data as input for models
assessment (e.g. driving behaviour)
 scales up to EU level via situational  interactions with other (equipped and
variables (as found in FOT-data vs. non-equipped) vehicles can be
EU average prevalence) modelled
 in principle only applicable for  higher penetration rates than found
functions that do not influence on the road in the FOT can be
7

8. behaviour of other vehicles or modelled
functions that cannot be modelled in
micro-simulation models currently
available and suitable for network
analysis. However, when possible,
direct effects will be analyzed, even if
the functions are going to be
modelled
CONCLUSION
Within the euroFOT project, the impact of advanced driver assistance systems is
analyzed using 1500 vehicles equipped with different functions in an FOT. The data
is collected, stored on a database and afterwards analyzed. The analysis task covers
the impact assessment, the user acceptance and usability as well as a socio-
economic cost benefit analysis.
The data analysis plan developed in SP6 of euroFOT provides a comprehensive
approach to the analysis of the FOT data, all the way from data acquisition to impact
assessment. Several challenges have been addressed, most notably the need to
integrate several previously developed assessment methodologies, to fill in missing
links, to handle the large amount of data generated in the FOT and to subdivide the
work in a VMC specific and a common part.
REFERENCES
[1] Jansen, W., Nodari, R., Brouwer, R., et al. (2008). Final assessment of
methods and specification of the AIDE evaluation methodology, AIDE
deliverable 2.1.4.
[2] Malone, K., Wilmink, I., Noecker, G., et al. (2008). Socio-economic impact
assessment of stand-alone and co-operative intelligent vehicle safety systems
(IVSS) in Europe, eIMPACT deliverable D10.
[3] Benmimoun, A., Benmimoun, M., van Noort, M., Wilmink, I, et al. (2009). Data
analysis plan German1-VMC, 2nd draft version.
[4] Lassare, S., Dozza, M., Jamson, S., et al. (2008), Data analysis and
modelling, FESTA deliverable D2.4.
8