1. CESAR WORKING DOCUMENT SERIES
Working document no.10
Workshop Urban Strategy Cartesiusdriehoek
Quantitative and Qualitative findings from a workshop on March 11, 2014
Peter Pelzer, Ralph Klerkx, Basten Kolthof
13 May 2014
This working document series is a joint initiative of the University of Amsterdam, Utrecht University, Wageningen University and
Research centre and TNO
The research that is presented in this series is financed by the NWO program on Sustainable Accessibility of the Randstad:
http://www.nwo.nl/nwohome.nsf/pages/nwoa_79vlym_eng

2. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
1. Introduction
This working document reports on a workshop conducted with the Planning Support System (PSS)
Urban Strategy on March 11 2014 in Utrecht, the Netherlands. The workshop was organized by the
TNO, the Municipality of Utrecht and Utrecht University. It focused on the redevelopment of the
Cartesiusdriehoek, a redevelopment area close to the center of Utrecht. The following pages will
first describe the tool, the case and workshop. Next it will elaborate on the results of three research
methods applied to study the workshop (evaluation, survey, observation). This will be done to
answer the research question:
How do practitioners perceive working with Urban Strategy in a workshop setting and what lessons
can be learned for the future?
2. The PSS: Urban Strategy1
The Dutch research firm TNO started around 2005 with the development of the PSS Urban Strategy,
aiming to overcome the existing flexibility bottlenecks and communication bottlenecks. Urban
Strategy aims to improve complex spatial planning processes on the urban- and regional level. To
do this, different computer models are linked to a central database and interface to provide
insights in a wide area of urban indicators and maps. The effects of interventions in infrastructure,
land use, build objects and their functions can be calculated and visualized. Because the PSS is able
to calculate fast and present the results in 1D, 2D and 3D visualisation this can be used in
interactive sessions with planning actors.
Starting point for Urban Strategy is the use of existing state-of-the-art and legally accepted models
in the Netherlands. To link these existing models a number of new elements were developed:
- a database with an uniform datamodel;
- interfaces that show a 3D image of the modeled situation, indicators and that offer functionality
to add interventions;
- a framework that structures the communication between the models and the interfaces.
The goal of Urban Strategy is to enable planning actors in workshop sessions to communicate their
ideas and strategies to the PSS and to learn from the effects that are shown. This interactivity calls
for fast calculations of all the model and fast communication between all elements. For this, the
models were enabled to respond on events (urban interventions from the participants in the
workshops). A new software architecture was developed to have all these elements communicate
1 This section is adapted from: Te Brömmelstroet (2013)

3. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 3
(figure 2).Because of this increased speed and the wide variety of models that are linked together,
the PSS aims to be highly flexible in offering answers to a large number of questions that a group of
urban planning actors can have.
Figure 1 Schematic overview of communication architecture of Urban Strategy.
The 3D interface generates, based on objects in the database, a 3D digital marquette of the urban
environment. To this, different information layers can be added, such as air quality contours, noise
contours and groundwater levels. Also, the objects can be colored according to their characteristics
(function, energy use, CO2 emissions, number of inhabitants, etc). The 2D interface can be used by
the end user (or chauffeur) to add changes to the database. Objects can be added or removed,
their location can be changed and the characteristics of the object can be changed. The 1D
interface shows indicators that are calculated by all the models that are included. Examples are the
percentage of noise hindrance, group risk in an area or the contribution of types of objects to CO2
emission.
Figure 2 The three interfaces of Urban Strategy

4. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 4
3. Case description: Cartesiusdriehoek
The plan area of the Cartesiusdriehoek is bordered by two railway tracks, giving it a triangle-like
shape (see Figure 1). Traditionally, the area had a primarily industrial function, with some
residential zoning. The last years it has become an explicit aim of the Municipality of Utrecht to
make the area more diverse in terms of functions. The main arterial in the area is the Cartesiusweg,
which cuts the area in two. East of the Cartesiusweg is the actual ‘Cartesiusdriehoek’. This is a so
called ‘transformation’ area, where the Municipality of Utrecht aims to encourage different
functions, such as housing, amenities and commercial functions. The area west of the Cartesiusweg
(‘Werkspoorkwartier’) contains industry and is bordered by a large industrial area (Lage Weide).
Therefore, no dwellings are planned in this area, the main aim is to attract small commercial firms.
Figure 3: the plan area of the Cartesiusdriehoek
Three main challenges and related interventions were discussed during the workshop:
1.) The Muncipality of Utrecht is planning to reduce to capacity of the Cartesiusweg
significantly, because of livability reasons (also beyond the scope of the plan area).
Currently the Cartesiusweg is 2 by 2 lanes, but in the future it will be 2 by 1 lane. In the
session it was discussed and analyzed what this change would mean for the plan area of the
Cartesiusdriehoek in terms of traffic flows and environmental impact.
2.) The aim of the Municipality of Utrecht is to significantly increase the density in the
Cartesiusdriehoek (particularly east of the Cartesiusweg), in the workshop the aim was on
CARTESIUSDRIEHOEK

5. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 5
the one hand to get more insight into what impact this would have for the area in terms of
traffic flows and related environmental impact (noise, air quality). On the other hand, the
stakeholders were also interested in the possibilities and the constraints of the plan area.
Because the Cartesiusdriehoek is adjacent to a rail track, building too close to it is legally
not allowed because of environmental reasons (noise, safety).
3.) Explore a range of mitigating measures for the area, such as the construction of walls
against noise pollution.
4. The workshop
The workshop was conducted on March 11, 2014 at TNO in Utrecht. It lasted for three hours, and
after the workshop a group evaluation was conducted. In addition, the workshop had the following
characteristics:
 The session took place in TNO’s workshop room (see Figure 4) in including a large table,
several screens and a touch table. For a brainstorm in two groups a separate room was
available.
Figure 4 Overview and pictures of the workshop setting

6. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 6
 Besides Urban Strategy, several other support tools were available: large paper maps, a
whiteboard, a touch table, and internet connected computers with large screens to access
Google Streetview and the website of the Municipality of Utrecht, which also has an
extensive information base. The support tools had a twofold function, gather knowledge of
the involved stakeholders (whiteboard, paper maps) and provide information to the
stakeholders (Urban Strategy, web browsers).
 The involved stakeholders (n=9) were all part of the area team for the Cartesiusdriehoek of
the Municipality of Utrecht. Central roles were fulfilled by the ‘area manager’ (leading the
meetings) and the ‘area secretary’ (facilitating internal and external communication). In
addition, the area team included several disciplinary specialists, including: an
environmental analyst, a transport planner, a housing specialist and an urban designer.
 The agenda of the session consisted of two main stages. The first 45 minutes were used to
explore to challenges and opportunities in two groups. This resulted in a list of problems
and solutions which were written down on a whiteboard. Next a selection was made of the
topics that could be further analyzed with help of Urban Strategy. The remainder of the
workshop focused on iteratively analyzing the problems that were raised in the brainstorm.
During this process some new issues evolved, which were then analyzed.
5. Research Methods
The following methods were applied to study the workshop: a survey, observations and a group
evaluation.

7. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 7
 The survey focused on four dimensions: background characteristics of the participants, the
focus of the workshop, the usability of the PSS, and the added value of the workshop. It
mainly consisted of Likert items and scales on a 1 to 7 scale (1= strongly disagree, 7 strongly
agree). However, stakeholders were also asked to identify one main added value from a list
of seven. A recent paper by Pelzer et al. (forthcoming) provided important input in
structuring the questions of the survey. The survey started with some open background
questions and ended with open questions about the session.
 The workshop was observed in two ways. First, two external researchers observed the
whole session and made notes. Second, the whole workshop was recorded with cameras
and audio devices. All that was said during the session has been described verbatim.
 After the session, an evaluation was conducted with all the participants, in which they
reflected on the workshop and Urban Strategy. In order to let the participants speak
frankly, the chauffeur and moderator were absent during this session. In addition, a
feedback interview was conducted with the contact person at the Municipality of Utrecht.
6. Findings
Since it was an interdisciplinary team, the background of the users varied significantly. As a
consequence the experience of the workshop also differs among the stakeholders. This is for
instance exemplified by relatively high standard deviations for almost all statements and scales.
Moreover, during the evaluation it was mentioned that Urban Strategy facilitated some disciplines
better than others (in particular: transport planning, environmental analysis), hereby some
participants acknowledged that they find it insightful to learn more about the discipline of their
colleagues. Many of the stakeholders had a focus on financial and economic aspects, something
that is only limitedly, through costs, covered by Urban Strategy.
In Table 1 and Figure 5 usability scores are depicted. All but one dimension score above the neutral
score of 4 (neither agree nor disagree). In addition, other things can be noted from this table.
Flexibility has the highest score and a relatively low standard deviation, implying that most of the
stakeholders agreed on this topic. Interactivity and calculation time, which relate to this dimension
also have relatively positive scores, implying this was not considered a problem. Transparency and
level of detail both have a relatively high standard deviation of around two, implying that the
attitudes are mixed. This implies that there does not seem a uniform way of presenting data and
model outputs. Finally, the score for user friendliness is remarkably low (3.00). In the background

8. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 8
interview, it was indicated that this could be related to the fact that Urban Strategy can only by
used through the assistance of a chauffeur.
Table 1 and Figure 5: Scores of Urban Strategy on a range of usability issues (n=9)
Dimension Score (1-7) SD
Flexibility 5.67 0.5
Interactivity 5.22 1.3
Calculation time 5.22 1.79
Communicative Value 4.89 1.05
Integrality 4.78 1.48
Reliability 4.78 1.39
Transparency 4.56 1.94
Level of detail 4.33 2.06
Data quality 4.11 1.27
User friendliness 3.00 1.41
The added value2
was measured in two ways. First, it was asked what participants considered the
main added value of Urban Strategy (Figure 6). As the figure shows, more than half of the
respondents indicated that the added value of Urban Strategy is related to the planning object and
the outcome (Learning about the object and Better informed outcome). This underlines the
emphasis of Urban Strategy as being an integral or comprehensive tool.
2 Note that technically it is not correct to speak of ‘added’ value, because there was no control group. We use this term to be consistent
with the debate.

9. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 9
Figure 6: Main added value of Urban Strategy as identified by the participants (n=9)
Second, Likert statements were used to evaluate the added value of the whole workshop (Table 2
and Figure 7). Again, standard deviations are high, so some caution is required when interpreting
the patterns. All dimensions again score positively and the spread among the dimensions is
limited3
. It is interesting to note that in some instances the impression of the support staff differs
significantly from the scores of the participants. For instance, the observant had expected a much
higher score for efficiency whereas the chauffeur and the moderator had expected lower scores for
communication. This underlines that to grasp the demands of users it is critical to be in a
continuous dialogue with the participants.
Table 2 and Figure 7: Scores of the participants and expectations of the support staff for different dimensions of added
value.
Dimension
Participants
(n=9) SD
Cronbach's
Alpha* Observant Δ ** Chauffeur Δ Moderator Δ
33. Collaboration 5.15 1.23 0.85 4.3333 0.81 4.00 1.15 5 0.15
32. Communication 4.93 1.35 0.78 5.00 -0.07 3.60 1.33 4 0.93
30. Learning about others 4.54 1.18 0.73 3.6667 0.87 4.00 0.54 5 -0.46
35. Efficiency 4.30 1.12 0.73 5.6667 -1.37 4.33 -0.03 5 -0.70
34. Consensus 4.29 1.53 0.82 4.00 0.29 4.67 -0.38 5 -0.71
* The dimensions 'Learning about the object' and 'More informed outcome' are left out, because Cronbach's Alpha was lower than 0.5.
** Negative is overestimation, positive underestimation.
3 Unfortunately, likely because of the low n, Cronbach’s Alpha of Learning about the planning object and Better informed outcome was
too low to include in the analysis. Therefore, figure 6 and figure 7 are hard to compare.

10. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 10
The group evaluation provided a qualitative insight into the added value of the session. It was
broadly acknowledged that Urban Strategy has potential to improve planning processes. However,
it could have facilitated the session more optimally, now the tool was to a certain extent a barrier
for collaboration and communication, because the interaction was with the tool and the chauffeur,
not with each other. Hereby it should be noted that this lack of communication is also related to
the fact that the participants had to get used to the tool and had a very different starting
knowledge base, in a follow-up session the group interaction would likely be better. Moreover,
some of the participants emphasized that it was not very clear to them what the purpose of the
session was. It had a very different format from the regular meetings. Hereby a complex issue was
the existing knowledge of the involved stakeholders; it was not always evident what important
information Urban Strategy added to the already existing knowledge. An important finding from
the observations is that the project manager played a dominant role during the session. He had a
strong influence on the focus of the workshops and the issues that were handled with Urban
Strategy. This might have influenced the experience of the workshop by the participants.
7. Reflections, Conclusions and Recommendations
An important reflection is that for all but one of the participants it was the first time they worked
with Urban Strategy. In the evaluation it was emphasized that the learning curve for Urban Strategy

11. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 11
is rather steep in a first workshop, a second workshop would probably lead to a more fruitful
experience. From a methodological point of view, it should be noted that the findings above should
be treated with some caution, particularly the surveys. The n (9) is very low and – to a certain
extent as a consequence – standard deviations are high and some of Cronbach’s Alpha’s low.
A first important conclusion is the low score for user friendliness. This could on the one hand be
explained by the rather complex and detailed visualizations that are depicted on the screens and on
the other hand by the fact that a chauffeur is needed to conduct calculations in Urban Strategy.
Hereby, the waiting times before the results were visible seemed on the long side, as was also
emphasized in the evaluation, although this was not considered a problem in the survey. Perhaps
the most important conclusion is that the process management of the workshop is at least as
important as the characteristics of the support instrument. It seems that Urban Strategy lead to a
focus on the support process, rather than a communication process; the latter being pivotal for an
interactive workshop. Some caution should be taken with generalizing these findings, however, for
more technically focused sessions a support process might be more important than a
communication process.
Moreover, it was emphasized during the evaluation and a feedback interview that Urban Strategy
should be applied selectively rather than continuously. Another aspect with regards to the process
is hierarchy and power relations. We found that most of the questions directed at Urban Strategy
only came from a few stakeholders, rather than resulted from a group consensus.
Based on the findings from the workshop several recommendations could be done for future
workshops with Urban Strategy in particular and possibly PSS general. First, it is critical to be aware
of the scale of the planning issue. The impact of the planning interventions done in the
Cartesiusdriehoek hardly resulted in visible changes related to traffic. Only when large scale
interventions where proposed (e.g. ‘add 7000 dwellings), this resulted in visible changes. It seems
that the traffic model is more relevant at the city scale than the neighborhood scale. Conversely,
running a session for a neighborhood without an interactive traffic model, but a fixed scenario
would reduce calculation time and complexity. Moreover, interventions on the neighborhood scale
(e.g. sound walls) lead to visible changes in terms of environmental impact (e.g. noise contours).
Second, it is recommendable to think very carefully about the role of Urban Strategy in future
workshops, both in terms of physical set up (less screens and detail) and the agenda (also focus on
dimensions that are not in Urban Strategy). A finding from this workshop is that unless there is a
dedicated question that has to be answered by the tool, Urban Strategy should stay in the

12. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 12
background as much as possible, because it might hamper the primary communication process
among stakeholders. However, whether this conclusion is representative for other workshops and
other cont Thirdly and finally, future sessions with Urban Strategy should have a more fixed
agenda. It is an illusion to have a completely interactive and open session with a planning support
tool. Knowing most of the issues on forehand leads to a quicker and more focused process.
References
Pelzer, Peter, Stan Geertman, Rob van der Heijden and Etiënne Rouwette (forthcoming) ‘The Added
Value of Planning Support Systems: a Practitioners’ perspective.’ Submitted to Computers
Environment and Urban Systems.
Te Brömmelstroet, Marco (2013) ‘Urban Strategy to support group learning: Randomized control
trial no.4’ CESAR working document series no 6.

13. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 13
Annex 1: Questionnaire conducted after the workshop

14. CESAR Working Document Series no. 10 Urban Strategy Workshop Cartesiusdriehoek
Page 14