As we head into a new summer, I think it is always good to reflect on and learn from what has been. It is also good to remember how our people tried their best to keep communities connected across this wide land in summers past.
Last summer brought with it some of the most devastating and frightening weather events Australians had ever seen, from bushfires to cyclones, storms and flooding.
1. BOOTS & ALL: SUMMER 2010/11
Firsthand accounts of how the Telstra Operations team
prepared for and responded to the summer’s devastating events.
2. CONTENTS
The Australian summer 2010/11 brought with it bushfires,
cyclones, storms and flooding – all of which were typical
events for this time of year. However, combine these with an
unprecedented La Niña event and you have an extraordinary
summer.
Extraordinary times call for extraordinary measures and some
wonderful stories were being told about how the Telstra
Operations team were responding. It is for this reason that a
‘call to action’ was delivered throughout this group, asking the
team to share their experiences.
This book is therefore dedicated to recognising the hard work
and tireless efforts of those within Telstra Operations.
This team is responsible for all aspects of the design,
engineering, architecture, construction and operation of
Telstra networks, technology and information technology, plus
the delivery of customer services across those networks.
It is acknowledged that the company’s response to the
summer events was an enormous Telstra-wide effort and
special thanks to colleagues within the Consumer & Country
Wide, Enterprise & Government, Telstra Business and the
Strategy & Corporate Services business units.
These Telstra groups offered invaluable support to stricken
communities and businesses, both large and small, including
keeping those impacted well-informed concerning progress
during Telstra’s restoration, reparation and reconstruction
phases.
The owner of ‘the boot’ on the front cover of this book is technician Rob White.
Rob is pictured here replacing pillar strips at St Lucia (Qld), 19/01/11.
Boots & All: Summer 2010/11 has been collated and edited by Sandra Mobbs,
Strategy and Corporate Services.
3. CONTENTS
Matt Totterdell and Roger Williams replacing a 400 pair copper cable, Carnarvon (WA). Their team manager at the time advised, “These guys had just got changed out of their wet clothes five minutes before this downpour.”
4. CONTENTS
CONTENTS
Messages from Telstra’s Chairman and Chief Executive Officer 6 PM offers heartfelt thanks to volunteers 192
Salvos emergency volunteers recognised 193
Introduction 10 Rod and the Reject Shop 194
An invitation to an informal event with His Royal Highness Prince William 195
Our approach to storytelling 14
Letter from the Federal Member for Wright 196
Firsthand accounts from the Telstra Operations team 18 Our people’s community contribution recognised 197
Hoping for the best, preparing for the worst 20 Last words from the Telstra Operations leadership team 200
Central East Region (New South Wales, Australian Capital Territory) 30 Note from Strategy and Corporate Services editor, Sandra Mobbs:
South East Region (Victoria, Tasmania) 50 ‘Boots & All’ has captured over a hundred stories, recording an unprecedented
North East Region (Queensland) 92 moment of this company’s history – the summer of 2010/11.The comprehensive
Central West Region (Western Australia, South Australia & Northern Territory) 158 nature of this publication has only been made possible due to the generous spirit of
the Telstra Operations team. Therefore, thank you to our storytellers and subjects
(See lists of all stories, under each of the headings above, on the following pages.)
for giving up their time to share their experiences and providing consent to use
images. My thanks also to Phill Sporton, Executive Director Service Delivery, for
Learning from experience 176 initially commissioning this employee engagement project in March 2011.
Past ideas and innovations
• Raised earth platforms 179
• Mobile Exchange on Wheels or the MEoW® 180
• Cell on Wheels (CoW) and Satellite Cell on Wheels (SatCoW) 180
• TECKs and TREKs 181
Ideas and innovations due to summer 2010/11
• RIM to CMUX conversion plate 182
• New website keeps customers informed during outages and disasters 183
Community, customer and industry recognition of our
people’s efforts 184
CFA says thanks 186
Floods disrupt National Relay Service 187
Queensland department thanks Telstra for speedy response 188
Commendations for quick action 189
A ‘Big Thank You’ from Volunteering Queensland 190
Anglicare warmed by quick and relevant contribution 191
5. CONTENTS
Hoping for the best, preparing for the worst 20 Working in waders John Pridgeon 66
Prior planning and preparation leads to rapid An extraordinary effort Terry Scott 67
network restoration Grant Nicholson 22 A tough couple of months Ian Baker 70
On the ground in bushfire exercise Joe Camilleri 24 The hard yards Daryl Crosbie 72
Unprecedented events tests capabilities Phil Astle 26 Charlton isolation Peter Craig 74
How to prepare for network impact caused Volunteering with the CFA Roger Smith 76
Bridgewater underwater Tony Ryan 77
by a Cat 5 cyclone Pat Rutter 28
Getting stuck in John Hossack 78
Central East Region (New South Wales, Australian Capital Territory) 30 The fastest CoW in the country Ron Wilson 80
Saved from a real dirty job Daryl Beseler 81
Walls of water Grant Carroll 32
Tack’ling the tough stuff John Hewett 82
Riverina-Murray recognition Loretta Willaton 34
Fire and flood John Fixter 84
The return of a slippery customer Con Parslow 35
Doing the nightshift Michael Ennor 85
Lightning storms, floods and road closures Bill Lloyd 36
Just out of harm’s way Brad Crocker 86
Back to business David Gillett 38
Kerang customer rapt Brad Shaw 87
Washed out Terry Walsh 39
What goes around comes around Janet Mackey 88
Call Central weathers the storm David Coxhell and Natalie O’Malley 40
Telstra blokes everywhere Craig Lawry 90
Late night deliveries Terry Lines 42
Record rain and river crossings Matt Shaw 91
Proudest moments Danny Freeman 43
Left-field solutions go a long way Stephen Chadd 44 North East Region (Queensland) 92
Many rivers to cross Peter Chapple 46 Ready for action Malcolm McKellar 94
Find the fibre access point Greg Byrne 48 Flying in to fix fibre Peter Nash 95
Quick fix connects customers Peter Spence 96
South East Region (Victoria, Tasmania) 50
The magnificent seven Shaun Walliss 97
Wet or what? Warren Shean 52 Homeward bound Mark Graham 98
Copping it early Ian Baker, Des Ryan and Trevor Goudie 54 The Army, Police and Dave to
Bit of bother Laurie Barber 56 the rescue David Webb and Craig Bartlett 99
Incident Control Centre support Nick Marotta 58 Exchange wrapped in plastic Steve Strugnell and Shane Golding 100
Sending in the reinforcements Ken Hodgson 59 Toowoomba – an insider’s view Sue Ikin and Dave Marshman 101
A changed vacation Rob Dahllof 60 Flash flooding hits with no notice Peter Sticklen 102
East coast focus for Tassie Glenn Turner 62 Meeting the challenge head-on John Parkin 104
North-west coast focus for Tassie Ian Pickering 64 Keep calm and carry on Craig Bartlett 106
Better safe than sorry Alistair Cowie 65 Staying the course Greg Anderson 108
6. CONTENTS
Brisbane CBD shuts down Lynne Bell 110 Central West Region (Western Australia, South Australia
Above and beyond in Rocky Peter Spence 111 and Northern Territory) 158
Capacity request met in difficult circumstances Neil Francis 112 The Stockport CoW Peter Andreopoulos 160
When your road becomes a boat ramp Susan Kuppens 114 100-year floods Alan Brown 161
Community comes together Shanne Wright 116 What did you do over Christmas 2010? Tim Leahy 164
The Kholo/Mt Crosby ‘break and enter’ incident Peter Leonard 118 Home Sweet Home Mick Cooper 166
Fault frog finds infamy Noel Hand 120 Fires at Lake Clifton Peter Old 167
Queensland Flood Relief Hotline John Tarlinton 121 Great Northern Highway disappears Mick Cooper 168
Coming to terms with Grantham Peter Scherer 122 Snakes, bugs and crocs Neil Cooke 170
Solution-oriented tech takes charge Tym Browne 124 Public support outstanding Shane Caratti 171
What a weekend! Steve Burke 126 The Kimberley cougar strikes again Joe Ganino 172
A surreal experience Heidi Pfeffer 128 Just keep going – a poem Tony Hunter 174
Snakes on joint go viral Pete Milward 129
Who let the CoWs out? Michael Steele and Aaron Kong 130
Pitching in Paul McCarthy 132
Help from across the ditch Malcolm McKellar 134
Oh geez – this is real! Paul Montiford 135
Thunderbird operators are go! Graham Ford 136
Roadside cabinet restoration work Michael Steele 138
MacGyver has nothing on our guys Phillip Stringini 139
My night in Townsville Exchange Wayne Watling 140
Ground Control John Shepherd 142
War Room approach hits the mark Dru Dingwall 143
Ugly but interesting – about our exchanges John Dempster 144
Bill the builder comes to the rescue Ross Auger 148
View from Cardwell tower Chris Frost 150
BAM and it’s gone Mick Young 151
Rick to the rescue Joanne Flood 152
Mark gets up close and personal Mark O’Connell 153
SatCoW flies to Palm Island Clint Dickson 154
Whipping up support Stephen Bowen 156
Stepping up Mark Pettiford 157
9. CONTENTS
A message from Telstra’s Chairman
Keeping communities connected during times of natural disaster demands a huge commitment, and
often personal risk, from our technicians and support staff in the field.
The stories you will read in this book are only a small fraction of what we as a company contributed
to the disaster response and recovery, but they highlight the lengths our people will go to keep our
customers and their communities connected.
Catherine Livingstone: Telstra Chairman
The dedication and commitment of Telstra’s people truly shines during difficult times.
The Board was regularly briefed about the challenges Telstra crews faced and how, in collaboration with
emergency service agencies, they supported local communities.
On behalf of the Board, I acknowledge with deep gratitude the efforts of all those who responded to the
natural disaster events of the summer of 2010/11.
Catherine Livingstone AO
Chairman
Telstra’s Chairman attending a briefing concerning progress by the Northern Flood Recovery team in Brisbane (Qld),
24/01/11.
8
10. CONTENTS
A message from Telstra’s CEO
Summer 2010/11 brought with it some of the most devastating and frightening weather events
Australians had ever seen.
We should never forget that it is our team, the Telstra team, which is one of the first to respond when
a disaster strikes.
Communication is essential during these times and Telstra people play a critical role ensuring communities
David Thodey: Chief Executive Officer
and emergency service organisations stay connected.
This book contains firsthand accounts of how the Telstra Operations team took swift and decisive
action and came up with some innovative solutions to meet the many challenges which presented
themselves along the way.
I would like to extend my sincere thanks to the people involved in preparing for possible network impacts,
monitoring our networks in the aftermath, restoring services, helping customers and supporting their
local communities during the summer 2010/11 peak load season.
We can all be enormously proud of the team’s efforts during these times.
David Thodey
Chief Executive Officer
Allen Brazier, John Parkin, David Thodey, Dave Liddell, Dave Kincaid, Greg Anderson and trainee Jason Lewis
during the CEO’s visit to check in with the troops working at St Lucia (Qld), 25/01/11.
9
13. CONTENTS
Each year Australians deal with natural disasters Ready – Prepare our people, customers, infrastructure Often these reviews spark ideas, ideas which result
where lives, homes and livelihoods are often on and business. in technological innovations and changes which
the line. improve the way we do things.
Respond – Determine the impact on our people,
Keeping connected – something most of us take customers’ infrastructure and businesses. Within these pages we will refer to some of these
for granted during the best of times – is of utmost innovations, which now form part of our Disaster
Restore – Prioritise the restoration of services in co-
importance during the worst of times. Planning (DISPLAN) communication support to
ordination with emergency service organisations.
emergency service organisations, including our
In an emergency, one of the first things we all do
Repair – Develop solutions which restore services Telstra Emergency Communications Kits (TECKs),
is reach for the phone to call for help or to make
as quickly as possible to isolated communities – this Cells on Wheels (CoWs) and Mobile Exchange on
contact with loved ones.
may involve temporary fixes. Wheels (MEoW®). For some facts and figures about
This book concentrates on the events of summer these important pieces of equipment and other
Reconstruct – Work to permanently repair or
2010/11 to provide the reader with firsthand, behind- innovations, under the ‘Learning from experience’
rebuild infrastructure damaged by the event.
the-scenes accounts of how Telstra Operations section of this book (p.176).
team members prepare for and respond to disasters (Note: See breakout box on the following page
The last sections of this book are dedicated to
which have the potential to impact communities and for more details of each phase of the 5R disaster
recognition by the community, customers, industry
their ability to keep these communities connected recovery framework.)
and the Telstra Operations senior leadership team,
during critical times.
After managing through major infrastructure highlighting further how team members went above-
The Telstra Operations team manage major, impacting events, the Telstra Operations team and-beyond what is expected in support of our
customer-impacting incidents in accordance with conducts a Post Implementation Review to gain customers during these times.
their ‘5R’ disaster recovery framework. The 5R knowledge and learn from experience.
phases are as follows.
Above: Road access was a huge issue. Opposite Top: John (Tack) Hewett rows a boat across a customer’s paddock to get to the job (Vic). Opposite Middle: Preparing for flooding in Victoria, Jim Rogers checks out this roadside
12 cabinet which has been well sandbagged. Opposite Middle Lower: Warren Collie working at St. Lucia (Qld). Opposite Bottom: Old rail bridge at Dubbo (NSW).
14. CONTENTS
Breaking down the 5Rs into actions (Depending on the event, these phases may include the following)
• Field workforce prepared (resource planning – techs on the ground)
• Communications plan established (Telstra internal and external, e.g. emergency service
agencies, media, etc.)
• Business Continuity:
- Central Business District (CBD) sites evacuated during floods
- Staff working from home and where necessary moving functions performed by staff at
other locations
Ready
• Protecting the core network:
- Network redundancy tested
- Critical infrastructure protection
- Power and access
- Sandbagging of exchanges, emergency power plants checked and refuelled.
See the ‘Hoping for the best, preparing for the worst’ section (p.20) for more details concerning
how the team gets ready.
When dealing with disasters of the kind we faced this summer, our focus is on the safety of our
people and the restoration of our core network, which is vital to ensure telecommunications
traffic into and out of affected areas, including 000 and to assure communications to emergency
service organisations.
The biggest challenge to keeping people connected during major storms, flooding, cyclones
and bushfires is the loss of mains power. From the outset of this summer’s weather events the
Telstra Operations team worked in direct alignment with power utilities and partner Silcar Energy
Solutions to ensure critical telecommunication sites were repowered.
Where critical mobile sites could not be restarted the team deployed Telstra’s Cells On Wheels
(or CoWs), which are portable mobile base stations. Telstra’s Mobile Exchange on Wheels (or
Respond, MEoW®) was used where critical exchanges went down.
Restore and The respond, restore and repair phases can also include:
Repair
• Telecommunications support to evacuation centres, recovery centres and volunteer
organisations
• Prioritisation of work based on customer needs
• Constant review of power to core network infrastructure to all key network sites
• Constant review to ensure access to sites is safe
• Temporary network restoration
• Temporary repair of damaged infrastructure, deployment of temporary network elements
• Focus on repair or replacement of roadside cabinets, getting customers and businesses
back online.
The final ‘R’ is ‘reconstruct’ and focuses on the team’s program of work to permanently repair or
Reconstruct
rebuild damaged infrastructure.
15. CONTENTS
Byron Griffiths, one of a team of five, who manually dug extremely sticky soil in hot, humid, fly-friendly conditions to locate and fix fibre issues at Florina
Station (NT). See related story, ‘What did you do over Christmas 2010?’ (p.164)
17. CONTENTS
“The main audience for this book are Telstra Operations
team members and their family and friends.”
If one picture could tell the story about the summer Where possible the editor has chosen shots taken
2010/11’s weather events, then this one; sourced during the event which feature Telstra crews at work,
from and used with the kind permission of the to fully capture the essence of the stories being told.
OUR Australian Government Bureau of Meteorology;
would be it.
However, some stories do not have images supplied
of the team members involved, as one construction
APPROACH TO The story this image tells is a simple one. During October
operative candidly put it when asked about photos,
“We were all too bloody busy to pose for photos!”
2010 to end-March 2011 (the definition of the summer
STORYTELLING period for the purposes of this book), approximately
ninety percent of our country received above-average
The stories have been listed by region* and are in date
order as to when the storyteller’s event occurred.
to unprecedented, ‘highest on record’ rainfall, with all
The main focus of the stories supplied here is the
states impacted. However, parts of Western Australia’s
team’s response when communication services and
southern regions remained extremely dry, also a recipe
communities were significantly impacted.
for disaster – bushfires. Parts of Victoria also suffered
bushfire events during this period. Finally, while devastating events happened globally
during this period and in some cases Telstra Operations
Within these pages many tales have been collated with
people were involved (for example the Australia-Japan
three types of stories told: firsthand accounts ‘by’ the
cable assessment following a major 8.9 offshore quake
actual person involved, second-hand accounts ‘as told
which impacted large areas of Japan’s northern Pacific
by’ a team mate about another’s efforts and a question
coast in March), this book has been limited to a national
and answer style ‘interview with’ those involved.
view of summer events.
Concerning all the stories captured here, the authentic *As our country’s regions are often defined differently by different groups,
voice of the storyteller shines through; therefore the telephone ‘area code’ regions have been used for the purposes of this book.
Therefore the regions used within these pages are as follows: Central East
language and tone of the stories vary greatly depending Region (New South Wales, Australian Capital Territory), South East Region
on the storyteller’s style and the situation team members (Victoria, Tasmania), North East Region (Queensland) and Central West
found themselves in at the time. Region (Western Australia, South Australia and Northern Territory).
The main audience for this book are Telstra Operations
team members and their family and friends. Therefore,
for ease of reading, each story within this book is self-
contained with acronyms and terms explained.
All the images contained in this book have been
sourced from team members involved. Many were
captured via their mobile handsets and therefore the
quality and resolution of photographs contained within
the book varies.
16
18. CONTENTS
Above image: Sourced from and used with the kind permission of the Australian Government Bureau of Meteorology. 17
19. CONTENTS
After a break and enter incident, Peter Leonard and Keiron Smith revisited the RIM to capture the scene of the crime.
Read how the pair put in a marathon effort to keep the Kholo/Mt Crosby (Qld) community connected (p.118)
22. CONTENTS
HOPING FOR THE BEST,
PREPARING FOR THE WORST
23. CONTENTS
PROACTIVE... PROACTIVE...
“Work continued non-stop throughout the night and into the next day ...”
Interview with Grant Nicholson, to engage stakeholders early – advising them of
Network & IT Operations network preparations and communicating potential
PRIOR Date: October 2010 onwards network risks and priorities.
The Network Assurance Operations teams meet
PLANNING AND Telstra’s Network Assurance Operations team
based at Telstra’s Global Operations Centre are
regularly to share information concerning the up
coming emergency and to ensure they have rostered
PREPARATION responsible for the control, co-ordination and
communication of high impacting customer
their teams appropriately and have made any
necessary preparations relating to their technology
LEADS TO RAPID incidents and networks at risk. They work closely
with the Telstra Operations groups on the ground
(for example the Voice and Traffic team generally
manually perform backups of the data for each of the
in the affected areas, including determining priorities
NETWORK for preparation and, afterwards, in the restoration
exchanges in the area).
In preparation for both the Queensland floods
efforts.
RESTORATION For this team, early engagement with key
and Cyclone Yasi, exchanges were made as
safe as possible, including removal of debris and
stakeholders for preparation saves a lot of time and sandbagging. Power preparation and refuelling
allows all groups to act effectively and efficiently strategies were put into place for exchanges most
should network impact be experienced. likely to lose power.
Preparations can only go so far, sometimes
Upon becoming aware of an emergency incident,
the unexpected takes place. In the case of the
the team first makes contact with Telstra’s assigned
Queensland floods, a third-party organised a major
Emergency Services Liaison Officers (ESLOs) telethon to raise funds from the community, but
to ensure all preparations are aligned with field did not advise Telstra. The Network Assurance
operatives (including emergency agencies involved Operations team had to think quickly to co-ordinate
and Telstra’s Service Delivery communication extra capacity.
technicians and Network Construction constructor
operatives). Annie Martin, one of the Major Incident
team managers co-ordinating the incident,
Following this, key network infrastructure within commented, “Technical staff were quickly
the area at risk of impact is identified. ‘Go/no go’ gathered together and implemented some
zones (red, amber, green) are constantly reviewed network changes to redirect call traffic using
with the relevant emergency service authorities to alternate technologies.
ensure safe passage for any field staff working in, or
Whilst the changes enabled callers to get through
travelling through, the zones.
and make their donation, it created some other
Early communication with internal stakeholders challenges for how the calls were handled, as the
(groups within Telstra that may also be impacted or centre also takes calls for Queensland SES and
can assist in preparation and/or restoration work) is Health. Work continued non-stop throughout
also very important and ensures business continuity the night and into the next day to address the
plans are put in place in preparation for the coming network capacity issues and return their service
emergency. One of the key steps in preparation is to normal.”
22
24. CONTENTS
At the onset of Severe Tropical Cyclone Yasi, the
Network Assurance Operations team developed a About the role of Emergency Services Liaison Officers
pre-impact priority restoration plan which allowed
Telstra’s Emergency Services Liaison Officers (ESLOs) provide the interface between Telstra and the
them to become a practical part of preparations,
emergency service organisations during an emergency such as flood, fire, cyclone, etc.
monitoring the network closely and working alongside
emergency service organisations and district disaster ESLOs are field managers from Service Delivery and are supported by a state and deputy ESLO.
management groups. During emergencies, ESLOs support the communications requirements of emergency agencies by
Once the cyclone had passed, restoration efforts co-ordinating requests including: restricting staff movement in the affected area by applying ‘no go’ zones
initially focused on protecting the IP core sites, (for safety reasons), the identification and preparation of infrastructure at risk, organising additional products
which was made very difficult due to power being and services as required, prioritising fault management for critical emergency and utility sites, and the
unavailable at a number of exchanges. As sites temporary and long-term restoration of communication services.
became safely accessible, Silcar Energy Solutions,
Service Delivery and Network Construction teams
worked tirelessly to hook up generators and
commence regular refuelling runs to keep the
generators topped up with fuel.
To safely access key sites Telstra goes to great
lengths, for example working with the Australian
Defence Force to fly in via Black Hawk helicopter.
Nick Kellett, one of the Network Operations
Managers working at the time recalls, “We knew
that the mains power would be disrupted for
a number of days so we had to come up with
a plan to keep all the network sites linking the
IP core together working. This meant getting
portable generators out to the sites that just
had batteries and keeping the overall generator
network refuelled.
Given the large geographical area we had to
cover and in some instances the lack of access
due to the massive disruption that the cyclone
left behind, we had to be creative. A couple of
times we were within minutes of losing our IP
core network which would have meant a loss of
all fixed and mobile communications to Telstra
customers in far North Queensland.
We leveraged our great relationships with Silcar
Energy Solutions, Service Delivery and the Australian
Defence Force to come up with a robust refuelling
plan which we implemented and monitored for two
weeks to avoid any sites failing.”
Top Left: Black Hawk at Theodore (Qld), 30/12/10. Top Right: Some of the team involved in the Major Incident room, at Telstra’s Global Operations Centre were (l-r) Grant Nicholson, Joe Camilleri, Soula Duval,
David Cheeseman, Glen Taylor and James Wright (back to camera). Bottom: Telstra’s Global Operations Centre provides 24/7 monitoring across all Telstra networks, including working with emergency services 23
and field groups to establish restoration priorities during emergency incidents.
25. CONTENTS
PROACTIVE... PROACTIVE...
“For Telstra, the exercise was an important
part of our seasonal preparation...”
By Joe Camilleri, Network & IT Operations SMR is used by the Police, CFA and Victorian
Date: 05/12/10 Department of Sustainability and Environment (DSE)
ON THE In December, members of the Telstra Major
in country Victoria as well as a number of other
emergency organisations. It is a vital communication
Incident Management and State Managed
GROUND IN Radio teams participated in the largest test
of Victoria’s bushfire preparation since Black
system in an emergency.
See related story (p.186) ‘CFA says thanks’.
BUSHFIRE Saturday (7 February 2009). Alongside the
Country Fire Authority (CFA) and SES, the exercise
EXERCISE
tested communication, collaboration and processes
between the organisations involved in emergency
situations.
For Telstra, the exercise was an important part of
our seasonal preparation for emergencies, providing
a unique opportunity to test our disaster planning
processes which includes the communications
support Telstra provides to emergency service
organisations to assist them in managing emergency
situations.
Co-ordinated from the CFA’s Kangaroo Ground
Incident Control Centre with a staging area at
Yarrambat Park, members of the Telstra Major Incident
Management and State Managed Radio teams were
there to see firsthand how communication systems
operate on the ground during an emergency.
The Telstra team were on hand to support the
deployment of communication systems and were
able to showcase our emergency communications
capability by deploying a Telstra Emergency
Communications Kit (TECK). The TECK provided
incoming and outgoing PSTN and fax services for
the SES and CFA staff on the ground.
Telstra teams were also able to observe how the
StateNet Mobile Radio Networks (SMR) operated
during the exercise.
24
26. CONTENTS
Top: Comms play an extremely important role in assisting emergency service organisations to manage in a crisis. Here we see the TECK in use during the exercise. Bottom: During December’s bushfire exercise. 25
27. CONTENTS
COLLABORATION...
“Restoration activities were difficult due to the unsafe
conditions and a record number of red zones...”
By Phil Astle, Network & IT Operations state, including Charlotte, Edison and Ipswich,
Date: December 2010 onwards preventing any further adverse impact.
UNPRECEDENTED The Queensland floods and Cyclone Yasi Protecting the core network and ensuring continuity
of service for the community, including emergency
were two unprecedented events that put our
EVENTS TESTS network and capabilities to the test. The other
weather events across other states, whilst they were
and essential services, were our key priorities. The
restoration of local power and site access restrictions
CAPABILITIES significant for the communities, had a comparatively
minimal impact on Telstra infrastructure and
being lifted saw a steady decline in the number of
impacted services.
customers.
The emergency incident management process
was invoked to manage all the natural disasters. Phil Astle: Onsite liaison
The Incident Operations team engaged a number
of teams including Service Delivery, Network When it became apparent the recovery effort
Construction, Silcar Energy Solutions and Strategy & for the Queensland floods was unprecedented,
Corporate Services to co-ordinate asset protection, Phil Astle, emergency and relationship manager
restoration activities and communications. within the Network Assurance Operations
Incident team, relocated from Telstra’s Global
Restoration activities were difficult due to the unsafe Operations Centre (GOC) in Melbourne to
conditions and a record number of red zones (‘no Queensland. Phil’s role was to attend twice daily
go’ zones) declared by emergency services across Regional Emergency Council meetings and
flood and cyclone impacted areas. Take Queensland assist with the management of the technical
for instance, where we had a total of 719 red zones bridge to ensure any issues relating to damage
declared by the end of the season. were quickly identified and escalated to make
Despite the significant customer impact the network sure the recovery effort was not compromised.
withstood the treacherous conditions quite well with
The role of team members at Telstra’s GOC
the major impact being caused by a loss of local
includes 24/7 monitoring and first in fix of
mains power and back-up batteries depleting due
incidents across all Telstra networks, customer
to site access issues.
experience monitoring, incident management,
It is highly probable the impact would have grown event management, emergency services
exponentially if we had lost any of our IP core sites. call tracing and working with field groups to
establish restoration priorities during emergency
With the collaborative efforts of the Telstra
incidents.
Operations teams we were able to successfully
protect a number of major exchanges across the
26
28. CONTENTS
Top Left: Joe Camilleri, Tom Farrell, Phil Astle, Will Visser and Paul Gerreyn at the December bushfire exercise. Top Right: Our field crews’ safety, due to road access issues, was monitored closely at the time.
Bottom Left: The devastation caused by Cyclone Yasi and flooding was shocking. Bottom Right: Restoring power was important. Here a 4WD forklift is used for a genset deployment to a hill top exchange at Cardwell (Qld). 27
27
29. CONTENTS
PROACTIVE... PROACTIVE...
“The preparation prior to Tropical Cyclone Yasi provided benefits in
reducing the impact to Telstra’s infrastructure in the affected areas...”
By Pat Rutter, Service Delivery Post-Cyclone
Date: 01/02/11 onwards
HOW TO PREPARE Telstra’s key learning from Tropical Cyclone
• The immediate three day focus was on power
restoration.
FOR NETWORK Larry (2006) was power would be a significant
issue if Tropical Cyclone Yasi crossed the coast
• A ‘4-Point Recovery Framework’
communications was established to provide
for
IMPACT CAUSED in a populated area in Far North Queensland.
This proved correct as 680 major network sites were
an easy flow of communications between the
four main groups involved at Telstra’s Global
BY A CAT 5
left without power. This required the deployment of Operations Centre, in Brisbane, Cairns and
generators and the co-ordination of power restoration Townsville. Open phone bridges (technical and
with Ergon Energy and Powerlink Queensland.
CYCLONE Pre-Cyclone
operations) were also in operation.
• Power restoration meetings continued at a high
• Core and priority network sites were identified level between Silcar Energy Solutions, Ergon
by the Global Operations Centre / Major Incident Energy and Telstra.
Management in Melbourne and the list reviewed • Restoration efforts including timing and access
by the cross-Telstra business unit team to include were co-ordinated with Ergon Energy and SES.
other known priorities based on local knowledge.
• Daily Major Incident Management status updates
• The Global Operations Centre / Major Incident provided visibility of Telstra zone status (ability to
Management then completed scenario modelling access sites), summary of customer impact, and
on the sites, with Telstra’s partnered contractor a power summary (including the number of sites
for the provision of emergency power, Silcar without mains power, sites restored by Ergon
Energy Solutions, in turn predicting the impact Energy and the number of sites being monitored
and requirements for power at these sites post- and managed by Silcar Energy Solutions for
cyclone. refuelling, batteries, generators, etc.).
• A pre-disaster technical phone bridge was The preparation prior to Tropical Cyclone Yasi
established to assist in planning for potential provided benefits in reducing the impact to Telstra’s
issues identified. infrastructure in the affected areas by pre-planning
the availability of generators for interim power to the
key network sites. Relationships already established
between Telstra, Telstra’s partnered contractor
and Ergon Energy allowed all parties to have a
co-ordinated approach to the restoration of power
to these essential sites.
28
30. CONTENTS
Our top three network priorities: Power! Power! Power!
Like many Telstra partners, Silcar Energy Solutions were only too happy to assist Telstra and their customers
with restoration of services following the devastating and destructive effects of ongoing flooding, including the
aftermath of Cyclone Yasi.
Silcar Energy Solutions deployed resources and material from both Queensland and New South Wales to assist
in around the clock efforts which initially focused on the restoration of the communications network.
Silcar Energy Solutions commitment with the recovery saw personnel based in these regions for in excess of
three weeks. During this time Silcar Energy Solutions’ employees worked 12 hour shifts alongside Silcar field
technicians, Telstra field teams and other agencies also engaged in recovery efforts.
Whilst many involved described the efforts as strenuous and demanding given the destruction caused, they also
said they would be happy to do it all over again.
Trevor Starcevich, Network Construction Contract Manager
Above Top: Craig Bartlett, the Emergency Services Liaison Officer (ESLO) who undertook the massive task of co-ordinating efforts with emergency service organisations for Queensland and Northern New South Wales during
the summer 10/11 period. Above Bottom: David Thodey visiting some of the Northern Region Operations team (l-r) Trudy Deighton, Jay Patel, Chris Hazelton, Tony Broadway, Tim Lostroh, Sym Puskaric, Simon McLean, 29
David Thodey, Shellie Cave, Sandra Perry and Pat Rutter
31. CONTENTS
Greg Cooke at a washout at Lue. See related story ‘Lightning, storms, floods and road closures’ (p.36).
32. CONTENTS
CENTRAL EAST REGION
(NEW SOUTH WALES, AUSTRALIAN CAPITAL TERRITORY)
Note: Due to Service Delivery’s Southern Region extending into parts of New South Wales,
to differentiate between stories from Service Delivery’s Southern and Central Regions
we have noted when a story is from Southern teams in this section only.
33. CONTENTS
COMMUNITY SPIRIT...
“Traffic was cut for about 20 hours and
there was extensive damage done to some
heritage buildings...”
As told by Grant Carroll, Countless SES meetings were attended by the local
Service Delivery [Southern] team manager David Gillett who supplied updates
WALLS OF Date: October 2010 onwards and assistance to customers. After waters receded
to a safe level David and two field technicians were
During the summer the Riverina Snowy ferried into North Wagga sitting in the bucket of a
WATER Field Service Area (FSA) experienced major
severe weather events. On each occasion, large
front-end loader to assess damage.
amounts of rain fell over very short periods, creating The last of these severe weather events for the
walls of water which rushed down mountains, rivers summer period took place in March, when the
and creeks, causing extensive damage to properties, largest amount of rain fell; 300-400 millimetres was
bridges and Telstra infrastructure. recorded over a period of 48 hours in the south
coast area around the Bega Valley.
Natural disaster zones were declared in numerous
local shires within the area, making it one of the Due to the mountainous terrain along the coast,
most challenging years on record. the sheer volume of rain at this time caused more
damage to infrastructure in the region than any of
The first of these events in October saw 100- the other events throughout the year.
200 millimetres of rain fall in a 24-hour period in
the Wagga and Albury areas. The small towns of Due to unsafe driving conditions, our team was not
Culcairn, Adelong, Holbrook, The Rock, Lockhart, allowed to drive their vehicles into the area to affect
Rand, Urana and Jingellic had to be evacuated due repairs.
to widespread flooding and inundation of homes, Our local team managers did however continue
properties and roads. Tragically, during this event to work closely with the SES and due to Peter
we saw the loss of life just outside the township of Chapple’s involvement in the fire brigade, and the
Lockhart. relationships built, Peter was able to arrange for two
The Wagga and surrounding areas were struck field techs to join him on flights into isolated areas by
by torrential rain again in December, causing the helicopter and commence restoration of damaged
Murrumbidgee River to peak at its highest level since fibre crossings.
1974. Towns along the river including Gundagai,
Peter was also able to set up a satellite phone in the
Narrandera and Wagga saw residents forced
small township of Rocky Hall for our customers to
to evacuate from their properties and homes
use until all services could be restored.
including the evacuation of 1,800 people from North
Wagga alone.
32
34. CONTENTS
My sincere thanks and appreciation go out to the
whole team who worked long hours over many
weeks to restore services. They all worked tirelessly
and considered the needs of the people in these
communities and the importance for them to be able
to communicate with family and friends.
About the Adelong area Chris Hargreaves, Service
Delivery [Southern] said, “Telstra’s network was
impacted by a 100-year flood event but we all
pulled together as a team and got the work done,
supplying telecommunications to the local people
as fast as we possibly could.
Traffic was cut for about 20 hours and there
was extensive damage done to some heritage
buildings, but the Adelong Telephone Exchange
remained just out of reach, with the water level
coming within millimetres of the floor level.”
Top Left: What was left of East Grahamstown Bridge, East Grahamstown Road, Adelong after flash flooding.
Top Right: Local team manager David Gillett took this aerial shot while surveying flooded North Wagga area with the local SES crew. Bottom Right: Travelling to Culcairn. 33
35. CONTENTS
RECOGNITION...
“Many of these towns were isolated
geographically as floodwaters
covered roads.”
By Loretta Willaton, Telstra Country Wide
Date: October 2010 onwards
RIVERINA- Late last year and again early this year, With an extra 22 people in his team, Service Delivery
floodwaters savaged the Riverina Murray area field manager Grant Carroll and his team managers
MURRAY washing away trees, which piled up against tirelessly led restoration efforts working long hours
bridges. Many bridges were destroyed. As the and weekends to fix faults as quickly as possible.
RECOGNITION bridges often carry our fibre links, many were broken
and we lost service to several exchanges.
Emergency satellite phones and temporary services
were delivered to several customers who were likely
The excessive flooding and resulting road closures to have longer outages due to major damage at
prevented Telstra Operations service teams from creek crossings.
being able to get straight in there to begin repairs.
Many of these towns were isolated geographically
During the first major impact in November 2010, we as floodwaters covered roads.
experienced problems with a number of our small
Ten extra services were provided to the SES in
rural exchanges due to fibre cables being washed
Tumut, a small town just outside of Wagga, to
away or broken, including one of the major back-up
ensure emergency workers had access to essential
fibres servicing the district.
communication services during the rescue and relief
Whilst all fibre-based services and related exchange operations.
outages were restored relatively quickly, there were
Then again, on 11 February, Wodonga was inundated
also huge numbers of individual service faults
with floodwaters and fibre was damaged with about
where lead-in or distribution cables were broken or
600 customers impacted. Once again Grant’s team
inundated with water. Our local service team had a
swung into action and had all customer services up
huge task repairing these.
and running within 10 hours.
Grant’s team were fantastic and, all in all, the network
held up remarkably well considering the breadth and
severity of the flooding.
34
36. CONTENTS
WILDLIFE... WILDLIFE... WILDLIFE...
By Con Parslow, Service Delivery
Date: 12/10/10
During the summer’s inclement weather When he returned, the snake had returned too and THE RETURN
even the wildlife had to find alternative was sunning himself wrapped around the pillar and
accommodation. the technician’s Loop-a-Line. OF A SLIPPERY
Two communication technicians, Mark O’Callaghan
and Robert Hancock, opened a pillar in Austral
The Wildlife Information, Rescue and Education
Service were contacted and they relocated the CUSTOMER
to test a cable length. A snake was inside the lid snake.
and dropped down into the pillar while they were *The Loop-a-Line is a product developed specifically for telecommunica-
working. tions technicians involved in fault location activities. It is the orange tool
with the blue and white sticker pictured here.
Once the snake removed itself, they were able to
continue their job, locking the pillar down when they
were finished.
The very next day Gavin Kaimoana, another
communication technician, came along and opened
the very same pillar, attached his Loop-a-Line* and
left to complete the job.
Far Left: Tech Russell Thompson crossing Tumut River, Darbalara.
Above: A slippery customer returns. 35
37. CONTENTS
TEAM WORK... TEAM WORK...
“Technicians spent several months working very long days and
giving their very best...”
As told by Bill Lloyd, Service Delivery Muswellbrook
Date: October 2010 onwards The Muswellbrook team experienced extensive
LIGHTNING During the summer at a Service Delivery
Central North Field Service Area (FSA) level we
floods over a period of three months in the Mudgee
area. Roads were washed away and flooding creeks
STORMS, FLOODS had 56 team members move out of our FSA.
The majority of moves being for more than a month
impacted our ability to serve our customers.
Wollemi National Park customers were isolated for
AND ROAD each and some being two-plus months to help out
in other areas.
days at a time. Lake Burrendong Dam went from
having just 12 per cent capacity for many years, to
CLOSURES
over 130 per cent capacity.
There were many other moves within the FSA, as
we moved communication technicians (CTs) to The team supported each other with the
serve our customers while maintaining our FSA’s Muswellbrook part of the team travelling the 400
appointment and commitment performance. kilometres round trip, for one week at a time, over
three months. The Construct & Maintenance Greater
Foster
West team also assisted by responding promptly to
Over summer the Foster team’s main customer volume holds* and cable outages.
impacting concerns were caused by lightning in
the mountain areas. The coastal areas had minimal Dubbo
weather impacts which allowed the team to support The Dubbo team experienced flooding right across
other areas. its vast area, with the Macquarie and the Castlereagh
Rivers reaching record levels. Coonamble and
Over the summer period, the team had on average
numerous communities were isolated. The Northern
three CTs away working in other regions. There was
Queensland rains then started to move down the country
a period of three weeks where the team pulled hard
causing flooding in the Darling River isolating Kilpa.
together to look after our customers to allow five
technicians to be away working in other regions. Meeting the challenges
Cessnock Controlled road closures and dirt roads being
impassable were a continual challenge as the
This summer the Cessnock team area was
country absorbed the welcome rain.
impacted by many lightning storms. The committed
CTs spent many hours clearing multiple faults on our Technicians spent several months working very
customers’ lightning damaged services. While this long days and giving their very best under imposing
was happening the team supplied two technicians conditions.
to go to Queensland for several months.
*A Volume Hold Queue is created by Service Delivery’s Service Outage
Cessnock team members also spent time in adjoining Management team when a number of fix line services are impacted by
one fault (say a cable break or a roadside cabinet is inundated with water
teams including the Muswellbrook, Newcastle and and stops functioning). Volume holds are put in place while the fault is
Dubbo teams. The region was also supported by being repaired. Creating a Volume Hold Queue provides Telstra teams
with a single point of reference to obtain updates on outages affecting our
Cessnock sending a technician to Central South FSA. customers in one particular area.
36
38. CONTENTS
“I am proud of my team. We did all this work and
still managed to support our colleagues who were
in a worse situation. The team has performed
excellently and always put the customer at the
front of their minds concerning all decisions and
actions made.” – Niall Carey, Service Delivery Area
Manager, Central North.
Top Right: Clouds building for another lightning storm across Cessnock. Bottom Left: Coonamble fields flooded. Bottom Right: Highway through Dubbo. 37
39. CONTENTS
COLLABORATION...
“Many technicians travelled from
around the country to help...”
As told by David Gillett, Service Delivery Many technicians travelled from around the
[Southern] country to help out as part of Service Delivery’s
BACK Date: 3 December 2010 peak load strategy. Whether it’s fires, floods
or cyclones anywhere in the country, we have
During flooding in Wagga Wagga the basement
TO car park of the local main shopping mall was
the flexibility to get our skilled people interstate
quick smart to assist in restoring services.
filled with water, up to 1.5 metres in places.
BUSINESS The communications room, which is located in the
car park, was inundated. The batteries, four access
panels, the backplane, socket and plug assembly,
including the DC power access, were partially
submerged causing the loss of communications
(fixed phone lines, internet, data, alarms, fax
machines and EFTPOS services).
With co-operation between the local Network
Construction and Service Delivery teams, equipment
was sourced and the recovery process commenced.
It certainly was an around-the-clock affair, with
technicians Phil Briggs and James Tibbetts working
through Friday night 3 December in the flooded car
park to get the local shopping mall back online.
At around the same time, after receiving advice from
the SES that two RIM cabinets were in danger of
inundation (one on Hammond Avenue and the other
on Schiller Street, Wagga Wagga), members of the
Wagga team got to work sandbagging the roadside
cabinets and potentially saved around 5,000
customer services from being impacted.
38
40. CONTENTS
TEAM WORK... TEAM WORK...
By Terry Walsh, Service Delivery [Southern] John positively identified the fault location as a
Date: December 2010 washed-out creek crossing and our technicians
During December I received a call from Service
were onsite by 4:30pm.
WASHED OUT
Delivery Enhanced Services technician John The site was a disaster area, too wet and boggy
Collins, informing us of a possible fibre outage to get any vehicle on site, including the excavator.
at Coolac and Jugiong affecting more than 170 The bridge was mostly washed away as well and
PSTN (or fixed line) customers. this meant the gear had to be carried, by the field
crew involved, across a plank on what was left of
While John continued his investigations, the bridge.
communication technicians Geoff Sonneman and
Lawrie O’Callaghan organised an excavator and With all the challenges, and to the team’s credit, the
travelled from Wagga to Jugiong. fibre was up and running by 8:30pm.
Colleague Craig Price hooked up the fibre trailer in Early communications between Service Delivery’s
Albury and also headed to Jugiong. Enhanced Services and field teams ensured a quick
restoration time for our customers.
Opposite: (l-r) Russell Thompson and Geoffrey Sonneman protecting our assets. Above Left: Bridge at Jugiong. Above Right: Washed-out optical fibre cable in Jugiong 39
41. CONTENTS
CUSTOMER DRIVEN...
“Our team spoke to people who lost a great deal and they handled
themselves admirably in the face of adversity...”
As told by David Coxhell and Natalie O’Malley,
Service Delivery
CALL Date: December 2010 onwards
Call Central Newcastle, though based in While we did our best to relay potentially displeasing
CENTRAL Newcastle, operates nationally and comprises
several teams including Rescheduling,
news to our customers, it’s understandable
many calls had to be escalated due to customer
WEATHERS Call Central Front of House, Service Delivery
Customer Care, Recalls, Fee For Service and
dissatisfaction. Our consultants are well versed in
dealing with unhappy customers and did what they
THE STORM Vetting and Validations, Outage Management
and Interim Provision & Recovery.
could to assist them within set guidelines (diversions,
Personal Interim Phone Services (or PIPS), trouble-
shooting, etc).
Rescheduling
Each day we managed escalations and liaised with
With some of the fiercest weather conditions seen in the On-The-Day Schedulers in Service Delivery to get
several years hitting multiple areas of Australia this the best possible results for our valued customers.
summer, it was not surprising workloads skyrocketed This has been one of the busiest periods the team
throughout the various areas of our business. has managed in over 10 years.
A case in point was the rescheduling work done by Our team spoke to people who lost a great deal
the dedicated Call Central staff in Newcastle. and they handled themselves admirably in the face
Calls to customers to reschedule work the field have of adversity and always treated our customers with
unfortunately had to delay, due mainly to unavoidable dignity, respect and understanding.
weather impacts, rose significantly during December With many of our people cross-trained across
2010 to February 2011 (a total of more than 70,000 various functions, we pulled together to perform as
reschedules actioned during this time). a true team should, putting our best foot forward
to help provide the best possible levels of service
The increase in workload meant Call Central had
which our customers expect.
to train several new classes of consultants in quick
time given the need to keep our customers informed The Service Outage Management
of the status of their services. With the fault volumes pouring in, and field crews
The training was very hands on and very successful. dependent on being provided accurate and timely
Call Central Front of House staff, many of whom information on where and how the network had been
were newly trained themselves, had also been damaged, a huge effort was required from everyone
affected by the increased workload brought on by involved in Service Outage Management. The team
the unpredictable weather patterns. really stepped up and the effort put in by this team,
to support the field crews and our customers, was
truly impressive.
40
42. CONTENTS
Interim Provisioning and Recovery (IPR)
With significant network damage across Queensland,
it became increasingly vital for Telstra to be able
to provide our customers with temporary Interim
Telephone Services.
However, in a particularly cruel blow, the floods that
had done such damage to the network infrastructure
also inundated the main depot that holds our interim
phones.
So with demand for interims through the roof and a
large number of our units also unavailable, the team
at IPR really had to dig deep. And dig deep they did,
working massive hours trying to get interim phones
back from where they were no longer needed, so
they could quickly be deployed to customers who
really required them in the flood-ravaged areas.
During January and February, the peak period of
the emergency, they recovered and redeployed
thousands of interims.
The majority of IPR staff are cross-trained and
helped other workstreams where they could. The
entire IPR team’s attitude and work rate over the
period of the emergency is a tribute to their passion
and commitment for their customers.
Top: Some of the members of our Interim Provision & Recovery team (front) Truli Balloch (l-r behind the partition) Clinton Yates and Kathy Cox.
Bottom: (l-r) Some members of the Service Outage Management team (l-r) Brenton Crabb, Phil Ross and Chris Box. 41
43. CONTENTS
DEDICATION... DEDICATION...
“Reid’s Flat community was isolated by floodwaters
for days with the optical fibre cable that feeds the
telephone exchange being washed away...”
As told by Terry Lines, Service Delivery
Date: 16/12/10
LATE NIGHT This is a story about technician Richard Leary He drove around the back blocks of Reid’s Flat
and shows the extraordinary lengths our people locating the customers and installing the satellite
DELIVERIES go to, to serve our customers. phones. He found one customer had already been
evacuated.
Reid’s Flat community was isolated by floodwaters
for days with the optical fibre cable that feeds the Even though in the end just two installs were
telephone exchange being washed away at a river required, it was a very long haul with Richard arriving
crossing. back home at 2:10am the next morning.
On Thursday 16 December, Richard received a call
from Dave Minney, Area Manager Central South,
asking if he had any satellite phones available in
Cowra.
There were three medical customers in Reid’s Flat
requiring satellite phones, but Richard only had one.
However, he did some follow-up and soon located
more satellite phones in Orange and arranged to
meet a communication technician between Cowra
and Orange to collect them.
Richard then drove on to Reid’s Flat. Luckily, by late-
afternoon that day, the floodwaters had receded
enough for Richard to be able to drive through and
he arrived just on dark.
42