By Adam Drenoyanis, Unit Manager – Digital Control Systems, Shoalhaven Water
Shoalhaven Water (SW) is now concluding a three-year multi-generational leap in its remote telemetry systems. SW had identified the need to improve a number of areas within its current organisation-wide Supervisory Control and Data Acquisition (SCADA) system including the data analytics capability to provide better intelligence for its business.
The gaps in the capabilities and features of the current SCADA system stem from the following main issues: by the very nature of the older technologies, there are limitations in the features of the existing analogue radio system and legacy telemetry units to support more sophisticated data diagnostics, and time and date information required to facilitate data analysis. With minimal ongoing annual capital works funding on SCADA enhancements the current system has fallen significantly behind state of the art.
Hence, the objectives of this project are to address the gaps in the capabilities and features of the current SCADA system as described above through the following upgrade works: enhancements to the current ClearSCADA applications software configuration, upgrade of the data communications network from analogue to digital, and implementation of new digital RTUs at all remote sites.
This article details the steps taken to achieve cutover of such a significant portion of remote sites in the first year of a major capital works project; the experiences learned along the way; as well as presenting some state-of-the-art SCADA displays, reporting and IoT systems integration.
Shoalhaven Water (SW) is a large regional utility on the New South Wales south coast. Tied to Shoalhaven City Council, the approximate 5,000km2 coverage area consists of many satellite wastewater schemes as well as delivery of potable water from two large water treatment plants (WTPs) in both the north and south.
The water system consists of five dams, four WTPs, and numerous water reservoirs, water pumping stations (WPS), valves and other dosing systems – treating and distributing approximately 45ML of potable water per day.
The wastewater system consists of 13 wastewater treatment plants (WWTPs), 220 sewer pumping stations (SPSs), numerous valves and other dosing systems to collect and treat approximately 18ML of wastewater per day.
All of these assets are monitored through SW’s SCADA system and have been for some time. With the recent wave of Internet of Things (IoT), the 1,100 low pressure pumping units (LPPUs) within the wastewater network are progressively being retrofitted with low-cost sensors for inclusion onto the existing SCADA system for exception reporting and alarming.
Historically, SW has operated on a Radtel-based SCADA system with a mix of 3000/5000/8000-series RTUs communicating over an analogue radio network.
With these RTUs now unsupported and well past their end-of-life, a major push for a digital telemetry and SCADA upgrade was initiated in 2015.
Through extensive expressions of interest (EOI) and request for tender (RFT) processes, three tender packages were awarded late in 2017, with the major rollout commencing in January 2018.
In the first year of the upgrade project, over 60 per cent of sites were cutover to the new digital telemetry system, and now 2.5 years into the three year project, 98 per cent of the approximately 350 remote sites have been upgraded with the remainder scheduled for upgrade in the coming months.
This has been supplemented by state-of-the-art ClearSCADA software and reporting enhancements; significantly improving day to day operations for field staff, emergency response, as well as management decision-making and forecasting system-wide.
This exceptional rollout pace has been thanks to a committed project team; consisting of both internal SW staff, and partnerships with key external contractors; SAFEgroup Automation and Downer EDI, for both hardware supply and installations, as well as SCADA system integration.
A 20+ year old telemetry system has its drawbacks, although for SW and many other water utilities, in its day, the RadtelSCADA telemetry systems very well served their purpose.
However, over the years and turnover of staff in both management and field, and SCADA services and water/ wastewater operations as a whole, what started as standards slowly, over time, became not-so.
For SW specifically, the siloed schemes allowed staff to request their own means of displays, reporting methods and alarming protocols.
To the point where maintenance between the 13 databases ceased. Subsequently when staff from opposing schemes were scheduled for on-call duties, responses to problem situations would vary from site to site, scheme to scheme.
The last telemetry upgrade project in 2012 conducted extensive consultation with mainly field staff, where the majority wanted “the same look and feel” of the existing system.
As such, this upgrade was purely a conversion from RadtelSCADA to ClearSCADA, reproducing the same screens without any rejuvenating of SCADA mimics or reporting functionality.
Significant money was spent for no real improvements.
The major driver for this upgrade
This current upgrade has been driven from management, after external consultants provided a 30-year asset assessment and upgrade forward plan based on the data extracted from the SCADA system above.
Due to the inaccuracy of these records, most sites had over one million data points, when even filte
red down to a usable 100 thousand points have proven to be a significant overshoot.
This has led to inaccuracies in asset life and system capacity predictions, where some major capital works projects could have been delayed for extended periods and thus decreasing costs.
Where to from here?
An EOI process was advertised in 2015, whereby the SW SCADA team went out to the industry to ask questions and test the market.
This became quite a long internal review process, though it enabled SW to trial a number of digital telemetry solutions throughout.
A comparison between a polled RTU analogue point (red) and the same analogue as a buffered SCADApack 334E point (blue) in Figure 2 shows a distinct difference in data integrity.
Learning a lot along the way, SW was able to clearly define where it wanted to go for the future. Then, in 2017, some two years after the EOI was advertised, SW was ready to advertise RFTs.
Clear decisions were made on the arrangement of these tenders, splitting the project into three separate tender packages:
- Package 1: SCADA software enhancements/System integration (SAFEGroup Automation)
- Package 2: Supply of Hardware (SAFEgroup Automation)
- Package 3: Installation of Hardware (Downer EDI)
This arrangement allowed SW to be risk averse, and award successful tenderers best-fit to each of the package’s defined scope.
Project planning and rollout
Working closely with SAFEgroup Automation system integrators (SIs), a clear plan was defined for a quick delivery of remote sites in both telemetry hardware and SCADA software enhancements.
Beginning with the wastewater section first, the integral part of this was to develop a new standard SPS code and mimic, covering as many remote sites with a single template as possible.
With the above, and some other valuable tools developed by SAFEgroup Automation, configuring new telemetry hardware in preparation for installations was significantly streamlined.
And, with a steady stream of fully-built RTU panels being ordered, delivered and installed, SW had successfully cutover 210 SPSs, four WWTPs, 228 LPPUs and a number of water assets in the first year of the three-year project, well over 60 per cent of telemetry sites.
SCADA overviews and reporting
The new telemetry station pages offer significantly more information on a single display, eliminating the previous requirement to jump from page to page just to trace back history, reports and events.
Compared to that shown in Figure 1, the move to more abnormal situation management (ASM)-inspired standard displays, as shown in Figure 3, has improved staff response to critical events and allows clear alerts to any states out of the ordinary.
More focused on the scheme-based overviews, the upgrade to ClearSCADA 2017 software has enabled the inclusion of map-based geographic displays.
This, in turn, provides the ability to overlay web-based map service layers such as internal GIS layers and Bureau of Meteorology data feeds.
This functionality has allowed vastly improved management of emergency response. These are shown in figures 4 and 5 respectively.
Furthermore, utilising fixed radar level sensors in SPS, and continued improvements in SCADA data collection and prediction has allowed for the plotting of standard, expected diurnal inflows at critical wastewater sites (Figure 6).
This gives the ability to alarm in real time lower than (or greater than) expected flows at specific time-slices of a day, allowing early identification of potential problems upstream from the corresponding SPS site.
One of SAFEgroup Automation’s latest ClearSCADA features is a Microsoft Teams integration (Figure 7), which, in the current pandemic has been heavily utilised to send daily data updates to management staff which do not require usual SCADA access from home.
The Internet of Things (IoT) SW began its IoT journey early in 2017, with a number of undergraduate engineers coming through since then focusing their university theses on IoT applications within the water/wastewater networks.
Over this time a significant network has been rolled out throughout the region (Figure 8).
Not only servicing other parts of council for smart cities-type applications, SW has many devices in development for applications, including monitoring of gravity sewer manholes; digital and analog level indications; pressure reducing valves (PRVs) (Figures 9 and 10) and an Implementation of an IoT Based Radar Sensor Network for Wastewater Management1 paper.
This paper offers a complete review of wastewater monitoring networks and was one of the first to offer robust testing of Long Range Wide Area Network (LoRaWAN) network capabilities in Australia.
The paper also provides a comprehensive summary of the LoRa protocol and all its functions.
It was found that a LPWAN, utilising the LoRaWAN protocol and deployed appropriately within a geographic area, can attain maximum transmission distances of 20km within an urban environment and up to 35km line of sight.
Additionally a second IoT network has been configured and deployed in the Kangaroo Valley and Lake Tabourie townships, with sensors developed in partnership with Ventia/Vianet to monitor the low-pressure sewer system (LPSS) pump network tied to each residence.
A low-cost retrofit solution was designed to supervise the previously unmonitored eOne LPSS (Figures 11 and 12).
Response to issues with these systems has been historically reactive, rather than proactive, reliant on notification of the resident to the local visible and audible alarm.
With the IoT devices in place it has permitted SW to transition to more of an ‘invisible’ utility – monitoring each LPSS pump alarms, runs, starts, runtimes and power consumption, allowing proactive attention to problem sites prior to residents becoming aware.
Large-scale deployment is complete, with over 700 of these devices installed and a further 400 scheduled for the townships of Bendalong and Lake Conjola.
The data collected from these devices has provided an excellent comparison between live ‘standard’ gravity sewer and low-pressure sewer systems.
This article has detailed the current SCADA upgrade currently nearing conclusion by Shoalhaven Water. From the historic system outlined above, to the improved system being delivered, the benefits are clear.
These are not only from visual software enhancements, but also in the way data is collected from sites. The move from legacy analog- based protocols to DNP3 has enabled a vastly greater trust in the data produced by SW’s SCADA system.
Leading on from this and into ‘Industry 4.0’, SW now has the opportunity to create IoT-capable devices that offer freedom from the reliance on mobile network providers, whilst avoiding congestion on the existing SCADA telemetry backbone.
This has allowed better-quality asset life forecasting as well as predictive alarming based on operationally historic trends.
SW would like to thank all who are currently involved in this upgrade project. Both internal operations staff and key stakeholders who have provided support and guidance to allow this project to endure, as well as the external contractors from SAFEgroup Automation and Downer EDI who have kept up with a rapid rollout pace, allowing SW to remain ahead of schedule and under budget.