MultiTrode Blog

Pump Station and Lift Station Technology.

Archive

Archive for February, 2009

Designing large-tag-count SCADA systems

February 27th, 2009

The magazine Control Engineering ran their monthly Information Control email today which included a tutorial from MultiTrode and Parasyn:

www.controleng.com/article/CA6640186.html 

They describe the tutorial:

SCADA systems aren’t scalable out of the box; you have to plan with the final footprint in mind,” says Tony Poole, managing director of Parasyn, a system integrator specializing in water/waste water applications. Steve Carson is with MultiTrode, a manufacturer of MultiSmart pump station manager units, which are replacement devices for pump controllers or PLCs/RTUs for lift stations. The devices add more monitoring and control capability to SCADA systems, and can also add 400 to 500 tags (data points) per site. In this tutorial, Carson and Poole provide best-practice advice for designing large tag count SCADA systems so they are manageable.”

Control Engineering  have a number of email newsletters that you can subscribe to. There’s plenty of good quality articles that make it worth the free subscription. Just visit http://www.controleng.com and you will see the Newsletters menu item on their site.

There’s a story behind how we came to write that article together, a subject for another day.

We find the subject of large tag count systems very interesting – we’ve run into it a number of times and in different elements of the SCADA solution. Parasyn’s approach made a lot of sense when they explained it to us and we certainly learnt from their experience.

“SCADA systems aren’t scalable out of the box; you have to plan with the final footprint in mind,” says Tony Poole, managing director of Parasyn.

Have a read of the article. It will be in the print edition of Control Engineering  in April. Read more…

General News , , ,

Vacuum Pumping Systems and Flovac

February 27th, 2009

We’ve been working with Flovac for quite a while. Our relationship started off with them using our older controller - the MT2PC (actually a pair of MT2PCs) for their vacuum pump stations. They’ve now standardized on the MultiSmart Pump Station Manager and there’s a News item on the main MultiTrode website.

There’s a great 3 minute video about how vacuum pumping works on their home page, www.flovac.com

John Radinoff, the Flovac Managing Director, explained to me why vacuum pumping is gaining ground: ”These systems are used in areas difficult to sewer using traditional technologies. Ideally suited for areas with high water table, rocky sites or flat ground, the system has proven to be extremely economical in such topography. 
Eliminating infiltration and exfiltration, vacuum sewerage is highly regarded by many EPA’s around the world and has proven to be the sewerage technology with the lowest carbon footprint. This is the reason why it is the naturally preferred choice of sewerage in high profile sustainable projects like Masdar in the UAE.”

It’s clear that gravity sewer is really expensive when you have  a high water table, rocky terrain or very low density housing.

There’s a good Wikipedia article about vaccum pumping.

Pressure sewer technology covers a similar list of advantages over gravity sewer.

For MultiSmart, the Flovac application is a 4-pump 2-well configuration. The “wells” are independent. On one side we are measuring the vacuum pressure and based on that starting and stopping the pair of vacuum pumps. On the liquid side, we are measuring the liquid level in the well and starting and stopping the wastewater pumps. Read more…

General News , ,

Lift Stations running on Generators – MultiSmart Profiles

February 21st, 2009

In Florida, and probably in lots of other areas that face power outages regularly, there are lots of generators on lift stations.

One of the requests we had was for the capability to limit the number of pumps that could run when in generator mode. We incorporated this into the Profiles feature of MultiSmart.

 

What’s a profile? In MultiSmart, a profile is mainly about setpoints but includes a few other parameters for pump control.

The idea behind setpoint profiles was to make it much easier for a water or wastewater utility to switch between different setpoints at different times or under specific circumstances. There were many requests for changing setpoints at different times automatically, for SCADA control and for operator intervention when on site. In another post we’ll look at spill management using profiles, which is another great use for the function.

Because of the requirement for Generator Profile, or GenSet Profile as it’s known by default in Multismart, we incorporated some pump control functions as well.

  • max pumps to run
  • max run time
  • max off time (usually used for odor control in wastewater, but has other applications)

This allowed a utility to have a 3-pump station where all three pumps could run normally, but under a generator-powered condition they could only run two pumps. It would be just as easy to have a 3-pump station which could normally only run two pumps, but under generator load, only run one pump. MultiSmart simply has a parameter called Max pumps to run which applies to each profile, including the default profile. You simply change the parameter for each profile (or leave it at the default which is “no limit”)

We also setup this profile so that by default the setpoints were a lot lower in the well, but this is easily changed through the setpoints menu.

Changing setpoints on MultiSmart

Changing setpoints on MultiSmart

 

How do you switch to this profile?

Generally profiles can be switched by:

  • Digital input
  • SCADA control
  • Date/time (there are 4 timers for each profile)
  • Operator interface
  • Customer logic, e.g. from the PLC extension to MultiSmart

So in the case of GenSet Profile you would probably configure it to become active when the Generator Transfer switch activated.

By the way there are 6 profiles in MultiSmart. Two of them are labelled up already: spill management and genset. However, if you wanted to use profiles and didn’t these you can change them –  any profile can be renamed and configured exactly how you want it.

 

Can the setpoints be changed via SCADA?

Yes, as well as switching profile via SCADA, all of the setpoints for each profile can be read and written to by a SCADA system.

 

How do you configure ”Max pumps to run” in a profile & when a Profile will be selected?

Because it’s a relatively advanced feature, not surprisingly it is under the “Advanced” menu. 

Locating the Advanced screen in MultiSmart

Locating the Advanced screen in MultiSmart

 Press Settings – Advanced and expand out Pump Control to select Profiles. Then you select the profile you want to configure and all the options are in there, including the pump control options, selecting a Digital Input on which to make it active or choosing timers.

 

In another post I’ll look at the Spill Management Profile.

Read more…

In the well , , ,

White Paper on SCADA reporting

February 19th, 2009
This post covers the same topic that you will find on our News page - I’ve duplicated the theme – although not the content – so that those who only follow the Blog don’t miss anything. In time, I expect that we will create less News items and do more Blog postings – but at the moment the Blog concept is still pretty new for a lot of people.

 

Our White Paper on SCADA reporting also appeared in Pumps & Systems – for those who don’t know it, an excellent US industry magazine. You can find the online version of the magazine at www.pump-zone.com.

What’s the White Paper about? Essentially it looks at where SCADA and telemetry reporting currently stands for a typical lift station network – compared with where it could be – and where the utility asset managers and operations managers would like it to be.

 

Operational Cost Breakdown

Operational Cost Breakdown

Anyone who has implemented a SCADA system, or even played with a SCADA package can tell you how easy it is to create an animation on the HMI that depends on a data point. For example, displaying a well emptying and filling as the level “tag” falls and rises in value. Or a motor changing from blue to red as the temperature tag increases in value.

It’s probably the first thing that you get taught on the SCADA course, or get shown by the salesperson when he/she demonstrates ease of use of that SCADA platform.

 

Reporting is a whole different question. There is the challenge of finding where the real data exists in the system, understanding data integrity and getting to grips with the reporting interface. But the bigger challenge for many end users and SI’s is that creating a good report means getting to grips with relational database concepts, and that’s a lot harder than linking a process value to a graphical animation.

 

As Paul Francis, MultiTrode CTO, said:

Reporting has always been a challenging aspect of any SCADA platform. Visualization and trending tools were some of the earliest adopted elements of SCADA systems and hence matured long before reporting interfaces did. Developing useful reports can also be more of a technical challenge than making a nice graphical interface for your plant or collection system; in part due to the nature and structure of the underlying data.”

 

Away from the mechanics of reporting, there are other challenges. For example, data acquisition in the field. Typical PLC and RTU systems aren’t bringing 100’s of tags back to the SCADA system. That requires a level of design that is rarely catered for. With minimal data, it’s hard to generate much in the way of reports – and this may well have contributed to the low expectations of asset managers.

 

Reports - Pump Efficiency Changes

Reports - Pump Efficiency Changes

And in the US, the de facto standard for water and wastewater telemetry protocols is Modbus. This works against any attempt to get asset rich data. (See, for example, DNP3: Part One or DNP3: Part Two).

 

None of this means that it’s too difficult to get reports that will cut operational costs or provide asset reports to allow better capital allocation. It’s just important to understand the different roadblocks that lie in the way.

 

If you want to read the SCADA Reporting White Paper you will have to fill in a short registration form (unless you already have a MultiTrode website login). Read more…

SCADA & Telemetry , , ,

Smart Waste Water Treatment with MultiSmart’s PLC Extension

February 15th, 2009

 

Recently, we have received a purchase order from a customer regarding a lift station (pump station) application for odor reduction for their sewage treatment plant. The sewage pit, as you all might know, fills up very quick during the morning as well as in the evening and is very slow during the late nights. For odor control the customers are dosing SulfaLock into the waste water pit depending on the flow rate. After observation of the pit for a few weeks the customers came up with a diurnal curve, which is to be followed to control the dosing. They also wanted the ability to change the data values for the diurnal curve, as and when the statistical data corresponding to the flow in the sewage pit in a day changes.

 

The diurnal curve is shown in the figure below. The output rate required is plotted against the time of the day in minutes.

  

Diurnal Curve - Waste Water Treatment 

Diurnal Curve - Waste Water Treatment

  

This unique RTU application requires custom programming which can become quite cumbersome when having to start from scratch. MultiSmart, having most advanced pump station management technology and the PLC extension built in, made this custom application development look very simple. With support for DNP and MODBUS communication, which are the popular communication protocols used in Water and Waste Water Treatment Telemetry, and the integrated 1000 plus tags related to Pump Station Management statistics it was extremely easy to configure the communication.

 

The functionality is programmed into the MultiSmart with the help of the PLC extension. More information about the PLC programming within MultiSmart is described towards the end of this blog. Data points at every hour mark is identified from the diurnal curve and saved to the parameter profile of the PLC program. This has the added advantage that the data values can be changed at any time without any change to the program whenever desired. However there are security measures in MultiSmart which will restrict any unauthorised access and modification to these parameter values which can affect its operation.

 

The PLC program looks at the current system time, calculates the value of the data point with respect to the diurnal curve and generates a corresponding output to the Analog output of the MultiSmart which is driving the dosing pump. The Analog output events were logged against the time using the built in Event Logger functionality in MultiSmart and we got the following result (graph plotted in Excel) when operated for nearly 20 hours.

  

 

MultiSmart Analog Output

MultiSmart Analog Output

 

The output stayed well within the expected range and was steady. The whole application was developed and the initial test was carried out successfully in less than 7 hours.

  

Brief Overview of MultiSmart’s PLC extension, ISaGRAF

 

ISAGRAF is a control software environment which supports all of the internationally recognized IEC61131-3 control languages and offers a combination of highly portable and robust control engine. Support for the latest release, version 5, of the ISaGRAF is available in MultiSmart.

 

This PLC extension of MultiSmart allows system integrators and end-users to further enhance or adapt the capability of MultiSmart.  The product has enormous amount functionalities built in, but there are always customers or applications that push the envelope.  The product has 1000s of tags, and System Integrator’s or End-Users can now interface to these with the PLC engine.  This means they can extend the capability without having to rewrite existing functions.

 

The ISaGRAF implementation of IEC61131-3 is added because it is the most proven platform and v5 is chosen as it offers a number of advantages over v3.  One example is the ability in v5 to run different ‘resources’, or applications, at different rates to allow more critical applications to run more frequently.

 

A sample ISaGRAF program using function block diagram is shown below. The capability to show the run time statistics, including the values of variables, in the MultiSmart faceplate helps immensely in the development of the PLC program and in troubleshooting.

 

Example of ISaGRAF PLC Program with Fucntion Block Diagram for MultiSmart

Example of ISaGRAF PLC Program with Fucntion Block Diagram for MultiSmart

 

The ISaGRAF TIC code can be dumped straight into the MultiSmart directory structure, which enhances the easiness in portability.

 

Read more…

Technical Notes , , ,

SafeSmart Level Alarm Relay and Failsafe Probe

February 12th, 2009

The story starts with some rats, quite a while ago.

 

One of our customers came to see us and together we had a few technical discussions about how to improve the single sensor probe and the MTR relay as an independent high level backup system.

The problem they had encountered was that rats had eaten the probe cable in a number of sites. When the probe isn’t covered by conductive liquid (e.g. wastewater) it registers as an open circuit. When the cable is broken, disconnected, or the target of hungry rodents it also registers as an open circuit. So the high level wasn’t picked up – a major problem.

 

Their idea was to bring an extra wire from the sensor in the level probe to the alarming relay and have the relay carry out a continuity check.

It seemed like a great idea. When we talked some more about the idea we first proposed one alarm output for both high level and for loss of probe. I think we were trying to work out if we could re-use most of an older product for the new design. That idea didn’t work for them for good reason – their PLC/RTU control system would start the pumps on a high level alarm and run them for a set period, but if the alarm signal was only indicating loss of probe, the well could be almost empty and they might run the pumps dry and airlock them. Then lots of work getting the pumps into service again!

So the product concept became a level alarm relay with two outputs – one for loss of probe, and one for high level.SafeSmart level alarm relay and probe

Like many product companies we then had lots more ideas for other variants, and with all the MultiSmart work we were doing at the time, the product took a lot longer than it should to make it out the door.

Well, we released the SafeSmart level alarm relay today. The part number is Safe-FS.

We learnt quite a bit as a company doing this simple project.
Something for everyone else to take away from the story is to bring your ideas and problems for discussion. They might turn into a great new product.

Read more…

General News , ,

Free Software Upgrades for MultiSmart – and Updated Specification

February 10th, 2009

A reminder for those who aren’t aware – if you have a MultiSmart you can get FREE software upgrades for the life of the product. This applies to any end-user or system integrator or contractor working for the end-user.

If you haven’t already registered on the MultiTrode website, all you have to do is click on the SignUp box in the top right on the main multitrode.com site:

 signup-website

 

 

When you get the member registration form, don’t forget to select the box which says: “Please give me access to the free MultiSmart firmware upgrades..” – see below..

 signup-multismart

 

 

If you have already registered on the new multitrode.com website and didn’t select this option at the time - but do want access to MultiSmart firmware, just send an email to webmaster@multitrode.com requesting an upgrade – best to include a phone number where we can contact you.

 

Specification Document Update for v2.0.4

Part of the reason for this post was to let people know that the specification document has been updated for MultiSmart.

The specification document generally has two main audience groups – those who want a chance to review the main features of a product – to understand what the product offers; and, those who want to write that functionality into a specification for a bid document.

The new MultiSmart spec document is relevant for v2.0.4 onwards of MultiSmart, and includes functionality like “locked level detection” - a subject of a future blog post.

v2.0.4 was released in December 2008. If you don’t have a copy and you are a MultiSmart end-user, or an SI or contractor for an end-user, please register on our website as described above. 

How can you get hold of the spec document? Visit http://www.multitrode.com/pump-station-manager/specifying.html to find the latest pdf and word copies. 

MultiSmart - specification

If you are looking for specifications for any MultiTrode product, simply navigate to that product by clicking Products & Software, then find the product you are looking for and select the Specifying link.

And if you stumbled across this blog post and want to know how to get future blogs without having to remember to visit the website 10 times a month – simply subscribe. If you are already reading this blog post in a reader (like Google Reader or Feedburner) then it looks like you are already ok. If you are reading this on our website, just click on one of the two links at the top right of this blog – subscribe by email – OR, better, subscribe by RSS (less clogged email inbox). Read more…

General News ,

Why use DNP3? Part Three – Security

February 5th, 2009

This post continues the themes from Part One and Part Two.

The subject today is security, and also why proprietary protocols aren’t the answer.

Security in communications is a hot topic, but in practice in the water and wastewater industry, not many people are actively implementing it.

It’s important to differentiate between “hacking the comms” and “hacking the server”. If there is a greater problem for the organization, it’s surely someone hacking your server through a firewall –  or from within your building – because now they can take control as well as present the operations staff with a completely false worldview.

However, if the SCADA server is highly secure and someone was very motivated to take control of your system, then they could potentially do a lot of damage by hijacking the communications. Imagine if they turned off all of the sewer pump stations in a city? You can send your staff out to put every station into manual over-ride – but only once you knew about it, and it would take some time to get to every single station. You would have lots of overflows, and you would have your whole team racing around from station to station. If it happened in a time of high inflows – e.g. a storm – then the problems would be much worse. In a water supply system it might be possible to burst pipes.

There are a lot of articles about communications security that start: “Since 9/11″ – probably because it gets higher exposure. But how much of a risk is it?  And is the risk greater from other sources than terrorism – like disgruntled ex-employees? It’s certainly getting attention from governments, but not much practical attention from the utilities themselves.

This article doesn’t try and address the risk factor. Instead, we’ll just explain a little about how communications to remote sites can be secured.

 

Security in Communications -  Are proprietary protocols the answer?

One subject that the promoters of proprietary protocols majored on in recent years is security. This is because they didn’t have a lot else to hang their hat on.

What am I talking about? Open protocols have been the perceived way forward for a long time, but especially in this century/millenium. For the last 5-10 years in the water & wastewater industry, almost anyone writing an engineering spec, or an operations manager or utility director who had done a small amount of research, knew that you needed to specify an “open protocol” for a new or upgraded system.

This presented a challenge for a number of companies who had their own protocols in their RTU and used these protocols to lock in customers. What to say to show they were progressive?

“At least no one knows how to hack our protocol, that’s an advantage..”

By the way, I’m not including in this list, companies with their own protocol who made them public. There are many companies, including ourselves, who in the 1990’s had their own telemetry protocol in their RTU because it seemed – rightly or wrongly - to have some advantages at that time. The important point is, once the move towards open protocols became desirable or a requirement, and high quality telemetry protocols became available, what did those suppliers do? The responsible ones published their protocol and made it easy for other parties, including competitors, to copy them.

In fact, most proprietary protocols aren’t that hard to reverse engineer.

 

In the world of encryption and authentication, the experts will tell you that openness is what allows the audit. Don’t tell the world that your protocol is “secure” because it is proprietary, unless you have invited a few hackers to break it. It probably won’t take them very long.

A good recent example is where one of our partner companies, Trihedral, reverse engineered a proprietary protocol from another supplier to allow them to break into their market – to replace the SCADA server software while still interfacing to the RTU’s in the field.

Only last year (2008) I read an article in a water industry magazine by a supplier saying how their RTU protocol was more secure than DNP3 because it wasn’t published..

Time to move on..

DNP3 Security – How Does it Work?

Security is one of those tricky subjects that most people actually don’t want to understand. As a user you just want to know it works. So I’ll stay away from the more technical aspects.

DNP3 is a published protocol with a very strong and a very technical user group. You can be sure that the people in the user group who published the security specification knew what they were doing.

Very simply, DNP3 security doesn’t encrypt the message, it authenticates the message.

If someone intercepted a command to an RTU: ”turn on Pump 1″ which might look like “digital tag 15 ON”, they could read it!

But if the bad guys then wrote a command to send to the RTU - ”turn off Pump 1″ – “digital tag 15 OFF” – the DNP3 authentication mechanism would reject it. The security mechanisms in DNP3 can determine when the command is a valid one by a trusted party.

This gives an insight into why the oil and gas suppliers might want RTU encryption, not authentication. In a highly competitive commercial environment you don’t want others to know how much volume you pumped  -for example.

In water and wastewater, even in the privatized but regulated market of the UK, it’s hard to see how anyone reading your pump station commands could cause you a problem.

The key point of DNP3 security is that while others can see what you are doing, they can’t pretend to be you and tell your system to do the wrong thing. That’s what authentication means.

 

If you want to know more, please take a look at our White Papers section. You need to fill in a short registration form to download any of the papers. The title of that paper is “Keeping SCADA systems open and secure from cyber-attack”.

There’s a lot more technical data available – check out the DNP3 Users Group.

And if you’re one of those people who like to understand more about “how everything works” – for the confusing world of encryption and authentication, a personal recommendation is Cryptography Decrypted by H.X. Mel & Doris Baker. It was published in 2001 so it might be a little dated, but I found it made subjects like public key encryption finally understandable. Read more…

SCADA & Telemetry , , ,

Energy efficiency in Florida

February 3rd, 2009




We recently did a survey of Florida water and wastewater utilities. Here is the pdf of the survey.

One of the surprising results from the survey was about energy cost and its importance:

 Energy efficiency in Florida

The surprising – and encouraging – aspect was that 47% already rated Energy Cost and CO2 emissions as very important. It wasn’t surprising to see that 85% thought that although it would increase in importance, other challenges would still be more important. Challenges like ensuring sewage doesn’t spill out into the streets, that it is properly treated before being discharged and that potable water is supplied to the residents and customers of the water authority.

 

Until we had the survey results, the view we had picked up from the ground in Florida was that energy cost was interesting but not particularly important. Contrast that with the UK, where the question to a typical water utility “What is the biggest issue you face?” got the response “Energy cost and reducing it.”

Perhaps the UK water utilities are fundamentally different because they are so much larger – typically 1000 – 5000 pump stations – and not government owned. Or perhaps they felt that although keeping sewerage off the streets and drinking water in the taps would always be their no 1 priority – it wasn’t their biggest challenge..

 

 Pump Efficiency

A while ago we did some research on energy efficiency in lift stations and found that pumps can lose a lot of efficiency even in clean water. Obviously wastewater is worse and the level of efficiency loss depends on the kind of material being pumped – if you have sand, grit and rocks going through the sewer main it’s obviously going to be worse. The Hydraulic Institute said:

“A pump’s efficiency can degrade as much as 10% to 25% before it is replaced, according to a study of industrial facilities commissioned by the U.S. Department of Energy (DOE), and efficiencies of 50% to 60% or lower are quite common. However, because these inefficiencies are not readily apparent, opportunities to save energy by repairing or replacing components and optimizing systems are often overlooked.”

 

Now the reason a lot of water and wastewater utilities have started using the MultiSmart pump station manager isn’t because of its energy efficiency functionality! But the more progressive ones have definitely seen the potential of MultiSmart to reduce their energy costs. As the Hydraulic Institute said, “Because these inefficiencies are not readily apparent..”  – If you don’t know what’s costing you money, where’s the incentive, or even the budget, to start fixing your problems?

 

More on Energy & Efficiency

There is a section on our website – Energy and the Environment – which has a few references to energy efficiency studies. And there is also a paper in the White Papers section – note that you need to fill in a short registration form to download any of the white papers.



Read more…

In the well