Cellular communications has made huge progress in the last few years, and many people would say is a viable solution for water & wastewater telemetry.

The first area that cellular comms started getting attention was for water supply and wastewater collection systems – as a backup to radio.

The requirements would state that radio was the primary communications and cellular GPRS was the backup on more important stations.

We also saw a few utilities who requested cellular as their primary communications method, either because they had had a lot of problems with radio in the past, or because their geography meant that building a radio network was clearly a lot more expensive than using cellular.

Let’s look at the thinking behind the backup first of all.

Historically, in water and wastewater, radio has the been the main method of RTU communications, with PSTN (phone lines) coming a long way behind in second place. One of the main benefits of radio was the fact that the utility owned the infrastructure and therefore “felt” some level of control over it.

In theory, if you own the infrastructure then you are able to run an operation which ensures an uptime that you are happy with, i.e., that meets the organizational requirements. If communications to one site goes down then you have a problem – but one that you can theoretically fix. If a repeater goes down, it’s the same situation.

Contrast that with cellular comms where if a cell tower goes down, or for some reason there are other comms problems in the backbone or to one area, you have to wait for the cellular operator to fix it.

But apart from those whose lifeblood is radio communications, this can also present a major disadvantage of radio communications. A lot of utilities don’t have the expertise to troubleshoot and fix radio systems, or to replace radio repeaters in the middle of a storm. Even if they do, there’s the question over response time. In later posts we will look at possible benefits of cellular, but for now, the radio “ownership” problem and the low cost of cellular comms raised the possibility of using cellular as a backup. Read more…

General News Cellular Communication, telemetry

v2.1 of MultiSmart firmware is now on the website. Take a look at the recent post to find out what is included in this release.

The firmware can be downloaded if you have been given access to the firmware area. Create a login on the main site and request MultiSmart firmware (one of the checkboxes). If you already have a login, click on “Edit my details” in the top right of the main site. Read more…

General News dnp3 security, MultiSmart, PID

v2.1 of the MultiSmart Pump Station Manager will be released in the next week or two. There’s some great new features included as well as lots of small enhancements and cosmetic upgrades:

• PID control, e.g. for constant pressure, constant flow via IsaGRAF
• New comms screen making all the communications much easier to setup
• Smart Outflow calculation to cater for high inflow conditions
• Updated DNP3 security (v2)
• Time to spill calculation on the main screen
• Generator functionality (e.g., run time and starts)

More about these features in later posts.

What else is in v2.1?

Fault finding tool – you can enable DNP3 and Modbus logging on the unit, then view the log on the LCD. We’ll do a post later which shows more about how this works in practice and why it’s useful.

Pump Running by External Control – the display shows when the over-ride is running the pumps (known via contactor auxiliary or currents). The main screen shows External Run next to that pump. And the starts and hours run accumulators also update. Thanks to Todd Burnett from Coastal Process & Steve Lahm from North East Water for requesting it (and maybe others as well).

DOUT from multiple sources – you can already do with this with IsaGRAF or the logic engine, but a few customers requested being able to configure a DOUT from a number of sources without using custom logic. So the user interface lets you choose a number of sources and select the “operator” – AND, OR, XOR.

Station Outflow alarm – there are already flow alarms by pump, this has been introduced to cover the complete station.

Cosmetic improvement - holding down a softkey repeats the keypress – greatly welcomed by anyone who has configured lots of faults and digital inputs for example.

Upgrading – the DSP upgrade is automatic, so you don’t have to remember to do that after a firmware upgrade. And when you put in a CF card with an upgrade image the unit prompts you to do an update.

Extra IO blocks in LCD – the Acromag and Adam IO units weren’t fully integrated into the LCD but now have been. Previously the available IO from external units didn’t show up in some of the settings screens (only in the advanced screens), but now do.

Custom names for IsaGRAF tags – the user can rename any IsaGRAF tag as well as change values, making the process a lot more intuitive. Read more…

General News MultiSmart

Following on from the post about MultiSmart v2..

Since v2 was released there have been a few minor releases. There are lots of small features and cosmetic improvements with a few more important additions. I highlight in bold the more interesting ones, but if you are a current MultiSmart user then any one of the smaller items may be important enough for you to want to upgrade.

If you do want to see every last “ticket” (as we call them), you need to get access to the MultiSmart firmware page, which you do by creating a login on the main site and requesting access to MultiSmart firmware. (Or, if you have already signed up, login and click “Edit my details”).

2.0.1 added

• Support for Single-Phase AC Monitoring and Fault Detection – important for smaller pump stations so that the “phase fail” functionality can be included
• Integrate Acromag IO Devices into screens
• Configurable units for pump efficiency
• Efficiency calculations during an overflow
• Show DSP Version number on Info page
• Invert Analog input value
• Telstra Modem support – CDR-882SEU & CDR-780SEU

 v2.0.2 added:

• Multiple thresholds on Insulation Resistance Test (IRT) -to allow a warning and a “fault pump” value
• Level device analog compare

v2.0.3 added

• Level locked feature – to indicate faulty level device
• I2T motor protection
• Protect reset accumulators via security login
• Support for redundant IP address

v2.0.4 added:

• Remote reconfiguration of DNP points list
• Display inversion – user selectable

A reminder that you can download the latest firmware for free from the MultiTrode website. We will shortly be releasing v2.1, so there will be a post about the main features in 2.1 very shortly. Read more…

General News

When you find useful stuff on the web, it can be hard to keep track of it all. You can bookmark it, and in Internet Explorer (or another browser), you assign it to a folder.

Later, you have some memory of a useful website that you found but when you look in a likely folder under your bookmarks there are 50 links. Is it there? It’s hard to be interested enough to find it, so you do another Google search and start over..

I started experimenting with Diigo (www.diigo.com) a few days ago.

Diigo is one of many “social bookmarking” websites but seems to have some handy add ons.

What’s a social bookmarking website anyway? They are websites that let you share interesting websites with friends or colleagues.

What diigo offers which makes it more interesting – even if you aren’t interested in any sharing, is you can:

• highlight a section (or sections) of a webpage
• pin a post it note with your comments
• save it to your Diigo bookmarks with one or more tags

The tags concept lets you create your own categories and later search for the bookmarks under these categories.

It’s a simple solution to the problem that anyone who uses Windows Explorer to store files has found – you can only store a file in one location so later, you might try 5 different places to find what you are looking for. If instead you can tag up a website with a few different keywords you have much more chance of searching for it – and finding it – later.

Here’s an example below:

Commenting on and highlighting a section of a website

If you go to www.diigo.com and sign up you can add the diigo toolbar to IE (or Firefox). The toolbar lets you easily highlight and add your sticky notes or comments to a site – and to review all the websites you have bookmarked.

I’m on diigo as “stevecarson”. I’m going to try it out some more and see how it works for the sharing side.. Read more…

General News

An almost universal rule of lift stations is that the engineer designing the lift station does his calculations and adds a safety margin. He or she passes it to someone else in the organization who adds a safety margin. Then it goes back to the city for approval and they add a safety margin.

Finally, there is always the chance that when the PO is with the supplier, he or she says “sorry, we’re out of stock of that model, but I have the next size up – tell you what, I’ll sell it to you for the same price as the smaller pump!”

The net result is pumps which are much too large for the application, running for very short times and not at all at their best efficiency point.

There’s a great article about how VFDs can improve the energy efficiency of pump stations by Joe Evans of Pentair writing for Pumps & Systems. The VFD lowers the effective output of the pump and runs it closer to its best efficiency point.

Another way that utilities solve the practical problem of the over-sized pumps is to add a jockey pump later on. This is a more appropriately sized pump for operation most of the time, with no concerns about worst case inflow condition because the larger pumps will take over.

Some stations are designed like this from the outset – especially stations with infrequent high inflow conditions where the engineer has recognized that large pumps are occasionally necessary but running these large pumps for very short periods is not ideal.

What’s the Lift Station Logic?

The lift station or pump station logic we use in MultiSmart is to group pumps.

So with the case of 2 large pumps and 1 jockey pump, Group 1 would have the jockey pump (let’s call it P1) and Group 2 would have both large pumps (let’s call them P2 and P3).

Group 2 would most likely be set to alternate – and it could be set to alternate based on efficiency to save energy automatically. Group 1 doesn’t need an alternation scheme because there is only 1 pump. (If there were 2 jockey pumps they would probably be set to alternate as well).

We configure the setpoints of P2 and P3 above the jockey pump, P1. And we set a parameter called Max Groups Running to 1, and Block Running Pumps to True (which is the default).

Now what happens is the level rises and P1 starts. The well empties and P1 stops. And the cycle continues. But once a time is reached when P1 can’t handle the inflow, the well level will keep rising and the start point for the lead pump in Group 2 is reached. When that happens, Group 1 (which just contains P1) turns off and the lead pump in Group 2 starts.

So long as this large pump empties the well, once the stop point for the lead pump is Group 2 is reached, Group 1 will again take over again from Group 2.

Depending on whether you have Duty/Assist or Duty/Standby you would set Max pumps to run in Group 2 accordingly. If you want both pumps to run together you don’t have to change the default (no limit), if you only want one pump to run, then you set Max pumps to run in Group 2 to 1. It’s an important point because a lot of stations don’t have the pipework to handle both pumps running together and all you do is double the energy consumed for a 10% increase in flow.

The way the logic works if Max pumps to run =1, and the level for the lag pump is reached, is the running pump stops and the other pump takes over.

Technical Note

The way the logic works by default, in high inflow conditions, the jockey pump, P1, will start each cycle, and then the larger pump will take over. If you don’t want P1 to always start, add some custom logic in IsaGRAF, or the logic engine. That logic would be very simple – once Group 2 starts, hold out Pump 1 (or Group 1), and then when the calculated inflow drops below a certain value, remove that hold out fault.

Summary of Configuration

Create a 3-pump station using the setup wizard
In Settings – > Alternation & Grouping:

• Create a new group
• Move P2 and P3 into Group 2
• Check Group 2 is in “Alternate (Std)” – the default
• Check that Group Alternation is “Fixed (Std)” – the default

Configure your setpoints as you want through Settings -> Setpoints -> Level/Control Setpoints
If you do want only one of the large pumps to run at one time, configure Max Pumps to Run in Group 2 – Settings -> Advanced – > Pump Control -> Group -> Group 2 -> Max pump running Read more…

In the well efficiency, high inflow, lift station

If you are relatively new to DNP3 you might under-estimate the value of certification.

Compared with a protocol like Modbus, DNP3 has way more features. That’s another way of saying that it is a lot more complex.

Modbus compliance is very easy to test. Suppose you have an RTU with Modbus Slave – a unit which will be polled by a master PLC or RTU. To test it, you can download any one of a number of free or demo Modbus tools and check that you can read a set of registers and write to a set of registers. That’s not the complete functionality of Modbus but it does confirm that the basic functionality works – and because the protocol is so simple, the product vendor is unlikely to have got it wrong. One tool readily available is the Kepware OPC server and you can download a demo version from their website.

As the end-user or system integrator implementing DNP3, if you have to put together a master from one supplier and a slave from another, when it doesn’t work who do you call? Or if you are taking advantage of the fact that DNP3 is an open protocol supported by many vendors you might be choosing a number of different field products – each one suited best for the application at hand.

DNP3 has the flexibility you need for reliable, secure telemetry but there’s a lot of features to test.

The smartest approach is to get the vendor to do it for you before you start using the product or software. If you ask for an independent certification you know that the protocol has been tested by someone who’s reputation is on the line if they miss something.

To get an idea of what gets tested in DNP3 certification you can see a copy of a full test report on the MultiSmart RTU on our main site, under the product manuals section of MultiSmart – www.multitrode.com/pump-station-manager/product-manuals.html Read more…

SCADA & Telemetry DNP3, RTU, SCADA, telemetry

Some people might be interested in the latest MultiSmart brochure now on the main website.

You can find it under the MultiSmart section of the website, at www.multitrode.com/pump-station-manager/data-sheets.html

Some people prefer to read a “real paper” version, so you can always ask your local MultiTrode sales office to give you a copy.

We like to encourage less paper but the reason we printed a few thousand is that we know it’s not an easy shift to have everything as a soft copy and not print it out.

Multismart-brochure

General News

This continues from the earlier DNP3 posts -

Part One: Date/Time Stamping – or Less Guessing
Part Two: Communications Options – Polling and Unsolicited Reporting
Part Three: Security

The DNP3 protocol also supports guaranteed delivery. What does this mean?

Suppose you want to send a command to start a pump. How do you know the RTU at site received the command? With some older and simpler protocols the only way to check is to read the status of the pump at a slightly later time – and hope you catch it in the act.

Or suppose you want to ensure that the message High level alarm or All pumps unavailable sent from the RTU was not missed by the master station or SCADA? With some protocols, like Modbus, there isn’t any mechanism for ensuring this.

DNP3 provides message acknowledgements. With unsolicited reporting, the RTU might send all changed data every half hour, or if the event buffer was full. The “message” that the DNP3 protocol sends includes all the tags that have changed with the date/time of each, and also includes a sequence number. The master station would send an acknowledgement to the RTU – or “outstation” – that that sequence number had been received.

In the event that the RTU / outstation didn’t get that confirmation, it would retry. And after a certain time period the site would go into a Comms Fail mode with probably a longer retry delay. I say “probably” because that depends on how the user sets it up, but that would be the sensible approach.

As you can see if you’ve been following this series on DNP3, the creators of DNP3 designed it for the challenging world of telemetry where communications is always suspect and often problematic.

There’s more to configure in the protocol of course, but each element is there to ensure data integrity:

• you know what happened
• exactly when it happened
• you can guarantee that the SCADA system knows about it
• and you can ensure that data is genuine and not from a hacker

Read more…

General News DNP3, reliability, RTU, SCADA, telemetry

We only launched the blog a month ago, so for people following the blog it seems worth doing a catch up on recent history.

v2.0 of Multismart was released in July 2008 and came with some major enhancements.

PLC functionality

IsaGRAF v5 was added in as an option to the product. This means that a full IEC61131-3 compliant PLC extension is available (all 5 languages).

How does it integrate and when is it needed?

First thing to mention is that the code behind MultiSmart is NOT written in a PLC language. It’s in C++ for a whole bunch of reasons that I hope to get our software team to elaborate on at a slightly later date.

What MultiSmart does do is make ALL of the many tags (1000’s) available to the PLC module for reading and writing. This then allows a PLC programmer to extend, change or add totally new modules, without having to rewrite the complete application or module from scratch.

Praveen wrote a post recently about a customer who wanted to match dosing of SulfaLock to the flow rate for odor control. It’s such a custom requirement that you wouldn’t expect to find it in the standard list of MultiSmart functions! So Praveen developed it for the customer in IsaGRAF. The customer or his SI could have done the same, so it’s not dependant on MultiTrode engineering staff writing the application. Of course, we like doing custom applications because it’s a great opportunity to learn more.

Another PLC application would be testing a high level alarm in a specific way – turning the pumps off periodically, letting the level rise, and ensuring the high level alarm actually operated. That’s a function that we might include as standard at some stage, but I’m sure you get the idea.

How might that work?  The PLC application would interface to the pump controller module - maybe based on date/time and certain other conditions being met (low flow rates, no pump alarms last 3 days), and turn the pumps off, then monitor the level to where the high level alarm should activate. If it does activate, start the pumps, send a “High level alarm check ok” signal to SCADA and reset the high level alarm. If it doesn’t activate, still start the pumps (!), send a “High level alarm FAIL” signal to SCADA and reset the high level alarm.

There is a lot of functionality in MultiSmart so a challenge for us is educating our customer base as to what’s there so someone doesn’t spend a week or two rewriting something we have already done and tested..

Another enhancement in v2 is to be able to view all tags on the LCD screen, including IsaGRAF tags and the entire tag database (go to the Info screen on MultiSmart) – makes on-site troubleshooting a lot easier!

DNP3 Security

The new standard for DNP3 security was out so we included it in MultiSmart v2. There’s a big concern about security in SCADA communications so we moved it to the top of our queue. You can find out more in this DNP3 Security post.

More Expandable I/O

We added some off the shelf I/O blocks and integrated them into the user interface. Why 3rd party I/O modules? Standard I/O is pretty much a commodity these days so there are plenty of great modules out there. We added two Adam units from Advantech, and in 2.01 added two UL-listed Acromag units. They connect on Ethernet using Modbus TCP protocol.

What we’ve done in MultiSmart I/O is focus on specific pump station requirements, so even though the most common I/O card you find in MultiSmart has 2x 4-20mA Analog inputs, 1x 4-20mA Analog output, 7 Digital outputs and 20 Digital inputs – there are some extras. The Digital inputs as default are volt-free contact closure inputs but can also be configured to read Flygt FLS, PTC thermistors, seal sensors, conductive probe inputs (for the MultiTrode probe), and a few can be configured to be Flygt CLS or high speed counters. The card also includes 3 inputs for reading phase-to-phase supply directly – up to 600v input.

And the energy and pump efficiency card reads 3-phase currents, and provides insulation resistance. Reading all this I/O usually means you have to buy lots of parts to do the signal conditioning. So most people don’t do it and spend lots of time driving to site to see what’s going on..

All of these pump station specific I/O makes the control panel, simpler, more intelligent, smaller – and of course lower cost.

But when it came to another card for 8x standard AINs or 16x standard Digital Inputs we thought why not just a card from someone else?

Import and export of DNP3 and Modbus tags

You can now export the DNP3 and Modbus tags as a .csv file and save it onto the CF card – or read it via ftp. And you can re-import the tag list back after modifying it.

This matters because as a default there are 400-500 tags in MultiSmart and if you want to play around with that configuration it takes a while through the user interface. Also you might want to synchronize data with a SCADA configuration file or master PLC.

Energy Monitoring and Energy Saving

v2 includes “Run the most efficient pump”, also known as “Alternation by efficiency”

This allows automatic saving of energy. You set a parameter for the ratio to run – e.g. run the most efficient pump 20:1 compared with the other pump(s). If it just ran the most efficient pump, the system could never get an opportunity to recheck – or to check the other pump runs ok.

How much do pumps drop in efficiency? I.e. how much can you save? It’s one of those values that people rarely know. As one major pump supplier said to us, “We all know the pump curve the day it leave the factory, but 1 year, 3 years later, what is it like? No one knows” They showed us a study they had done but it’s confidential. We did put some studies that we found on the Energy & Environment page on our main site, which indicated that wastewater pumps were often more than 15% below their original efficiency. Even clean water pumps can degrade significantly – always a surprise.

So the Run most efficient pump feature gives a change to automatically and easily save energy costs.

Other features

There’s a couple of other minor features:

• Digital Output pulse and delay functionality
• Support for relative level

How do I get v2?

Easy. I’m assuming you have a MultiSmart of course. If you don’t, and you have a lift station, try one out!

If you do, you can download the firmware for FREE from the multitrode.com website. You can find the link under the Training and Support section at http://www.multitrode.com/firmware-upgrades.php

You will need to signup on the site. If you don’t already have a login, just click the Signup button in the top right (of the main website), and check the box – Please give me access to the free MultiSmart firmware upgrades & release information. If you do already have a login and haven’t been in the MultiSmart firmware access section before, make sure you are logged in and click the Edit my account details link, then check the MultiSmart firmware access box.

We process that request manually so it usually takes a day, sometimes less - but if you are in a hurry you can always call up your MultiTrode office and speed things up.

How do I find out about New Versions?

Easy. Follow this blog. If you haven’t already subscribed just click the Subscribe by RSS    or Subscribe by Email icon in the top right. Find out about Subscribe by RSS.

Alternatively, or as well, to make sure you don’t miss anything, in the instructions above (”How do I get v2?”) about editing your login details to the MultiTrode site, you can check the box Please update me on email with all new releases of software or hardware in the MultiTrode family of products (MultiSmart/PumpView/Outpost2).

If you are a MultiSmart customer it’s well worth finding out the details of all the new features. It’s free to upgrade!

More about some minor MultiSmart updates since v2 in a later post.. Read more…

General News dnp3 security, isagraf, plc, pump efficiency

The magazine Control Engineering ran their monthly Information Control email today which included a tutorial from MultiTrode and Parasyn:

www.controleng.com/article/CA6640186.html 

They describe the tutorial:

SCADA systems aren’t scalable out of the box; you have to plan with the final footprint in mind,” says Tony Poole, managing director of Parasyn, a system integrator specializing in water/waste water applications. Steve Carson is with MultiTrode, a manufacturer of MultiSmart pump station manager units, which are replacement devices for pump controllers or PLCs/RTUs for lift stations. The devices add more monitoring and control capability to SCADA systems, and can also add 400 to 500 tags (data points) per site. In this tutorial, Carson and Poole provide best-practice advice for designing large tag count SCADA systems so they are manageable.”

Control Engineering  have a number of email newsletters that you can subscribe to. There’s plenty of good quality articles that make it worth the free subscription. Just visit http://www.controleng.com and you will see the Newsletters menu item on their site.

There’s a story behind how we came to write that article together, a subject for another day.

We find the subject of large tag count systems very interesting – we’ve run into it a number of times and in different elements of the SCADA solution. Parasyn’s approach made a lot of sense when they explained it to us and we certainly learnt from their experience.

“SCADA systems aren’t scalable out of the box; you have to plan with the final footprint in mind,” says Tony Poole, managing director of Parasyn.

Have a read of the article. It will be in the print edition of Control Engineering  in April. Read more…

General News large tag count, reporting, SCADA, telemetry

We’ve been working with Flovac for quite a while. Our relationship started off with them using our older controller - the MT2PC (actually a pair of MT2PCs) for their vacuum pump stations. They’ve now standardized on the MultiSmart Pump Station Manager and there’s a News item on the main MultiTrode website.

There’s a great 3 minute video about how vacuum pumping works on their home page, www.flovac.com

John Radinoff, the Flovac Managing Director, explained to me why vacuum pumping is gaining ground: ”These systems are used in areas difficult to sewer using traditional technologies. Ideally suited for areas with high water table, rocky sites or flat ground, the system has proven to be extremely economical in such topography. 
Eliminating infiltration and exfiltration, vacuum sewerage is highly regarded by many EPA’s around the world and has proven to be the sewerage technology with the lowest carbon footprint. This is the reason why it is the naturally preferred choice of sewerage in high profile sustainable projects like Masdar in the UAE.”

It’s clear that gravity sewer is really expensive when you have  a high water table, rocky terrain or very low density housing.

There’s a good Wikipedia article about vaccum pumping.

Pressure sewer technology covers a similar list of advantages over gravity sewer.

For MultiSmart, the Flovac application is a 4-pump 2-well configuration. The “wells” are independent. On one side we are measuring the vacuum pressure and based on that starting and stopping the pair of vacuum pumps. On the liquid side, we are measuring the liquid level in the well and starting and stopping the wastewater pumps. Read more…

General News environmental, user interface, vacuum pumping

In Florida, and probably in lots of other areas that face power outages regularly, there are lots of generators on lift stations.

One of the requests we had was for the capability to limit the number of pumps that could run when in generator mode. We incorporated this into the Profiles feature of MultiSmart.

What’s a profile? In MultiSmart, a profile is mainly about setpoints but includes a few other parameters for pump control.

The idea behind setpoint profiles was to make it much easier for a water or wastewater utility to switch between different setpoints at different times or under specific circumstances. There were many requests for changing setpoints at different times automatically, for SCADA control and for operator intervention when on site. In another post we’ll look at spill management using profiles, which is another great use for the function.

Because of the requirement for Generator Profile, or GenSet Profile as it’s known by default in Multismart, we incorporated some pump control functions as well.

• max pumps to run
• max run time
• max off time (usually used for odor control in wastewater, but has other applications)

This allowed a utility to have a 3-pump station where all three pumps could run normally, but under a generator-powered condition they could only run two pumps. It would be just as easy to have a 3-pump station which could normally only run two pumps, but under generator load, only run one pump. MultiSmart simply has a parameter called Max pumps to run which applies to each profile, including the default profile. You simply change the parameter for each profile (or leave it at the default which is “no limit”)

We also setup this profile so that by default the setpoints were a lot lower in the well, but this is easily changed through the setpoints menu.

Changing setpoints on MultiSmart

How do you switch to this profile?

Generally profiles can be switched by:

• Digital input
• SCADA control
• Date/time (there are 4 timers for each profile)
• Operator interface
• Customer logic, e.g. from the PLC extension to MultiSmart

So in the case of GenSet Profile you would probably configure it to become active when the Generator Transfer switch activated.

By the way there are 6 profiles in MultiSmart. Two of them are labelled up already: spill management and genset. However, if you wanted to use profiles and didn’t these you can change them –  any profile can be renamed and configured exactly how you want it.

Can the setpoints be changed via SCADA?

Yes, as well as switching profile via SCADA, all of the setpoints for each profile can be read and written to by a SCADA system.

How do you configure ”Max pumps to run” in a profile & when a Profile will be selected?

Because it’s a relatively advanced feature, not surprisingly it is under the “Advanced” menu. 

Locating the Advanced screen in MultiSmart

 Press Settings – Advanced and expand out Pump Control to select Profiles. Then you select the profile you want to configure and all the options are in there, including the pump control options, selecting a Digital Input on which to make it active or choosing timers.

In another post I’ll look at the Spill Management Profile.

Read more…

In the well generator, genset, profiles, setpoints

Our White Paper on SCADA reporting also appeared in Pumps & Systems – for those who don’t know it, an excellent US industry magazine. You can find the online version of the magazine at www.pump-zone.com.

What’s the White Paper about? Essentially it looks at where SCADA and telemetry reporting currently stands for a typical lift station network – compared with where it could be – and where the utility asset managers and operations managers would like it to be.

Operational Cost Breakdown

Anyone who has implemented a SCADA system, or even played with a SCADA package can tell you how easy it is to create an animation on the HMI that depends on a data point. For example, displaying a well emptying and filling as the level “tag” falls and rises in value. Or a motor changing from blue to red as the temperature tag increases in value.

It’s probably the first thing that you get taught on the SCADA course, or get shown by the salesperson when he/she demonstrates ease of use of that SCADA platform.

Reporting is a whole different question. There is the challenge of finding where the real data exists in the system, understanding data integrity and getting to grips with the reporting interface. But the bigger challenge for many end users and SI’s is that creating a good report means getting to grips with relational database concepts, and that’s a lot harder than linking a process value to a graphical animation.

As Paul Francis, MultiTrode CTO, said:

Reporting has always been a challenging aspect of any SCADA platform. Visualization and trending tools were some of the earliest adopted elements of SCADA systems and hence matured long before reporting interfaces did. Developing useful reports can also be more of a technical challenge than making a nice graphical interface for your plant or collection system; in part due to the nature and structure of the underlying data.”

Away from the mechanics of reporting, there are other challenges. For example, data acquisition in the field. Typical PLC and RTU systems aren’t bringing 100’s of tags back to the SCADA system. That requires a level of design that is rarely catered for. With minimal data, it’s hard to generate much in the way of reports – and this may well have contributed to the low expectations of asset managers.

Reports - Pump Efficiency Changes

And in the US, the de facto standard for water and wastewater telemetry protocols is Modbus. This works against any attempt to get asset rich data. (See, for example, DNP3: Part One or DNP3: Part Two).

None of this means that it’s too difficult to get reports that will cut operational costs or provide asset reports to allow better capital allocation. It’s just important to understand the different roadblocks that lie in the way.

If you want to read the SCADA Reporting White Paper you will have to fill in a short registration form (unless you already have a MultiTrode website login). Read more…

SCADA & Telemetry reporting, RTU, SCADA, telemetry

The story starts with some rats, quite a while ago.

One of our customers came to see us and together we had a few technical discussions about how to improve the single sensor probe and the MTR relay as an independent high level backup system.

The problem they had encountered was that rats had eaten the probe cable in a number of sites. When the probe isn’t covered by conductive liquid (e.g. wastewater) it registers as an open circuit. When the cable is broken, disconnected, or the target of hungry rodents it also registers as an open circuit. So the high level wasn’t picked up – a major problem.

Their idea was to bring an extra wire from the sensor in the level probe to the alarming relay and have the relay carry out a continuity check.

It seemed like a great idea. When we talked some more about the idea we first proposed one alarm output for both high level and for loss of probe. I think we were trying to work out if we could re-use most of an older product for the new design. That idea didn’t work for them for good reason – their PLC/RTU control system would start the pumps on a high level alarm and run them for a set period, but if the alarm signal was only indicating loss of probe, the well could be almost empty and they might run the pumps dry and airlock them. Then lots of work getting the pumps into service again!

So the product concept became a level alarm relay with two outputs – one for loss of probe, and one for high level.

Like many product companies we then had lots more ideas for other variants, and with all the MultiSmart work we were doing at the time, the product took a lot longer than it should to make it out the door.

Well, we released the SafeSmart level alarm relay today. The part number is Safe-FS.

Read more…

General News failsafe probe, high level alarm, safesmart

A reminder for those who aren’t aware – if you have a MultiSmart you can get FREE software upgrades for the life of the product. This applies to any end-user or system integrator or contractor working for the end-user.

If you haven’t already registered on the MultiTrode website, all you have to do is click on the SignUp box in the top right on the main multitrode.com site:

When you get the member registration form, don’t forget to select the box which says: “Please give me access to the free MultiSmart firmware upgrades..” – see below..

If you have already registered on the new multitrode.com website and didn’t select this option at the time - but do want access to MultiSmart firmware, just send an email to webmaster@multitrode.com requesting an upgrade – best to include a phone number where we can contact you.

Specification Document Update for v2.0.4

Part of the reason for this post was to let people know that the specification document has been updated for MultiSmart.

The specification document generally has two main audience groups – those who want a chance to review the main features of a product – to understand what the product offers; and, those who want to write that functionality into a specification for a bid document.

The new MultiSmart spec document is relevant for v2.0.4 onwards of MultiSmart, and includes functionality like “locked level detection” - a subject of a future blog post.

v2.0.4 was released in December 2008. If you don’t have a copy and you are a MultiSmart end-user, or an SI or contractor for an end-user, please register on our website as described above. 

How can you get hold of the spec document? Visit http://www.multitrode.com/pump-station-manager/specifying.html to find the latest pdf and word copies. 

If you are looking for specifications for any MultiTrode product, simply navigate to that product by clicking Products & Software, then find the product you are looking for and select the Specifying link.

And if you stumbled across this blog post and want to know how to get future blogs without having to remember to visit the website 10 times a month – simply subscribe. If you are already reading this blog post in a reader (like Google Reader or Feedburner) then it looks like you are already ok. If you are reading this on our website, just click on one of the two links at the top right of this blog – subscribe by email – OR, better, subscribe by RSS (less clogged email inbox). Read more…

General News Add new tag, Technical

This post continues the themes from Part One and Part Two.

The subject today is security, and also why proprietary protocols aren’t the answer.

Security in communications is a hot topic, but in practice in the water and wastewater industry, not many people are actively implementing it.

It’s important to differentiate between “hacking the comms” and “hacking the server”. If there is a greater problem for the organization, it’s surely someone hacking your server through a firewall –  or from within your building – because now they can take control as well as present the operations staff with a completely false worldview.

However, if the SCADA server is highly secure and someone was very motivated to take control of your system, then they could potentially do a lot of damage by hijacking the communications. Imagine if they turned off all of the sewer pump stations in a city? You can send your staff out to put every station into manual over-ride – but only once you knew about it, and it would take some time to get to every single station. You would have lots of overflows, and you would have your whole team racing around from station to station. If it happened in a time of high inflows – e.g. a storm – then the problems would be much worse. In a water supply system it might be possible to burst pipes.

There are a lot of articles about communications security that start: “Since 9/11″ – probably because it gets higher exposure. But how much of a risk is it?  And is the risk greater from other sources than terrorism – like disgruntled ex-employees? It’s certainly getting attention from governments, but not much practical attention from the utilities themselves.

This article doesn’t try and address the risk factor. Instead, we’ll just explain a little about how communications to remote sites can be secured.

Security in Communications -  Are proprietary protocols the answer?

One subject that the promoters of proprietary protocols majored on in recent years is security. This is because they didn’t have a lot else to hang their hat on.

What am I talking about? Open protocols have been the perceived way forward for a long time, but especially in this century/millenium. For the last 5-10 years in the water & wastewater industry, almost anyone writing an engineering spec, or an operations manager or utility director who had done a small amount of research, knew that you needed to specify an “open protocol” for a new or upgraded system.

This presented a challenge for a number of companies who had their own protocols in their RTU and used these protocols to lock in customers. What to say to show they were progressive?

“At least no one knows how to hack our protocol, that’s an advantage..”

By the way, I’m not including in this list, companies with their own protocol who made them public. There are many companies, including ourselves, who in the 1990’s had their own telemetry protocol in their RTU because it seemed – rightly or wrongly - to have some advantages at that time. The important point is, once the move towards open protocols became desirable or a requirement, and high quality telemetry protocols became available, what did those suppliers do? The responsible ones published their protocol and made it easy for other parties, including competitors, to copy them.

In fact, most proprietary protocols aren’t that hard to reverse engineer.

In the world of encryption and authentication, the experts will tell you that openness is what allows the audit. Don’t tell the world that your protocol is “secure” because it is proprietary, unless you have invited a few hackers to break it. It probably won’t take them very long.

A good recent example is where one of our partner companies, Trihedral, reverse engineered a proprietary protocol from another supplier to allow them to break into their market – to replace the SCADA server software while still interfacing to the RTU’s in the field.

Only last year (2008) I read an article in a water industry magazine by a supplier saying how their RTU protocol was more secure than DNP3 because it wasn’t published..

Time to move on..

DNP3 Security – How Does it Work?

Security is one of those tricky subjects that most people actually don’t want to understand. As a user you just want to know it works. So I’ll stay away from the more technical aspects.

DNP3 is a published protocol with a very strong and a very technical user group. You can be sure that the people in the user group who published the security specification knew what they were doing.

Very simply, DNP3 security doesn’t encrypt the message, it authenticates the message.

If someone intercepted a command to an RTU: ”turn on Pump 1″ which might look like “digital tag 15 ON”, they could read it!

But if the bad guys then wrote a command to send to the RTU - ”turn off Pump 1″ – “digital tag 15 OFF” – the DNP3 authentication mechanism would reject it. The security mechanisms in DNP3 can determine when the command is a valid one by a trusted party.

This gives an insight into why the oil and gas suppliers might want RTU encryption, not authentication. In a highly competitive commercial environment you don’t want others to know how much volume you pumped  -for example.

In water and wastewater, even in the privatized but regulated market of the UK, it’s hard to see how anyone reading your pump station commands could cause you a problem.

The key point of DNP3 security is that while others can see what you are doing, they can’t pretend to be you and tell your system to do the wrong thing. That’s what authentication means.

If you want to know more, please take a look at our White Papers section. You need to fill in a short registration form to download any of the papers. The title of that paper is “Keeping SCADA systems open and secure from cyber-attack”.

There’s a lot more technical data available – check out the DNP3 Users Group.

And if you’re one of those people who like to understand more about “how everything works” – for the confusing world of encryption and authentication, a personal recommendation is Cryptography Decrypted by H.X. Mel & Doris Baker. It was published in 2001 so it might be a little dated, but I found it made subjects like public key encryption finally understandable. Read more…

SCADA & Telemetry DNP3, RTU, SCADA, security

We recently did a survey of Florida water and wastewater utilities. Here is the pdf of the survey.

One of the surprising results from the survey was about energy cost and its importance:

The surprising – and encouraging – aspect was that 47% already rated Energy Cost and CO2 emissions as very important. It wasn’t surprising to see that 85% thought that although it would increase in importance, other challenges would still be more important. Challenges like ensuring sewage doesn’t spill out into the streets, that it is properly treated before being discharged and that potable water is supplied to the residents and customers of the water authority.

Perhaps the UK water utilities are fundamentally different because they are so much larger – typically 1000 – 5000 pump stations – and not government owned. Or perhaps they felt that although keeping sewerage off the streets and drinking water in the taps would always be their no 1 priority – it wasn’t their biggest challenge..

 Pump Efficiency

A while ago we did some research on energy efficiency in lift stations and found that pumps can lose a lot of efficiency even in clean water. Obviously wastewater is worse and the level of efficiency loss depends on the kind of material being pumped – if you have sand, grit and rocks going through the sewer main it’s obviously going to be worse. The Hydraulic Institute said:

“A pump’s efficiency can degrade as much as 10% to 25% before it is replaced, according to a study of industrial facilities commissioned by the U.S. Department of Energy (DOE), and efficiencies of 50% to 60% or lower are quite common. However, because these inefficiencies are not readily apparent, opportunities to save energy by repairing or replacing components and optimizing systems are often overlooked.”

Now the reason a lot of water and wastewater utilities have started using the MultiSmart pump station manager isn’t because of its energy efficiency functionality! But the more progressive ones have definitely seen the potential of MultiSmart to reduce their energy costs. As the Hydraulic Institute said, “Because these inefficiencies are not readily apparent..”  – If you don’t know what’s costing you money, where’s the incentive, or even the budget, to start fixing your problems?

More on Energy & Efficiency

There is a section on our website – Energy and the Environment – which has a few references to energy efficiency studies. And there is also a paper in the White Papers section – note that you need to fill in a short registration form to download any of the white papers.

Read more…

In the well

We posted a news item on our main site about Trihedral and MultiTrode.

Background

This brief blog post gives a bit of background. Canadian-based Trihedral have been making inroads into the water and wastewater industry – our main market – for some time. From our perspective, we have seen them win a lot of customers in the SE of the USA. They might be winning customers in lots of other areas too.. we’ve just noticed the SE region.

The Florida market has had a major supplier who locked up the customer base with a proprietary protocol. What Trihedral did was reverse engineer the protocol. As a result, the Trihedral VTS platform can be implemented by these customers who can continue to use their existing field hardware but are now free of their restraints! They change their SCADA platform, keep their old hardware (so don’t have to change everything overnight), but they can start using new products.

These customers who have switched to VTS can now introduce any product they want in the field – or the plant – so long as it supports open protocols like Modbus or DNP3. 

The Partnership

That’s the background, but the news item is about what Trihedral have done, in partnership with MultiTrode. VTS will shortly have an “add site” function for MultiSmart.

This function essentially removes a lot of the legwork, or integration, out of bringing a MultiSmart site with 100’s of tags (data points) into the VTS system.

The new version of VTS isn’t quite out yet – expected in February. But I saw a demo of it a few months ago and was very impressed. The MultiSmart pump station manager was on an ethernet link (could have been a radio link) to the VTS SCADA server and the whole process was automated.

The remote MultiSmart site created a compressed version of the XML configuration (we use XML as standard for all configuration files), transmitted it using DNP file transfer (industry standard), VTS loaded it, unzipped it, and presented the user with a few checkbox options for configuration – including graphics, alarms and controls.

I know some of the Florida customers are very keen to get their hands on it. And some of our engineers are very interested in what Trihedral have done.

2+2 is more than 4 – if you choose the right combination

That’s the beauty of working with other companies who are great at what they do and have a passion for making things better! New ideas that spur our engineers on to come up with even more innovation.. and make life easier for the customer. Read more…

SCADA & Telemetry Add new tag, DNP3, MultiSmart, SCADA, telemetry

This follows on from the first blog post on DNP3 where we covered Date/Time Stamping – or Less Guessing - and a little bit of a comparison with Modbus.

This post is about some of the mechanisms for communicating between the SCADA, or master station, and the remote site. By the way, in DNP3 speak, you often hear the term “outstation” for the remote site. We’ll tend to stay with RTU (depending on who you talk to it either means “Remote Telemetry Unit” or “Remote Terminal Unit”), which is essentially the device that communicates.

Polling 

Let’s start with a Modbus comparison again, just to give a little context. Modbus is a totally polled environment – that is, the master station (typically a PLC) requests data from a site. The remote site can’t choose to send some data to the master station, it has to wait until the master requests it. So a very typical arrangement for a collection system is where the master PLC polls each lift station in turn and when it is has finished, starts again.

This has a lot of disadvantages as anyone with a growing network of lift stations has found. The time taken to get around the network goes up and up. You might get to a point where it’s 15 minutes between polls and decide to prioritize certain sites to get polled more frequently, or add a radio frequency (and therefore a new base station and repeater radio) to split the network into more manageable pieces.

It’s not an ideal situation, because Modbus isn’t an ideal telemetry protocol. (It’s a great protocol for other applications).

Unsolicited Reporting

If waiting until the master station asks you how you are doing isn’t the best way, what about unsolicited reporting. What’s unsolicited reporting? The remote site, also known as the RTU, sends data to the master station without being asked.

If you are used to Modbus polling systems, this might sound like anarchy. And if you have every site sending a message every time an event takes place, it could well be anarchy. Will the radio network stand up to lots of sites all trying to communicate at once?

The only way you might be able to prevent chaos, is by having a very limited amount of data getting sent, or a high radio bandwidth. One of the guys in MultiTrode told me about a system he was involved in with a previous company where they used unsolicited reporting for every event, and during a major storm the entire network shut down due to “collisions” in the radio network. The solution was for someone to go and visit every site and reset each RTU. Then the network started communicating again. But they lost all the data for their critical event.

That’s not great. What’s the solution?

Communication Choices and Different Classes

The DNP3 protocol has some great features to avoid the problems above.

Firstly, you can group events into different classes. Then secondly, you can choose how those different classes communicate from the RTU to the master station.

An example is the best way to explain it. Suppose you want to know about all high level alarms within 1 minute, and otherwise you are happy for all stations to be communicating at least once every 30 minutes.

You would put the alarm “High level” into class 1, and everything else into class 2. Then you would set class 1 to unsolicited reporting immediately. And class 2 you might set one of two ways – either to report every 30 minutes or sooner if the event buffer was full; or you might have the master station polling every 30 minutes.

Now you can find out immediately about high level alarms without risking the network turning into treacle and you still find out about all the changes in your network.

It’s the flexibility in DNP3 that is one of its great features. As Paul Gibson, one of the key people behind the development of the MultiSmart pump station manager, said:

“If we didn’t have DNP3, we’d be trying to design a protocol just like it to put into the product.”

The Communications Network still needs Design

Just because there’s a flexible protocol doesn’t mean you don’t have to do anything. Every communications network needs design. How much data? What is the bandwidth? What are the delays?

Read more…

SCADA & Telemetry DNP3, Modbus, MultiSmart, RTU, SCADA, telemetry