In January 2015, Rapid7 worked with Jack Chadowitz and published research related to Automated Tank Gauges (ATGs) and their exposure on the public Internet. This past September, Jack reached out to us again, this time with a slightly different request. The goal was to reassess the exposure of these devices and see if the exposure had changed, and if so, how and why, but also to see if there were other ways of identifying potentially exposed devices that may have skewed our original results.
As you may recall, in the original study, we sent a TLS-350 Get In-Tank Inventory Report request (I20100) to all hosts on the public IPv4 Internet with 10001/TCP open. A device speaking TLS-350 and supporting this function will respond with something similar to:
OCT 1, 2015 6:07 PM <station number><station name> <streety address> <city/state/zip/etc> 12345 IN-TANK INVENTORY TANK PRODUCT VOLUME TC VOLUME ULLAGE HEIGHT WATER TEMP 1 REGULAR 4812 4771 4708 44.45 0.00 71.95 2 PLUS 3546 3507 5974 35.83 0.00 75.15 3 PREMIUM 3377 3344 6143 35.31 0.00 73.92
In this most recent study we completed on October 1, 2015, we repeated this request, but made the following additional requests:
- TLS-350 System Revision Level request (I90200). A device speaking TLS-350 and supporting this function will respond with something similar to:
OCT 1, 2015 6:18 PM SOFTWARE REVISION LEVEL VERSION 131.02 SOFTWARE# 346330-100-B CREATED - 10.01.20.17.44 S-MODULE# 330140-145-a SYSTEM FEATURES: PERIODIC IN-TANK TESTS ANNUAL IN-TANK TESTS CSLD BIR FUEL MANAGER PLLD 0.10 AUTO 0.20 REPETITIV WPLLD 0.10 AUTO 0.20 REPETITIV
- TLS-250 Inventory Report on all Tanks request (200). A device speaking TLS-250 and supporting this function will respond with something similar to the TLS-350 Get In-Tank Inventory Report response
- TLS-250 Revision Level request (980). A device speaking TLS-250 and supporting this function will respond with something similar to the TLS-350 System Revision Level response.
While there are literally hundreds of other TLS-250 or TLS-350 or other ATG TLS protocol variant commands we could be sending and attempting, the goal of these studies was to identify ATGs with unprotected dangerous functionality or sensitive information. Attempting more of these commands likely would have identified more ATGs, however these protocols are not well documented, and, like so many other IoT things, we aren't so sure how resilient they are to repeated poking so it is best to play nice. Plus, these ATGs are connected to tanks full of untold gallons of flammable liquid, so excess caution is not totally unwarranted.
When analyzing the data from January and October's ATG studies, each response to a particular request was categorized as follows:
- Good: response appears to be a valid, non-error response for the protocol in question
- Error: response appears to be a valid, error response for the protocol in question
- Unknown: unknown data was received after connecting and attempting request
- Empty: no data was received; either connection failed or no response upon connecting and attempting request
With this knowledge, we observed the following:
|Data Point||January Initial TLS-350||October Enhanced TLS-250/TLS-350||Notes|
|Default ATG TLS-250/TLS-350 port (10001/TCP) open||1712285||1070728||Our blacklist also increased by ~35m between these scans|
|Responded Unknown/Empty for all requests||n/a||1064225||99.4% of the devices with 10001/TCP open are not ATGs|
|Responded Good for TLS-350 Get In-Tank Inventory Report request||5893||5214||This shows the change between January and now most accurately, though it is caused partially by blacklist increases and predominantly by natural fluctuation.|
|Responded Good or Error for at least one TLS-250 or TLS-350 request||n/a||6502||This is rough representation of the number of ATGs that are likely directly exposed but not necessarily exposing sensitive data/functionality over TLS-250 or TLS-350|
|Responded Good for at least one TLS-250 or TLS-350 request||n/a||6483||This is a rough representation of the number of ATGs that are exposing sensitive data/functionality over TLS-250 or TLS-350 in some way.|
|Responded Unknown for all requests||n/a||5260||These devices are all likely not ATGs|
|Responded Error for at least one TLS-250 or TLS-350 request||n/a||804||These devices are all likely ATGs, all speaking TLS-250 or TLS-250, but our request was rejected an unknown reason|
|Empty for all TLS-250 requests, responded Good for TLS-350 Get In-Tank Inventory Report request, but Error for TLS-350 System Revision Level request||n/a||4||This response profile aligns with how GasPot behaves|
As you can see from the table above, things haven't changed all that much. While the one data point that can be compared dropped by ~13%, combined with the fact that the number is so small to begin with (~5-6k), that our blacklist grew by ~5% (+~35m), and that what is exposed where changes all the time on the public Internet, I view this as an insignificant change.
While the drop in exposed, vulnerable ATGs in the last 10 months is insignificant, one fact becomes readily apparent and should be alarming -- there are over 5000 improperly protected, IoT-style devices connected to tanks storing millions of gallons of flammable, valuable liquids all over the world. Jack Chadowitz, the community member we've been working with on this, recently presented his findings from this work at the 2015 ICS Cyber Security Conference in Georgia.
As mentioned in the original publication we did back in January, there are a variety of solutions to protect these exposed ATGs, including using VPN or firewall-based solutions or simply configuring a secure password. We cannot draw any conclusions about the number of ATGs protected by VPN or firewall-based solutions because it is assumed that these solutions would prevent 10001/TCP from being found open in the first place. Regarding the use of passwords as a solution, we can make a stretch observation. We assume that any device with 10001/TCP open that either did not respond (Empty) or received an Unknown response to all of our TLS-250 and TLS-350 requests is either not an ATG or is an ATG that is secured through other manners, from that we propose that any device that responded Good or Error is an ATG of some sort. We know that there were 804 devices with 10001/TCP open that responded Error, and there are several reasons why we'd get Error, including us sending an ATG request that the device just happened to not support or considered invalid, or if there was authentication configured. In other words, some of those 804 devices may be using authentication, but how many is unknown and likely few.
To aid in identifying potentially vulnerable devices on your networks, we've added a simplistic Metasploit module for detecting and interacting with ATGs. This was written and tested against GasPot, a honeypot simulating some ATG functionality, however it is likely to work on real ATGs as well; still, use at your own risk.
We welcome your feedback!