Friday, June 17, 2011

Safe Door on a Screen Tent


So… you built the world’s most impregnable Widget. Good for you. Sadly, that means bupkis.

In todays installment of “Ranting ‘bout the Grid”, we will discuss the pitfalls of poor implementation. SmartGrid components are no different than enterprise components in many ways. It is completely within the realm of possibility to take and incredible architecture and design and totally FUBAR the implementation.

I call it “Putting a safe door on a screen tent”.

The first stumbling block is a having a security policy. Most utilities have one, it’s just that the line folks don’t know it exists because they have never had to deal with it. Until recent events brought to light the sheer inanity of using a single password for every meter, re-closer, synchrophaser, etc., most meter shops and dispatch centers have not had reason to be concerned with simple security measures. The simple fact is that a majority of utility security policies I have read are geared toward the enterprise and lack specifics for the field. That puts the policy into the realm of “Speculative Fiction”.

So, step one; update the utility security policy to address SCADA systems. That done, it follows logically that process, procedure, SoP, etc. need to be updated as well. When you do this, please engage a security professional who understands control system concerns. All too often, I see polices written with the best intentions by the uninitiated, and it leads to mis-interpretation, confusion, and ultimately fails. Remember a security policy is supposed to be like the law. It requires legislative (Board of Directors) approval. It informs and directs the process and procedure.

Great, you have a policy and you have a process to use it. Now lets look at the landscape for a bit.

Working for a manufacturer, I always try to emphasize the concept of having as many points where you can exercise security controls as is practical. Separate the control network from the enterprise network from the outset. The isn’t much data that needs to flow directly from a control system to an enterprise desktop and what data DOES need to move into the enterprise, like billing data, can be moved through a proxy or bastion host. The weakest point in any network is inside the perimeter where you have to count more on administrative controls rather than technical ones. All it takes is one Stupid User Trick ® (SUT) to bring a botnet to the office.

Beyond segregating the control network from the enterprise network, it’s a good idea to develop trust zones and control movement between them. One way to look at it is, the Head End (HE) is one zone, the WWAN another, the control components networks a third, and finally the control components themselves. By doing this, you can contain a compromise to a trust zone and act appropriately. Control components represent the largest attack surface most security professionals have or will ever deal with. Ask yourself, how much do you trust the traffic from an area like this? Not so much. With that in mind, you need to assure that the data flowing into the system has appropriate confidentiality and integrity. Crypto controls like DSA and AES help with this, but only if applied appropriately. If you have a good design, a hacker compromising one control component has accomplished just that. One component. Granted, if that one component can dump several hundred mega-watts, it’s still a Very Bad Thing, but at least it’s not ALL of that type of component that is now compromised. The WWAN represents the next zone, and while it has a smaller attack surface, it is a higher value target. Adding application and packet firewalls as well as IDS/IPS at the entry/exit points of this zone will help ensure that if someone effects a compromise at the carrier or on a Metro Wireless network, you still have a barrier. In theory, a well-designed system will have all the control data encrypted and signed, so a compromise of the backhaul SHOULD only allow DoS type attacks. There is a lot of “SHOULD” in this world.

In the next installment, I will discuss security at the Head End, Personnel Vetting, and Responsible Patching. 

Thursday, June 16, 2011

Security Testing and the SmartGrid (or why we need to pull the industry's head out of it's butt)

*DISCLAIMER*
I work as the Head of Security Research and Testing for one of the major SmartMeter/AMI vendors, so that will naturally slant my views. Deal with it.


I just love going over security assessments with vendors in the SmartGrid and SmartMeter space. I have been doing this for a few years now and it never fails to amuse me when I get the response, "But, you are not supposed to do that with that interface!" Golly Mr. (or Ms.) Developer Person, you are indeed correct, however security testing is about seeing what I CAN do with an interface, not what I SHOULD do with it. This seems to come as a shock to some folks.

Here is a news flash for hardware, firmware, and software developers in the SG/SM space. Security assessors are coming to mess your stuff up. We will take the fruits of your labors and we will rip them open, probe, prod, shock, dissect, analyze, under-volt, over-volt, glitch, x-ray, de-solder, re-solder, JTAG, ADA-Pro, and generally do rude things to them. We will then write smug reports about how easy it was to break into your stuff.
There are two things you can do here;

  • You can either whine about how hard it is to secure this or that feature and how no one would EVER try that! 
  • You can step up and find ways to plug the many, many holes (sieve like in some cases) so we can retest it prior to going to production installs.


Security, when approached properly, is a matter of balancing risk and mitigation. The power industry is a tricky place to play games with assessing risk however. When a utility make a risk decision, it can impact it's neighbors. When a manufacturer makes a risk decision, it can impact the entire industry. Think about it this way; Widget, Inc decides that incorporating security features in a meaningful fashion into its product has a significantly negative impact on the sales margin for that component (because in a competitive market like this, security never adds to the price, only the cost). Widget's component is installed in a small part of the distribution or transmission grid. M@D_sk11ls_Skr1p7_M@s73r decides, just for LULZ, to attack this component and publish results. Next day the headline reads,

"SMARTGRID HACKED!!! ARMAGEDDON AT HAND!!!"


Of course all vendors, manufacturers, utilities, regulatory bodies, and consultants are now considered Satan's lackeys. All because security features don't increase shareholder value.

There is good news in all this. Equipment manufacturers are stepping up and being responsible (my own employer being one that has been doing this for some time) and implementing Secure Development Lifecycle programs. This is bringing quite a bit of gear under the scrutiny of security researchers and assessors and that is a Very Good Thing (r) in my opinion.

The bottom line? Test your product. Hire an in-house team to do this, and then send it to one of the reputable labs for MORE testing. Find the flaws before you ship because you can be assured that if you don't, someone will. AFTER you ship.

Friday, September 17, 2010

The new rules... umm, sort of... but...not really...

NIST-IR 7628

So big it needs three PDFs to contain it...

Lots of words, any value?

Let's hear your thoughts!

http://csrc.nist.gov/publications/nistir/ir7628/nistir-7628_vol1.pdf
http://csrc.nist.gov/publications/nistir/ir7628/nistir-7628_vol2.pdf
http://csrc.nist.gov/publications/nistir/ir7628/nistir-7628_vol3.pdf

Thursday, July 1, 2010

New rules in California

http://info.sen.ca.gov/pub/09-10/bill/sen/sb_0801-0850/sb_837_bill_20100622_amended_asm_v93.pdf

(scroll to page 7 and read section 8364.5)

It looks like California is beginning to pay attention to SmartMeters and security. Starting in 2012, power and gas utilities that use SmartMeter/AMI systems will have to post the results of their security audits so the public can review them. There are some additional rules that apply to the vendors that state they have to provide the details of any encryption methods employed as well as provide the results of their own security audit of the system to be deployed.

While these provisions will add burden to the utilities and vendors, I am firmly of the opinion that they are long overdue. One of the major problems developing for the industry is trust. The public has an inherent distrust of large corporations and utilities, and the introduction of AMI has added fuel to the fire. Conspiracy theorists see Big Brother looking out at them from every meter, some (not all) independent security researchers have hyped vulnerabilities as precursors to the Apocalypse without any risk context. The advent of Time of Use and more granular billing structures has had a significant impact on some of the more economically challenged customers, and the press has taken all of this and sensationalized it to sell copy. By requiring these audits and other provisions, the CPUC has mandated that measurable results be published in such a way that they can be used to refute many of the claims that SmartMeters are inherently unsecure.

 I am sure the utilities and vendors will raise a ruckus about having additional regulation and about how this will cost them pile of money to comply. I am also certain that there are members of my profession who will cry foul about posting an audit that exposes the weaknesses of a system. I am going to take a stand against the conventional wisdom in the information security industry and say that this is a GOOD thing. The best way to make sure that your weaknesses are not published to the world? Deal with them! Fix the holes, tighten up the procedures, improve the process. That way when you have to publish an audit, you can publish something that has nothing to give you away.

Sadly, security isn't even a second thought in many cases, it is less than an afterthought. Because it is very difficult to quantify a return on investment from security, it is most often only given attention when is is breached, or someone points out that the emperor has a pretty transparent wardrobe. These audits make a business case to improve security.

Time will tell how these rules impact the deployment of AMI...

Friday, June 18, 2010

Wake up!!

Time to reactivate this blog. the only thing worse than no blog is a dead one.

Ask yourself a question;

How would you feel about your electric meter being controlled from the Internet? How about getting your Internet from your power meter? Sounds silly, right? Maybe not.

Over the last few conferences, I have gotten the impression that some utilities are looking for a way to reduce infrastructure costs and possibly monetize their proposed AMI networks. There has been a strong push toward higher bandwidth, IP to the meter, and even WiFi on the meter. This all smells like Internet via the meter to me.

I CAN see a reasonable commercial/business reason for this. Internet connectivity reaches most places people live these days so why not use this existing network and save the capital costs? Alternatively, if you are going to all the trouble to install this huge network, why not find a way to make it pay for itself? The Telcos and Cable operators did it, why not the utilities?

Why indeed...

There are a few darn good reasons not to. Consider the risks of combining a major control network with a public access medium. The obvious is that it allows remote access to the meter. I don't care how good you think your built-in firewall is, it isn't good enough to mitigate this risk. To an accomplished hacker a firewall is nothing more than a logging router that does NAT. Remote access to a large block of meters is a primary target. With this you can hold an entire city for ransom, wreak havoc, or even weaponize it for a nation/state actor. Next is the Denial of Service (DOS) attack. it doesn't even need to be directed or deliberate. Once the network used to control meters and collect data is overrun with the latest worm traffic, your expensive new AMI system has reverted to the functional equivalent of the old style electro-mechanical meters.

In coming days and week, I will offer fact, opinion, news, and trivia on the new power grid initiatives from an insiders perspective.

Stay tuned!

Robert

Thursday, March 25, 2010

Travis Goodspeed's Blog

Another plug for a security blog

Travis is the guy who found and published the PRNG weakness in the TI ZigBee chipset.


Travis Goodspeed's Blog

Cassandra Security

Just a plug for the guys at Cassandra

Check em out, well reasoned and written fact and opinion


Cassandra Security