Pilz was hit with a ransomware cyberattack on October 13, 2019. In the following article, written by Mike Beerman, CEO, Pilz USA, discusses how Pilz not only survived the attack, but is coming back stronger than ever.
Surviving a Global Cyberattack - Company Culture Plays a Pivotal Role
Keynote Speech from Mike Beerman, CEO | Pilz Automation Safety, L.P.
Pilz was hit with a ransomware cyberattack on October 13, 2019. Pilz chose not to comply with the attacker’s demands because doing so would have rewarded their efforts and financed additional attacks on others. More importantly, Pilz chose to stand strong to guarantee their data and the data of their customers wasn’t further compromised, potentially leaving them vulnerable to future attacks.
In the following article, written by Mike Beerman, CEO, Pilz USA, discusses how Pilz not only survived the attack, but is coming back stronger than ever. Learn what you can do to help make your company stronger in any crisis.
Materials and examples include:
- The role of corporate culture
- Communication – who, what, when and why
- Gaining customer and supplier support
- Encouraging and fostering a creative atmosphere among employees
- Turning a negative into a positive
On average, there is a cyberattack every 39 seconds. That’s 2,244 attacks a day. That means, in the time it takes you to read this article on surviving a cyberattack, 31 companies have been attacked.
Let that sink in.
Here are some other rather alarming statistics:
-
On average, a ransomware attack – which is just ONE form of a cyberattack - occurs every 14 seconds – 6,171 times/day!!!!
-
57% of companies experienced phishing and social engineering attacks in 2019
-
The average time to identify a breach in 2019 was 206 days – that’s over 6 months!
-
The average lifecycle of a breach was 314 days - from the breach to containment – that’s almost an entire year!
-
And the statistic that can really make or break a company – the average cost of a cyberattack. Including turnover of customers, increased customer acquisition activities, operational/productivity loss, reputation losses and diminished goodwill, the cost is approximately $1.67 million!
It’s scary. Even scarier when you suddenly realize your company is a static.
Let’s go back so I can give you a short overview of what actually happened to Pilz.
Sometime in May of 2019, the first phase of the attack was initiated – infection. In general, the infection phase consists of Emails with malicious attachments or links, sent blind, with the aim that recipients click on them and thus let malware into the company. So, yes, we essentially let the attackers in.
Once in, THEY spied on us. THEY spent the next few months learning our IT structure … in many cases better than we knew it ourselves. During those months, not only did THEY go undetected, THEY learned how to beat us at our own game. This learning allowed THEM to write specific code to lock our data down so securely, it would be impossible for us to ever unlock, unless we paid THEM to unlock it for us. In Phase 4, the attack becomes visible – and at the same time, an encryption Trojan was activated automatically, globally. Data was encrypted and ransom demands were sent. At this stage of the game, almost all means we had to defend ourselves were turned off – for instance, our antivirus server was encrypted. Because this attack was specifically written to take us down, there were no store-bought solutions to fight off this attack and get us back to ‘normal’ in a somewhat timely manner.
October 13, 2019.
That was when Pilz found out that we were under a full-on, no-holds-barred, cyberattack. The monitoring systems of our web servers raised the alarm and identified the hacker attack.
At this point, we shut everything down – took it all off-line to stop the infiltration of the attack. We not only alerted the responsible authorities, we worked side-by-side with them fully utilizing our external IT security partner as well and began our forensics investigation.
They investigated where the attack came from, when it started and what had been contaminated. Pinpointing the date of the attack was very important for us to find out from what point in time we could restore our backups.
The ‘responsible authorities’ in Germany, this includes the BSI – Federal Office for Information Security. The BSI – which has the task of protecting the critical infrastructure in Germany and the federal government – didn’t actually help us, however because of the General Data Protection Regulation (GDPR), they needed to be informed about cyber attacks. The authority that actually helped us was the special cyber unit of the German Police department; who, by the way, praised us for the quick response to report our attack. Apparently, for many German companies, days pass before such attacks are reported, which can blur important traces for the investigators. At the moment, we cannot comment much about the results of the forensics, since the investigation is still ongoing. What we can say imperially, is that neither customer nor supplier data was stolen and we could not find any viral spread of the attack.
Apparently, there was a ransom request, but THEY obviously didn’t know who THEY were playing with – on a non-IT level, anyway. Pilz doesn’t take to threats too well. Any kind of threat, actually. In fact, we took on the cyber attackers the same way we take on our competition – we don’t give up, we don’t give in, we play to our strengths to win … for our customers and employees alike.
While most companies would just pay the ransom, and get on with business. Pilz is not ‘most companies’. Pilz chose not to comply with the attacker’s demands because doing so would have rewarded their efforts and financed additional attacks on others. More importantly, we chose to stand strong to guarantee our data and the data of our customer’s wasn’t further compromised, potentially leaving us vulnerable to future attacks. A payment doesn’t guarantee that the data will actually be decrypted, either.
We were now in full-on fighting mode!
For various reasons, some of you may not agree with our stance, and that’s ok. Just because we knew it was the right thing to do, we also knew it was not going to be easy. After all, all Pilz computers, worldwide had to remain switched off. Production sites came to a standstill. This fight was not going to be quick, and it very well could have been more expensive than what the ransom would have been, had we known the ransom amount. However, let me tell you WHY we knew it was the right thing to do for Pilz; why we knew we could survive by taking this stance.
Company Culture.
That’s how we knew we’d survive… and thrive again … as a company.
Pilz is a global safe automation company – with over 2,400 employees in 42 countries. We are family owned. For three generations, so far. Since 1948, Pilz has made a name for itself in the area of safety automation – introducing a two-handed relay in 1968, the first control system in 1969, the first programmable controller in 1974, the first compact programmable controller in 1978, the first E-STOP relay in 1987, the first programmable safety and control system in 1995, the first safe, open safety fieldbus in 1999, the first modular safety system in 2002, the first safe camera system in 2006, the first servo amplifier available for numerous safety functions in 2007, the first automation system for standard and safety in 2009, and the first internationally certified qualification for machinery safety in 2013, and many more ‘firsts’.
We have a strong, innovative history in this industry. For the past 72 years, and I have a good feeling it will continue for at least another 72, Pilz has weathered many a storm because of our integrity.
We’re still here, and we will continue to be here, because we do what’s right for our employees and our customers. From the top down. From our managing partners, Thomas Pilz and Suzi Kunschurt (that 3rd generation I mentioned) on down. We seem to all be cut from the same cloth. We are all following the same moral compass.
Knowing that all 2,400+ employees would back them up, Thomas and Suzi knew they were not going to pay the cyber attackers from the very beginning. They knew Pilzlers from around the globe, along with our customers and suppliers, would all work together to get through this ‘nightmare’, because it was the right thing to do.
And they were right.
So on that fateful day, Sunday, October 13, 2019, with Pilz in a crisis, and our global offices slowing opening up for the week, how do we get the message out to our employees?
Communication.
Think of the ways we communicate with each other in mass, all around the globe. Email is the first to come to mind, right? But nope, our email is part of our IT systems… which were down – even our webmail, can’t use that. Texting. Well, that may have worked if we had an emergency texting system in place. However, since emergency texting systems are usually controlled from the business systems, which were down, it wouldn’t have worked either. Phone calls it is!
Honestly, I’m not sure Thomas and Suzie even considered other forms of communication other than a personal phone call to each of the 42 subsidiaries. That’s just the way they are. They have always given that personal touch – with good news or bad news. By calling, they could personally convey the message with the proper thoughts and feelings behind it – something that couldn’t be done via email or text. They could be sure each subsidiary leader received the message as well as had an opportunity to ask questions – to actually have a conversation about the situation, not just a one way push.
I got the call from Thomas Pilz. Since October 13th is my birthday, I thought he was just calling to wish me a happy birthday. Which he did, but we also discussed the crisis.
Thomas gave me some insight into what was going on, so I could then relay the information down to our employees in the US. I then, immediately contacted our IT manager to start the process.
On Monday, October 14, 2019. Employees came to work like any other Monday. Only this time there was a sign posted on the door to the office. Since about half of our employees in the US work from home, calls and text messages were sent to them to also not use their computers. Once all employees were in the office, and sort of awake in other time zones, an all-company meeting/call-in was held. Granted, we still didn’t know too much, but what we knew, I shared with employees. Thomas was open and honest with me, and I continued that openness and honesty. That’s the way we roll.
As you can imagine, there was shock, there were questions, but mostly there was a ‘What can we do to help?’ attitude. While I wasn’t in Germany, based on what Thomas communicated to me, this was the same attitude there as well. Which, in any crisis situation, is exactly what you want from employees.
At this point, we were all in a holding pattern. Well, more like ‘frozen in time’ for some employees. We needed to utilize the employees with the ‘can do’ attitude and get them to get the ‘frozen’ employees ‘thawed out’ so to speak. Get everyone moving and continuing to be productive in some way – lead by the ‘can do-ers’ in the group. So that’s what we did in the first week, we went back to work! Our data may have been held hostage, but our employees weren’t going to be!
Interestingly enough, this attack was only on our IBM systems – our communication systems. In the US, we use iPhones and our sales group also use iPads in the field, which were safe. Had we used Android products, that wouldn’t necessarily be the case. I’m not advertising for Apple, or telling you that all cyberattacks work this way, I’m just saying that this fact was one ray of sunshine in our situation. We could continue to use our phones! In the office we could use desk phones, and in the field, we could use our cell phones.
-
Sales got back to work contacting their customers they had phone numbers for … stored in their phone – explaining our situation and working to get customers what they needed now
-
Safety Consultants continued completing safety services as best they could by taking pictures on their phones and hand writing notes
-
Training used clean pdf’s of training materials that had been saved on USBs – on their personal computers
-
Numerous others used their personal computers, set up gmail accounts and worked as best they could with the information they had or could recreate
-
For others, they pitched in to do those jobs that seem to never get done … we now had time for them!
Our supply room sparkles. I think our warehouse doubled in size – after we got rid of all the crap that was just taking up space. Desk files were cleaned out to the tune of a dumpster filled to the brim. Ironically, while we threw a lot of paper away, a lot of the information we needed to continue to do our jobs, was not printed – it was stored on servers. So while we were in no way a ‘paperless’ office, maybe we were?
We were getting on with business, however, and that’s what we needed to do – to make the situation feel somewhat ‘normal’ for our employees – until we found out more to tell our employees and customers, and to keep selling products and performing our safety services.
Pilz Germany was busy working on the major issues of keeping our systems safe from further attacks and getting us tools to help us communicate and work in the short-term. As you can imagine, it was a major challenge for the German team to regularly communicate progress to our employees worldwide. To help with global communication, Pilz Germany had us utilizing a secure communication service - Threema accounts – so we could safely share information, via texting, with other Pilz personnel and even customers to continue to work as best we could. Conference calls from Germany to subsidiaries and even video conference calls were periodically added to personally deliver positive messages to all subsidiaries – keeping the lines of communication open and sharing our successes quickly. All of these tools greatly helped in motivating our employees that, yes, we were on the right track.
Gaining Customer and Supplier Support.
But we, of course, couldn’t forget our customers, suppliers and others in our communication plans.
The Pilz website was taken off line and replaced with a basic black and white page giving a broad message about our cyberattack. New ‘local’ email addresses for orders and technical support were quickly added when they became available. Phone numbers remained the same, but were also listed on the website for clarity. There were also press releases and social media posts. Then we talked to customers personally. Whatever we knew, we shared with customers.
While Pilz Germany was playing to their strengths, we (Pilz USA) played to our strengths. In the first weeks of the attack:
-
IT set up a ‘war room’ where all company computers and used USBs were quarantined. Our TWO IT employees were in contact with IT from Germany, and really IT employees from around the Pilz world sharing information on what to do next.
-
Our sales employees worked from their iPhones and iPads. Their iPads included the Pilz Sales App which connected to our CRM – well, it did when our system was up and running – but it also houses all our product and services presentations, videos and literature. Granted sales couldn’t email these items at this time, but they could still use these with customers they were scheduled to see – a good short term solution.
-
Our employees asked to, and were allowed to, work on their own personal computers, so that they could continue to work. Let me repeat – OUR EMPLOYEES ASKED to use their personal computers! Prior to the attack this was not allowed, but our employees wanted to continue to work, and they knew they could get some things done using their personal computer. This allowed our safety consultants to continue to prepare the service reports for customers. Orders were still processed – offline. Customer quotes and proposals were sent out. Finance processed payments. I do not know for a fact, but it sure seemed like every employee that had a personal computer, used it in some why to help us get through this crisis.
-
Employees set up specific gmail accounts to communicate to customers, suppliers, and each other
-
For those employees with company iPhones, they could still ‘see’ their past emails, so they could get some contact information for their customers, suppliers and others so they could continue to get the job done.
-
Each employee took their talents, what they knew, what they could do and just did it – then shared what they did with others in an effort to help everyone.
-
Marketing worked with our email blast vendor that housed most of our customer contact information – so she used her personal computer, downloaded the information, cleaned it up and sent it out to the sales engineers via gmail, so sales now had some customer contact information to work with.
-
Operations set up their ‘war room’ using a total of 4 personal computers to post orders and shipments and send out invoices.
That was in the first weeks of the attack. But we didn’t stop there. We continued to get creative with ways to get things done. Our customers were instrumental in helping us as well. As we were open and honest with our customers, they were patient and honest with us.
For instance, with our systems down, we obviously had no access to prior quotes or proposals – and the pricing on each quote and proposal. So what do you do when a customer calls to place an order based on their quote? You tell them the truth about the situation, how we stand with regard to stock for that item (by checking with the warehouse), and then ask them what price we quoted. Looking back once we could check, our customers told us the truth about their pricing! We were open and honest to them – and had always been – and they were patient and honest with us. We had laid that groundwork of being honest and going the extra mile for our customers and our customers paid us back by being patient and understanding about our crisis.
One of our sales engineers relayed this story:
“Just before our cyberattack, one of my customers in the packaging industry had committed to using Pilz products for all their new production. I felt a personal approach with this customer was best, so I made three trips to the customer’s facility that was located 146 miles away. Since our systems were down, new quotes had to be created using previous paper copies as examples. I personally delivered the quote along with samples borrowed from our technical support team. The Pilz team in Canton, Michigan, worked their magic to ship the products and ensure this new data was kept secure.
The customer had previously gone through a cyberattack and as a result were empathetic to our situation. Given the challenges they had recovering from their attack, they were cautiously optimistic that we would deliver in time. They were ecstatic when we came through and delivered as we promised.”
Road to Recovery.
Now let’s go back to the progress Pilz made along the road to recovery. Actually, the results are quite impressive – when compared to the ‘averages’. Remember, the cyberattack became visible on Sunday, October 13th:
-
We were able to deliver again a week after the attack
-
A the end of October, our final assembly in Osfildern started back up again
-
The SMD line in Ostfildern and production at our French location in Betschdorf followed in early November
-
Production was resumed at our Swiss location a month after the attack
-
At the end of November our Chinese production facility also went back into operation
-
Everyday we celebrate small victories – new and better tools are now at our disposal, customers returning, progress continues to be made!
Today, I can honestly say that we have managed our way through this crisis, bested our attackers and are strengthened from this experience!
Lessons learned,
While technology may fail, the cohesion and coexistence of people and the will to solve challenges together have supported us.
Oure employees have demonstrated a particularly high degree of flexibility – globally! Many employees just naturally jumped in to support others where and when we needed it most - for example, IT and production.
We have used this experience as an opportunity to question our existing processes, procedures and systems to find better solutions in order to keep growing and bringing value to our customers.
While the cyberattack on Pilz was on our IT systems, our products and solutions, which we offer in the area of safety and security, were not affected. That is why we at Pilz, as a company in secure automation, will expand our existing know-how in the field of safety and security with the experience from our recent cyberattack.
We know that it will still be some time before all IT services are available again to the ‘normal’ level for all employees. We also know it will take us time to win back customers we may have lost in the wake of the attack. However, we have learned a lot since October 13th, we’ve come a long way, and we are very positive about our future. The internal changes that we have made – and continue to make because of the attack has made Pilz stronger than ever.
The content & opinions in this article are the author’s and do not necessarily represent the views of RoboticsTomorrow
Comments (0)
This post does not have any comments. Be the first to leave a comment below.
Featured Product
