Late on July 7 at his Noblesville home, J.J. Thompson lay awake in bed.
The CEO of Indianapolis-based Rook Security mulled over a tweet he received on his iPhone that day about a never-before-seen flaw in the Adobe Flash Player used by millions of websites, which made it easy for hackers to take control of a computer.
The secret was disclosed in documents stolen during a hack two days earlier of the Italian surveillance firm Hacking Team, which had been providing cyber-spying tools to a total of 97 intelligence and policing agencies in 35 countries, including the FBI.
The July 5 breach of Hacking Team, and the posting of 400 gigabytes of its files online, meant the kind of cyber weapons governments use to spy on one another were now out in the open, waiting for someone to grab and use them.
“Imagine a ticking clock and the whole world is chasing these cyber nukes,” Thompson said.
Over the next 10 days, that’s exactly what happened. A team of 10 security analysts and software developers at Rook worked with other IT security firms and a unit of the FBI’s Indianapolis office called Infragard to develop the first easy-to-use tool that can detect if any of the Hacking Team programs have infected a computer.
The tool, called Milano, was downloaded nearly 5,000 times in the first four days it was available.
The episode, discussed in more than a half-dozen interviews with those working behind the scenes to find a solution, gives a rare and dramatic glimpse into the growing threats posed by hackers and the work of the security professionals that try to stop them.
IT security has rocketed into the nation’s attention since Sony Pictures had four of its unreleased films stolen and an unending string of embarrassing internal emails posted online last year. And after Indianapolis-based Anthem Inc. had nearly 79 million customer records stolen by hackers in December and January. And after the U.S. Office of Personnel Management announced in June that files on 22 million federal employees—including detailed personal histories on top-ranking officials and spies—were stolen by hackers.
But experts say the Hacking Team breach takes everything to a new level. Its tools and methods were used to target specific people’s phones and computers. For example, some programs allowed hackers to record every sound a smartphone heard, look at every text message, record every key stroke, and take screen shots of whatever the user was viewing.
Before the breach, critics had accused Hacking Team’s tools and methods of being used by repressive regimes to target political dissidents and journalists. Of interest, several of Hacking team’s programs were disguised as news applications.
“This data dump is akin to the fall of the Soviet Union in a way. When the U.S.S.R. fell, global black markets were overflowing with Soviet weapons and, more importantly, knowledge of WMDs,” wrote Amit Serper and Alex Fraser, analysts at Cybereason, an IT security firm founded by ex-Israeli military officers, in a blog post. “The widespread availability of this data is going to empower hacking teams across the globe, providing them with much more sophisticated techniques to launch their own attacks.”
Decision to launch
The morning after his insomnia, Thompson gathered his top developers and analysts in Rook’s unmarked and dimly lit headquarters in a downtown Indianapolis office building.
There was a lot of weird stuff happening in the IT world that morning of July 8. The computer systems of the New York Stock Exchange, United Airlines and The Wall Street Journal all went down. And while those companies later said they were not hacked, the Rook team couldn’t be sure of that at the time.
“Is this really something to worry about?” asked Thompson, 35, sporting a shaved head, an unshaven face, jeans and cowboy boots. “Or are people just sensationalizing?”
Thompson told his team to find the Hacking Team files and figure it out for themselves.
This was not, at the time, a clear-cut decision. The giants of the IT security business—Symantec and McAfee—had done nothing at that point—and still have yet to release their own updates to protect against attacks using the Hacking Team files. Messages left for each company were not returned.
One reason Rook hesitated was because of the legal questions. Would Rook itself get in trouble if it downloaded all the stolen files from Hacking Team?
Thompson called an attorney, but was told there was no directly relevant case law for the international situation.
Thompson also called the Indianapolis office of the FBI Infragard, which coordinates a group of public agencies and private companies that works to address threats to critical infrastructure. It has cyber-security specialists on its staff and is forming an Indianapolis Cyber Security Task Force that includes the Indiana National Guard, the Indiana State Police, the Indiana Office of Technology, Purdue University, as well as such private security companies as Rook, RedLegg and Optiv.
But the FBI was in a sticky situation, because it was a client of Hacking Team. Documents from Hacking Team revealed at least two instances, in 2012 and 2014, in which FBI agents were using Hacking Team’s tools to break into and monitor the Web browsers of targets of their investigations, according to the online journalism site The Intercept.
At the same time, the FBI is set up to investigate hacks that have already occurred, not the threat of future hacks.
Chris Collins, the FBI coordinator of the Indiana chapter of Infragard, said the result was that the public and private groups focused on IT security thought a solution was coming, but none was.
“Everybody just assumed that everybody else was doing something," he said.
Meanwhile, more unknown vulnerabilities surfaced as IT researchers trolled through the Hacking Team files. iSight Partners, a Texas-based firm that tracks threats to IT security, said it noticed a flurry of activity aimed at exploiting newly revealed problems in such programs as Adobe Flash, Internet Explorer, Oracle Java and a back door into Facebook using Apple’s OS X operating system.
Those companies now have released patches to fix those problems.
Ultimately, Thompson decided the risk was too great to wait.
“I’m just sh--ting bricks, worried for our clients,” he recalled.
Testing the nukes
Tom Gorup, head of Rook’s security operations center, sought out the original 400-gigabyte file that had been posted on the dark Web—the part of the Internet not indexed by search engines. It took him six tries to find the actual file and then it took 24 hours to download all of it—which included not only Hacking Team’s tools, but also its client lists, internal emails and other documents.
In the meantime, one of Rook’s analysts found the source code of Hacking Team’s spying tools on GitHub, the primary service programmers use to host code online. It included 53 project files with malicious software and instruction manuals on how to use them.
Those instructions, along with the fact that Hacking Team was developing tools that could be easily deployed by government without the world’s best IT teams, make the Hacking Team tools especially dangerous.
“Hack-in-a-can type of thing,” Thompson quipped.
At 11:37 a.m. on July 8, Rook’s team set up a HipChat conversation and started a flurry of messages back and forth.
As Rook’s analysts worked through each of the 53 project files from Hacking Team, they sent updates to the FBI, the U.S. Secret Service, the Department of Homeland Security and other agencies on what they were finding.
By the morning of July 9, Gorup had downloaded all the Hacking Team files. He wrote a computer program that would search the files for keywords that could identify them as malicious.
This was no easy task. The really dangerous files are those that are executable—which mean they run a program. In computers running Windows, those files usually have the characters “.exe” tacked on to the end of the file name.
But the Hacking Team had hidden executable files inside files marked as simple pictures, which have the characters “.jpg” on the end.
Gorup kept running searches the entire day until 2 a.m. on July 10. He finally set up a program that would take three hours, and headed home.
“At that point, it was a lot of searching, reading, What does it do? Does it work? OK, nothing. Move on,” said Gorup, a bearded Army veteran who won the Purple Heart during his service in Iraq and Afghanistan.
The Rook team went through the same process for most of July 10. At the end of the day, one Rook analyst—who didn’t want to be named for fear of reprisals from hackers—tested one of Hacking Team’s malicious tools.
At 8:16 p.m., he messaged his peers: “Holy sh-- it worked.”
Working through the weekend and into the next week, Rook’s team finally churned though all the Hacking Team files that could harm computers running Windows. But that was only about 40 percent of the more than 2,000 project files they’re going through.
They plan to update their Milano tool soon so it can find files aimed at computers and phones running Linux, Apple’s OS and Android.
On July 14, the FBI and U.S. Secret Service sent representatives to Rook’s office to see what it had found.
Collins, the coordinator of the FBI Infragard team, said, “This was a collaborative effort where we relied upon private-sector partners.”
Shawn Axsom led four other developers in finishing the Milano tool. Even though he and his wife have a 1-month-old daughter, Axsom and his team stayed at Rook’s office past 1 a.m. every night and resumed every morning at 7 a.m.
“We knew the gravity of the situation,” he said.
Paul Mitchell, president of the Energy Systems Network, said his member companies were interested in how Rook, the FBI and others responded to the Hacking Team threat. That’s because those companies—including IPL, Duke Energy, Vectren, Cummins, Toyota, Allison Transmission and others—have a growing amount of their power-generation systems connected to the Internet and are potentially vulnerable to cyber attacks.
“It’s like, ‘OK, wow! This is one blip in a universe where there’s a whole bunch of these threats out there,’” Mitchell said. “This step taken by Rook is a perfect example of the kind of best practices that we need to be learning from and supporting.”•