A $10 billion-a-year effort to protect sensitive government data, from military secrets to Social Security numbers, is struggling to keep pace with an increasing number of cyberattacks and is unwittingly being undermined by federal employees and contractors.
Workers scattered across more than a dozen agencies, from the Defense and Education departments to the National Weather Service, are responsible for at least half of the federal cyberincidents reported each year since 2010, according to an Associated Press analysis of records.
They have clicked links in bogus phishing emails, opened malware-laden websites and been tricked by scammers into sharing information. One was redirected to a hostile site after connecting to a video of tennis star Serena Williams. A few act intentionally, most famously former National Security Agency contractor Edward Snowden, who downloaded and leaked documents revealing the government's collection of phone and email records.
Then there was the federal contractor who lost equipment containing the confidential information of millions of Americans, including Robert Curtis of Monument, Colorado.
"I was angry, because we as citizens trust the government to act on our behalf," he said. Curtis, according to court records, was besieged by identity thieves after someone stole data tapes that the contractor left in a car, exposing the health records of about 5 million current and former Pentagon employees and their families.
At a time when intelligence officials say cybersecurity now trumps terrorism as the No. 1 threat to the U.S. — and when breaches at businesses such as Home Depot and Target focus attention on data security — the federal government isn't required to publicize its own brushes with data loss.
On Monday, the U.S. Postal Service said it was the victim of a cyberattack and that information about its employees, including Social Security numbers, may have been compromised. And last month, a breach of unclassified White House computers by hackers thought to be working for Russia was reported not by officials but The Washington Post. Congressional Republicans complained even they weren't alerted to the hack.
"It would be unwise, I think for rather obvious reasons, for me to discuss from here what we have learned so far," White House press secretary Josh Earnest later said about the report.
To determine the extent of federal cyberincidents, which include probing into network weak spots, stealing data and defacing websites, the AP filed dozens of Freedom of Information Act requests, interviewed hackers, cybersecurity experts and government officials, and obtained documents describing digital cracks in the system.
That review shows that 40 years and more than $100 billion after the first federal data protection law was enacted, the government is struggling to close holes without the knowledge, staff or systems to outwit an ever-evolving foe.
"It's a much bigger challenge than anyone could have imagined 20 years ago," said Phyllis Schneck, deputy undersecretary for cybersecurity at the Department of Homeland Security, which runs a 24/7 incident-response center responding to threats.
Fears about breaches have been around since the late 1960s, when the federal government began shifting its operations onto computers. Officials responded with software designed to sniff out malicious programs and raise alarms about intruders.
And yet, attackers have always found a way in. Since 2006, there have been more than 87 million sensitive or private records exposed by breaches of federal networks, according to the nonprofit Privacy Rights Clearinghouse, which tracks cyberincidents at all levels of government through news, private sector and government reports.
By comparison, retail businesses lost 255 million records during that time, financial and insurance services lost 212 million and educational institutions lost 13 million. The federal records breached included employee usernames and passwords, veterans' medical records and a database detailing structural weaknesses in the nation's dams.
Marc Maiffret, a hacker turned cybersecurity expert, said "today's a little scarier" than when he was breaking into systems in the '90s. Malware and viruses can be purchased or rented, so advanced coding skills aren't required. And there's more mischief to be made, because the government depends on technology for everything from missile targeting to student loan processing.
"There's also a much bigger allure to use these skills to make money, in a criminal sense," said Maiffret, co-founder of the cybersecurity firm Beyond Trust, whose customers include the military.
From 2009, when the government began breaking out different types of incidents, to 2013, the number of reported breaches just on federal computer networks — the .gov and .mils — rose from 26,942 to 46,605, according to the U.S. Computer Emergency Readiness Team or US-CERT, which helps defend against cyberattacks.
Last year, US-CERT responded to a total of 228,700 cyberincidents involving federal agencies, companies that run critical infrastructure like nuclear power plants, dams and transit systems, and contract partners. That's more than double the incidents in 2009. And employees are to blame for at least half of the problems.
Last year, for example, about 21 percent of all federal breaches were traced to government workers who violated policies; 16 percent who lost devices or had them stolen; 12 percent who improperly handled sensitive information printed from computers; at least 8 percent who ran or installed malicious software; and 6 percent who were enticed to share private information, according to an annual White House review.
Internal documents released to the AP show how workers were lured in.
U.S. Department of Education employees — who had been warned repeatedly: "Think Before You Click!" — received an email a few weeks before Christmas 2011. "Your Amazon.com order of "Omron XEZ-740V Fat Loss" has shipped!" said the subject line, suggesting they click on a link.
"Unfortunately, several of your co-workers have fallen victim to this particular attack," said an urgent message from an incident response team. The department did not release information to the AP about any resulting damage. Meantime, reported cyberattacks at the agency have increased from 10 breaches with actual data loss in 2011 to 89 in 2013.
Reports from the Defense Department's Defense Security Service, tasked with protecting classified information and technologies in the hands of federal contractors, show how easy it is for hackers to get into DOD networks.
One security analyst, when notified that an account appeared to be infected with a virus, looked at the user's history and found he was redirected to a hostile site after trying to play a tennis video. Another military user sought help after receiving messages that his computer was infected when he visited a website on schools. Officials tracked the attacker to what appeared to be a Germany-based server.
"No matter what we do with the technology ... we'll always be vulnerable to the phishing attack and ... human-factor attacks unless we educate the overall workforce," said Eric Rosenbach, Assistant Secretary of Defense for Homeland Defense and Global Security.
In June, the General Accountability Office released a scathing review of smaller federal agencies' protections. One problem was not fully implementing security training programs for staff.
Email encryption, which protects the contents of messages, is one way the government is seeking to shore up cybersecurity. Fifty-one percent of all federal agencies in 2013 reported using a federally approved encryption service, up from 35 percent in 2012. But some departments, including State, reported zero percent compliance with any approved encryption provider. The State Department wouldn't comment, citing security concerns.
Federal systems grow more susceptible to attack as the government's online offerings expand to user-friendly websites and apps, experts said.
At a hacking convention in Las Vegas in August, Joe Abbey, the director of software for Arxan, an app security service, showed how easy it can be to break in.
Abbey demonstrated how someone could take an iPhone from a doctor who had downloaded a free .gov app to track Medicare and Medicaid payments, run several malicious files and return the device. From that point on, he said, the thief could track every payment and medical record entered.
"This now exposes everything that's inside this app," he said.
Another challenge is that cybersecurity experts are expensive and in short supply. A June study by RAND found the federal government, which doesn't pay as well as private firms, is particularly short-handed.
There are some 90,000 federal information technology security workers, one-third of them contractors. And while the government is projected to hire thousands more and spend $65 billion on cybersecurity contracts between 2015 and 2020, many experts believe the effort is not enough to catch up with a growing pool of hackers whose motives vary.
There are government-sponsored attacks: Cybersecurity firm Mandiant this year tracked Iranian-based hackers targeting several unidentified U.S. government agencies, while the Pentagon last year said Chinese government hackers stole plans for more than two dozen U.S. weapons systems, including an F-35 fighter.
A cyberattack similar to other hacker intrusions from China penetrated computer networks for months at USIS, the government's leading security clearance contractor, before the company noticed earlier this year, officials told the AP. The intrusion compromised the private records of at least 25,000 Homeland Security employees.
Thieves interested in selling valuable data also dig in. Others have less nefarious motives. In February 2013, someone broke into the Emergency Alert System, broadcasting warnings about a zombie attack in California, Michigan, Montana and New Mexico.
Jeremy Hammond, of Chicago, considers himself an activist who hacks to expose wrongs or promote social justice. He is serving 10 years in prison for breaking into the networks of security think tank Stratfor, whose clients include the departments of Homeland Security and Defense.
"I hacked a lot of stuff but the government, they're hacking all over the world," Hammond said in an interview from prison.
Only a small fraction of attackers are caught. Last year, the Justice Department filed 146 cases under the government's computer hacking statute. On Oct. 20, the FBI announced the arrest of a National Weather Service employee accused of illegally downloading sensitive files from the National Inventory of Dams in 2012.
For every thief or hostile state, there are tens of thousands of victims like Robert Curtis.
He declined to talk about specifics of his case. According to court records, a thief in September 2011 broke into a car parked in a San Antonio garage and stole unencrypted computer backup tapes containing Social Security numbers and medical information of Pentagon employees and troops. The car belonged to an employee of a federal contractor tasked with securing those records.
Ever since, criminals have tried to get cash, loans, credit, emergency funds — even establish businesses — in Curtis' name. He and his wife have frozen bank and credit accounts. His credit union once transferred $32,500 out of his account. That, he got back.
Curtis described the experience as "devastating." This summer a lawsuit brought by victims against the contractor, the Defense Department and a military health insurer was dismissed.
"It is very ironic," said Curtis, himself a cybersecurity expert who worked to provide secure networks at the Pentagon. "I was the person who had paper shredders in my house. I was a consummate data protection guy."
Follow Martha Mendoza on Twitter: https://twitter.com/mendozamartha