Machine learning can also aid the cyber enemy: NSA research head – ZDNet
Machine learning is one of the biggest buzzwords in cybersecurity in 2017. But a sufficiently smart adversary can exploit what the machine learning algorithm does, and reduce the quality of decision-making.
Cyberwar and the Future of Cybersecurity
Today's security threats have expanded in scope and seriousness. There can now be millions -- or even billions -- of dollars at risk when information security isn't handled properly.
"The concern about this is that one might find that an adversary is able to control, in a big-data environment, enough of that data that they can feed you in misdirection," said Dr Deborah Frincke, head of the Research Directorate (RD) of the US National Security Agency/Central Security Service (NSA/CSS).
Adversarial machine learning, as Frincke called it, is "a thing that we're starting to see emerge, a bit, in the wild". It's a path that we might reasonably believe will continue, she said.
As one example, an organisation may decide to use machine learning to develop a so-called "sense of self" of its own networks, and build a self-healing capability on top of that. But what if an attacker gets inside the network or perhaps was even inside the network before the machine learning process started?
"Their behaviour now becomes part of the norm. So in a sense, then, what I'm doing is that I'm protecting the insider. That's a problem," Frincke said.
"What's also interesting in the data science, is that if you are using a data-driven algorithm, [that algorithm] is what feeds the machine learning technique that you disseminate. Unless you keep that original data, you are not going to know what biases you built into your machine learning approach.
"You would have no way of that needle in the haystack, because you threw away the haystack, and all that's left are the weightings and the neural networks and so on."
Machine learning has other limitations too.
In 2016, for example, Monash University professor Tom Drummond pointed out that neural networks, one of the fundamental approaches to machine learning, can be led astray unless they're told why they're wrong.
The classic example of this problem dates back to the 1980s. Neil Fraser tells the story in his article Neural Network Follies from 1998.
The Pentagon was trying to teach a neural network to spot possible threats, such as an enemy tank hiding behind a tree. They trained the neural network with a set of photographs of tanks hiding behind trees, and another set of photographs of trees but no tanks.
But when asked to apply this knowledge, the system failed dismally.
"Eventually someone noticed that in the original set of 200 photos, all the images with tanks had been taken on a cloudy day, while all the images without tanks had been taken on a sunny day," Fraser wrote.
"The military was now the proud owner of a multi-million dollar mainframe computer that could tell you if it was sunny or not."
Frincke was speaking at the Australian Cyber Security Centre (ACSC) conference in Canberra on Wednesday. While she did point out the limits of machine learning, she also outlined some defensive strategies that the NSA has found to be effective.
Organisations can tip the cybersecurity balance of power more in their favour by learning to deceive or hide from the adversary, for example.
By its very nature, network defence is asymmetric. That imbalance is usually expressed as the defender having to close off every security vulnerability, while the attacker only has to be right once.
"On the face of it there should be something we should be able to do about that. You'd think there'd be some home-court advantage," Frincke said.
Traditionally, organisations have tried to make their data systems as efficient as possible. It makes the network more manageable. But from an attacker's point of view, it's easy to predict what's going on in any given system at any given time.
Taking a defensive deception approach, however, means building an excess capacity, and then finding ways to leverage that excess capacity to design in a deceptive or a changing approach. That way, an attacker can't really tell where the data is.
If you process data in the cloud, then one simple example might be to duplicate your data across many more nodes than you'd normally use, and switch between them.
"If you're trying to do an integrity attack, changing that data out from under me, you don't know which of, say, those hundred nodes I'm using. Or I might be looking at a subset of those nodes, say three, and you don't know which ones I'm using. So you could try to change them all at once [but] that's a lot harder," Frincke said.
The RD's research has shown that this approach increases the attacker's cognitive load and plays on their cognitive biases.
"We can try to lead them into wrong conclusions. In other words, we're frustrating them. We're trying to make them work too hard, to gain ground that they don't need. And that will make it easier for us to find them," Frincke said.
"It's a little bit like the old honeypot [or] honeynet writ large, but designed into the system as an integral part of the way that it works, and not an add-on."
The downside to defensive deception is that it's harder to manage.
"Now I have to do more work as a system manager, and as a designer, to be sure I know which one of those three of the hundred I should use, otherwise I could end up shooting myself in the foot, especially if I've [been] deploying some kind of misleading changes for the adversary," Frincke said.
More here:
Machine learning can also aid the cyber enemy: NSA research head - ZDNet
- Builder tied to house collapse that killed 3 slapped with NSA - Times of India - July 4th, 2025 [July 4th, 2025]
- We are working to retrieve all documents on abandoned facilities NSA Boss - Citi Sports Online - July 4th, 2025 [July 4th, 2025]
- NSA and CISA urge shift to languages improving memory safety - Developer Tech News - July 2nd, 2025 [July 2nd, 2025]
- Credit Rating For The Unrated REITs (Part 5): National Storage Affiliates Trust (NYSE:NSA) - Seeking Alpha - July 2nd, 2025 [July 2nd, 2025]
- NSA, CISA Release CSI Urging Adoption of Memory Safe Languages for Enhanced Software Security - ExecutiveGov - June 28th, 2025 [June 28th, 2025]
- Brandonville native named Sailor of the Year at NSA Mechanicsburg - The Shenandoah Sentinel - June 28th, 2025 [June 28th, 2025]
- NSA and CISA Release CSI Highlighting Importance of Memory Safe Languages in Software Security - National Security Agency (NSA) (.gov) - June 28th, 2025 [June 28th, 2025]
- NSA Doval Emphasizes Anti-Terror Cooperation During High-Level Beijing Talks With Chinese Foreign Minister - The Hans India - June 24th, 2025 [June 24th, 2025]
- NSA Doval and Chinese Foreign Minister discuss future meet on boundary issue - Tribune India - June 24th, 2025 [June 24th, 2025]
- NSA Ajit Doval to deliver strong message on terrorism on his upcoming China visit - Moneycontrol - June 22nd, 2025 [June 22nd, 2025]
- Bangladesh NSA In Washington, Talking To Trump Officials. More Regional Shifts? - IndiaWest - June 22nd, 2025 [June 22nd, 2025]
- Naval Academy, NSA Annapolis closed Monday for mysterious world events. Both reopened Tuesday. - Baltimore Sun - June 22nd, 2025 [June 22nd, 2025]
- Pakistan is useful to the world: Former NSA Shivshankar Menon explains why countries still support Islam - The Economic Times - June 22nd, 2025 [June 22nd, 2025]
- Midland University Receives Grant from NSA - Midland University - June 20th, 2025 [June 20th, 2025]
- NSA Approves Wave Relay Devices for Securing Classified Information - AFCEA International - June 7th, 2025 [June 7th, 2025]
- NSA Validates Wave Relay devices to Protect Classified Information - PR Newswire - June 5th, 2025 [June 5th, 2025]
- Cyberattacks Surge in 2025: Data Analysts Urged to Bolster Privacy with PETs and NSA-CISA AI Security Guidelines - WebProNews - June 1st, 2025 [June 1st, 2025]
- India is ready and has capability to fight terrorism on its own: Former Dy NSA Pankaj Saran in London - The Economic Times - June 1st, 2025 [June 1st, 2025]
- NSA Teams With Int'l Cyber Agencies to Craft Guidance for Implementing SIEM, SOAR Platforms - ExecutiveGov - May 28th, 2025 [May 28th, 2025]
- NSA, ASDs ACSC, and other agencies publish three Cybersecurity Information Sheets with gu - National Security Agency (.gov) - May 28th, 2025 [May 28th, 2025]
- Punjab MP and NSA detainee Amritpal Singhs jailed aides look to speed up trials in other FIRs, file plea - Times of India - May 28th, 2025 [May 28th, 2025]
- NSA Ajit Doval down with flu, calls off visit to Russia - Hindustan Times - May 28th, 2025 [May 28th, 2025]
- Former NSA Director and SandboxAQ CEO on Quantitative AI and its inevitable integration - MSN - May 28th, 2025 [May 28th, 2025]
- NSA Ajit Doval speaks with Chinese FM Wang Yi amid rising India-Pak tension 'War not India's choice' - The Economic Times - May 11th, 2025 [May 11th, 2025]
- 'War was not India's choice and was not in the interests of any party': NSA Ajit Doval speaks to China's - Times of India - May 11th, 2025 [May 11th, 2025]
- NSA to cut up to 2,000 civilian roles - The Hill - May 10th, 2025 [May 10th, 2025]
- NSA Ajit Doval speaks with US Secretary of State 'shortly after' Indian strikes on Pak - Deccan Herald - May 10th, 2025 [May 10th, 2025]
- NSA to cut up to 2,000 civilian roles as part of intel community downsizing - The Record from Recorded Future News - May 10th, 2025 [May 10th, 2025]
- Operation Sindoor: NSA Doval engages with counterparts from US, UK, China, and Russia - Social News XYZ - May 10th, 2025 [May 10th, 2025]
- CIA, NSA to face major layoffs as Trump pushes intelligence reform - Times of India - May 5th, 2025 [May 5th, 2025]
- Dont see a major war with India, but have to be ready: Pakistan ex-NSA - Al Jazeera - May 5th, 2025 [May 5th, 2025]
- Donald Trump set to axe thousands of jobs at CIA, NSA and other agencies - Daily Mail - May 5th, 2025 [May 5th, 2025]
- 757Teamz softball Top 15: NSA moves up as Hickory perseveres to remain No. 1 - The Virginian-Pilot - May 5th, 2025 [May 5th, 2025]
- NSA head Mike Waltz and his deputy Alex Wong to exit Trump admin amid Signal chat fiasco - The Economic Times - May 5th, 2025 [May 5th, 2025]
- Trump speaks out on NSA shakeup, addresses third term talk - Fox News - May 5th, 2025 [May 5th, 2025]
- Mike Waltz, Alex Wong to resign: Here's who may replace NSA head and deputy - Hindustan Times - May 5th, 2025 [May 5th, 2025]
- A Lot of People Want the Job: Trump Says Hell Choose Waltzs NSA Replacement in Next 6 Months - The Daily Signal - May 5th, 2025 [May 5th, 2025]
- Will Steve Witkoff replace Mike Waltz as Donald Trump's new NSA? - Times of India - May 5th, 2025 [May 5th, 2025]
- Beavercreek native recognized for NSA Codebreaker achievement - Fairborn Daily Herald - May 5th, 2025 [May 5th, 2025]
- Marco Rubio to serve as acting NSA; Mike Waltz removed by President Trump - FOX 35 Orlando - May 5th, 2025 [May 5th, 2025]
- Trump says he will name new NSA within 6 months - LiveNOW from FOX - May 5th, 2025 [May 5th, 2025]
- Mike Waltz out as NSA, Rubio to serve in the interim - LiveNOW from FOX - May 5th, 2025 [May 5th, 2025]
- Mike Waltz Leaves White House for UN Witkoff Tipped as Trumps Next NSA - Hungarian Conservative - May 5th, 2025 [May 5th, 2025]
- McConnell calls out Trump for hiring amateur isolationists at Pentagon, firing NSA director - The Hill - April 8th, 2025 [April 8th, 2025]
- Trumps firing of NSA chief is rolling out the red carpet for cyber attacks - Politico - April 8th, 2025 [April 8th, 2025]
- A conspiracy theorist convinced Trump to fire the NSA director - Vox - April 8th, 2025 [April 8th, 2025]
- William Hartman Named Acting NSA Director Following Dismissal of Top Officials - ExecutiveGov - April 8th, 2025 [April 8th, 2025]
- NSA and partners Issue Guidance on Fast Flux as a National Security Threat - National Security Agency (NSA) (.gov) - April 8th, 2025 [April 8th, 2025]
- Security News This Week: NSA Chief Ousted Amid Trump Loyalty Firing Spree - WIRED - April 8th, 2025 [April 8th, 2025]
- Head of NSA and US Cyber Command reportedly fired - Cybersecurity Dive - April 8th, 2025 [April 8th, 2025]
- Trump fires Gen. Timothy Haugh from leadership of Cyber Command and NSA - DefenseScoop - April 8th, 2025 [April 8th, 2025]
- Gen. Timothy Haugh, head of NSA and Cyber Command, is fired - CBS News - April 8th, 2025 [April 8th, 2025]
- Trump's mixed tariff messaging and NSA director and deputy fired: Morning Rundown - NBC News - April 8th, 2025 [April 8th, 2025]
- NSA Director and Deputy Reportedly Dismissed: What We Know - Newsweek - April 8th, 2025 [April 8th, 2025]
- Haugh fired from leadership of NSA, Cyber Command - The Record from Recorded Future News - April 8th, 2025 [April 8th, 2025]
- Trump administration fires head of NSA and U.S. Cyber Command, along with other top officials - CBS News - April 8th, 2025 [April 8th, 2025]
- US Cyber Command, NSA Chief Gen. Timothy Haugh ousted by Trump admin - Breaking Defense - April 8th, 2025 [April 8th, 2025]
- Face the Facts: Rep. Himes talks about firing of two top NSA officials - NBC Connecticut - April 8th, 2025 [April 8th, 2025]
- NSA Issues Advisory on Fast Flux Cyberthreat - ExecutiveGov - April 8th, 2025 [April 8th, 2025]
- Loomer, far-right activist, urged Trump to remove NSA director and others: Sources - ABC News - April 8th, 2025 [April 8th, 2025]
- The NSA Sounds Security Alarm For Billions Of iPhone And Android Phones - HotHardware - April 8th, 2025 [April 8th, 2025]
- NSA director fired after Trumps meeting with right-wing influencer Laura Loomer - The Verge - April 8th, 2025 [April 8th, 2025]
- Trump fires head of NSA and Cyber Command - Nextgov - April 8th, 2025 [April 8th, 2025]
- What are the national security concerns of Trump firing the NSA, Cyber Command head? - CBS News - April 8th, 2025 [April 8th, 2025]
- Who is Timothy Haugh? The NSA chief fired amid cyber security concerns - Times of India - April 8th, 2025 [April 8th, 2025]
- NSA, CISA, FBI, and International Partners Release Cybersecurity Advisory on Fast Flux, a National Security Threat - Hstoday - April 8th, 2025 [April 8th, 2025]
- Senator King Responds to Reported Firing of NSA Director General Timothy Haugh - WAGM - April 8th, 2025 [April 8th, 2025]
- NSA warned of vulnerabilities in Signal app a month before Houthi strike chat - CBS News - March 26th, 2025 [March 26th, 2025]
- Trump said poised to fire NSA Mike Waltz for including journalist in top secret war chat - The Times of Israel - March 26th, 2025 [March 26th, 2025]
- Not the last Waltz: Trump defends NSA after security breach - The Times of India - March 26th, 2025 [March 26th, 2025]
- NSA warned about vulnerabilities in Signal prior to White House group chat fiasco - SiliconANGLE News - March 26th, 2025 [March 26th, 2025]
- NSA warned the Signal app was vulnerable last month - WTIC - March 26th, 2025 [March 26th, 2025]
- Codebreakers and Covert Agents: The Women Behind the NSA and CIA heads to Illinois State Museum - WAND - March 26th, 2025 [March 26th, 2025]
- NSA warned about using Signal a month before leak of Houthi strike chat - CBS News - March 26th, 2025 [March 26th, 2025]
- 'Putin is giddy': NSA knew Signal was vulnerable to Russian hackers before security breach - AlterNet - March 26th, 2025 [March 26th, 2025]
- RAW: NSA MIKE WALTZ EXPECTED TO VISIT GREENLAND - Local 3 News - March 26th, 2025 [March 26th, 2025]
- US NSA likely to visit India in third week of April - Hindustan Times - March 26th, 2025 [March 26th, 2025]
- Statement from Secretary Rubio and NSA Waltz on Call with Zelenskyy - Department of State - March 22nd, 2025 [March 22nd, 2025]
- Europe must invest more in defence amid global shifts: Greeces NSA Ntokos - Firstpost - March 22nd, 2025 [March 22nd, 2025]
- NSA Bahrain, NAVCENT Hold First-of-its-Kind Exercise Vigilant Resolve - navy.mil - March 22nd, 2025 [March 22nd, 2025]