Machine learning security needs new perspectives and incentives – TechTalks
At this years International Conference on Learning Representations (ICLR), a team of researchers from the University of Maryland presented an attack technique meant to slow down deep learning models that have been optimized for fast and sensitive operations. The attack, aptly named DeepSloth, targets adaptive deep neural networks, a range of deep learning architectures that cut down computations to speed up processing.
Recent years have seen growing interest in the security of machine learning and deep learning, and there are numerous papers and techniques on hacking and defending neural networks. But one thing made DeepSloth particularly interesting: The researchers at the University of Maryland were presenting a vulnerability in a technique they themselves had developed two years earlier.
In some ways, the story of DeepSloth illustrates the challenges that the machine learning community faces. On the one hand, many researchers and developers are racing to make deep learning available to different applications. On the other hand, their innovations cause new challenges of their own. And they need to actively seek out and address those challenges before they cause irreparable damage.
One of the biggest hurdles of deep learning the computational costs of training and running deep neural networks. Many deep learning models require huge amounts of memory and processing power, and therefore they can only run on servers that have abundant resources. This makes them unusable for applications that require all computations and data to remain on edge devices or need real-time inference and cant afford the delay caused by sending their data to a cloud server.
In the past few years, machine learning researchers have developed several techniques to make neural networks less costly. One range of optimization techniques called multi-exit architecture stops computations when a neural network reaches acceptable accuracy. Experiments show that for many inputs, you dont need to go through every layer of the neural network to reach a conclusive decision. Multi-exit neural networks save computation resources and bypass the calculations of the remaining layers when they become confident about their results.
In 2019, Yigitan Kaya, a Ph.D. student in Computer Science at the University of Maryland, developed a multi-exit technique called shallow-deep network, which could reduce the average inference cost of deep neural networks by up to 50 percent. Shallow-deep networks address the problem of overthinking, where deep neural networks start to perform unneeded computations that result in wasteful energy consumption and degrade the models performance. The shallow-deep network was accepted at the 2019 International Conference on Machine Learning (ICML).
Early-exit models are a relatively new concept, but there is a growing interest, Tudor Dumitras, Kayas research advisor and associate professor at the University of Maryland, told TechTalks. This is because deep learning models are getting more and more expensive computationally, and researchers look for ways to make them more efficient.
Dumitras has a background in cybersecurity and is also a member of the Maryland Cybersecurity Center. In the past few years, he has been engaged in research on security threats to machine learning systems. But while a lot of the work in the field focuses on adversarial attacks, Dumitras and his colleagues were interested in finding all possible attack vectors that an adversary might use against machine learning systems. Their work has spanned various fields including hardware faults, cache side-channel attacks, software bugs, and other types of attacks on neural networks.
While working on the deep-shallow network with Kaya, Dumitras and his colleagues started thinking about the harmful ways the technique might be exploited.
We then wondered if an adversary could force the system to overthink; in other words, we wanted to see if the latency and energy savings provided by early exit models like SDN are robust against attacks, he said.
Dumitras started exploring slowdown attacks on shallow-deep networks with Ionut Modoranu, then a cybersecurity research intern at the University of Maryland. When the initial work showed promising results, Kaya and Sanghyun Hong, another Ph.D. student at the University of Maryland, joined the effort. Their research eventually culminated into the DeepSloth attack.
Like adversarial attacks, DeepSloth relies on carefully crafted input that manipulates the behavior of machine learning systems. However, while classic adversarial examples force the target model to make wrong predictions, DeepSloth disrupts computations. The DeepSloth attack slows down shallow-deep networks by preventing them from making early exits and forcing them to carry out the full computations of all layers.
Slowdown attacks have the potential ofnegating the benefits ofmulti-exit architectures, Dumitras said.These architectures can halve the energy consumption of a deep neural network model at inference time, and we showed that for any input we can craft a perturbation that wipes out those savings completely.
The researchers findings show that the DeepSloth attack can reduce the efficacy of the multi-exit neural networks by 90-100 percent. In the simplest scenario, this can cause a deep learning system to bleed memory and compute resources and become inefficient at serving users.
But in some cases, it can cause more serious harm. For example, one use of multi-exit architectures involves splitting a deep learning model between two endpoints. The first few layers of the neural network can be installed on an edge location, such as a wearable or IoT device. The deeper layers of the network are deployed on a cloud server. The edge side of the deep learning model takes care of the simple inputs that can be confidently computed in the first few layers. In cases where the edge side of the model does not reach a conclusive result, it defers further computations to the cloud.
In such a setting, the DeepSloth attack would force the deep learning model to send all inferences to the cloud. Aside from the extra energy and server resources wasted, the attack could have much more destructive impact.
In a scenario typical for IoT deployments, where the model is partitioned between edge devices and the cloud, DeepSloth amplifies the latency by 1.55X, negating the benefits of model partitioning, Dumitras said. This could cause the edge device to miss critical deadlines, for instance in an elderly monitoring program that uses AI to quickly detect accidents and call for help if necessary.
While the researchers made most of their tests on deep-shallow networks, they later found that the same technique would be effective on other types of early-exit models.
As with most works on machine learning security, the researchers first assumed that an attacker has full knowledge of the target model and has unlimited computing resources to craft DeepSloth attacks. But the criticality of an attack also depends on whether it can be staged in practical settings, where the adversary has partial knowledge of the target and limited resources.
In most adversarial attacks, the attacker needs to have full access to the model itself, basically, they have an exact copy of the victim model, Kaya told TechTalks. This, of course, is not practical in many settings where the victim model is protected from outside, for example with an API like Google Vision AI.
To develop a realistic evaluation of the attacker, the researchers simulated an adversary who doesnt have full knowledge of the target deep learning model. Instead, the attacker has asurrogatemodel on which he tests and tunes the attack. The attacker thentransfers the attack to the actual target. The researchers trained surrogate models that have different neural network architectures, different training sets, and even different early-exit mechanisms.
We find that the attacker that uses a surrogate can still cause slowdowns (between 20-50%) in the victim model, Kaya said.
Such transfer attacks are much more realistic than full-knowledge attacks, Kaya said. And as long as the adversary has a reasonable surrogate model, he will be able to attack a black-box model, such as a machine learning system served through a web API.
Attacking a surrogate is effective because neural networks that perform similar tasks (e.g., object classification) tend to learn similar features (e.g., shapes, edges, colors), Kaya said.
Dumitras says DeepSloth is just the first attack that works in this threat model, and he believes more devastating slowdown attacks will be discovered. He also pointed out that, aside from multi-exit architectures, other speed optimization mechanisms are vulnerable to slowdown attacks. His research team tested DeepSloth on SkipNet, a special optimization technique for convolutional neural networks (CNN). Their findings showed that DeepSloth examples crafted for multi-exit architecture also caused slowdowns in SkipNet models.
This suggests thatthe two different mechanisms might share a deeper vulnerability, yet to be characterized rigorously, Dumitras said. I believe that slowdown attacks may become an important threat in the future.
The researchers also believe that security must be baked into the machine learning research process.
I dont think any researcher today who is doing work on machine learning is ignorant of the basic security problems. Nowadays even introductory deep learning courses include recent threat models like adversarial examples, Kaya said.
The problem, Kaya believes, has to do with adjusting incentives. Progress is measured on standardized benchmarks and whoever develops a new technique uses these benchmarks and standard metrics to evaluate their method, he said, adding that reviewers who decide on the fate of a paper also look at whether the method is evaluated according to their claims on suitable benchmarks.
Of course, when a measure becomes a target, it ceases to be a good measure, he said.
Kaya believes there should be a shift in the incentives of publications and academia. Right now, academics have a luxury or burden to make perhaps unrealistic claims about the nature of their work, he says. If machine learning researchers acknowledge that their solution will never see the light of day, their paper might be rejected. But their research might serve other purposes.
For example, adversarial training causes large utility drops, has poor scalability, and is difficult to get right, limitations that are unacceptable for many machine learning applications. But Kaya points out that adversarial training can have benefits that have been overlooked, such as steering models toward becoming more interpretable.
One of the implications of too much focus on benchmarks is that most machine learning researchers dont examine the implications of their work when applied to real-world settings and realistic settings.
Our biggest problem is that we treat machine learning security as an academic problem right now. So the problems we study and the solutions we design are also academic, Kaya says. We dont know if any real-world attacker is interested in using adversarial examples or any real-world practitioner in defending against them.
Kaya believes the machine learning community should promote and encourage research in understanding the actual adversaries of machine learning systems rather than dreaming up our own adversaries.
And finally, he says that authors of machine learning papers should be encouraged to do their homework and find ways to break their own solutions, as he and his colleagues did with the shallow-deep networks. And researchers should be explicit and clear about the limits and potential threats of their machine learning models and techniques.
If we look at the papers proposing early-exit architectures, we see theres no effort to understand security risks although they claim that these solutions are of practical value, he says. If an industry practitioner finds these papers and implements these solutions, they are not warned about what can go wrong. Although groups like ours try to expose potential problems, we are less visible to a practitioner who wants to use an early-exit model. Even including a paragraph about the potential risks involved in a solution goes a long way.
More:
Machine learning security needs new perspectives and incentives - TechTalks
- Open source machine learning systems are highly vulnerable to security threats - TechRadar - December 22nd, 2024 [December 22nd, 2024]
- After the PS5 Pro's less dramatic changes, PlayStation architect Mark Cerny says the next-gen will focus more on CPUs, memory, and machine-learning -... - December 22nd, 2024 [December 22nd, 2024]
- Accelerating LLM Inference on NVIDIA GPUs with ReDrafter - Apple Machine Learning Research - December 22nd, 2024 [December 22nd, 2024]
- Machine learning for the prediction of mortality in patients with sepsis-associated acute kidney injury: a systematic review and meta-analysis - BMC... - December 22nd, 2024 [December 22nd, 2024]
- Machine learning uncovers three osteosarcoma subtypes for targeted treatment - Medical Xpress - December 22nd, 2024 [December 22nd, 2024]
- From Miniatures to Machine Learning: Crafting the VFX of Alien: Romulus - Animation World Network - December 22nd, 2024 [December 22nd, 2024]
- Identification of hub genes, diagnostic model, and immune infiltration in preeclampsia by integrated bioinformatics analysis and machine learning -... - December 22nd, 2024 [December 22nd, 2024]
- This AI Paper from Microsoft and Novartis Introduces Chimera: A Machine Learning Framework for Accurate and Scalable Retrosynthesis Prediction -... - December 18th, 2024 [December 18th, 2024]
- Benefits and Challenges of Integrating AI and Machine Learning into EHR Systems - Healthcare IT Today - December 18th, 2024 [December 18th, 2024]
- The History Of AI: How Machine Learning's Evolution Is Reshaping Everything Around Us - SlashGear - December 18th, 2024 [December 18th, 2024]
- AI and Machine Learning to Enhance Pension Plan Governance and the Investor Experience: New CFA Institute Research - Fintech Finance - December 18th, 2024 [December 18th, 2024]
- Address Common Machine Learning Challenges With Managed MLflow - The New Stack - December 18th, 2024 [December 18th, 2024]
- Machine Learning Used To Classify Fossils Of Extinct Pollen - Offworld Astrobiology Applications? - Astrobiology News - December 18th, 2024 [December 18th, 2024]
- Machine learning model predicts CDK4/6 inhibitor effectiveness in metastatic breast cancer - News-Medical.Net - December 18th, 2024 [December 18th, 2024]
- New Lockheed Martin Subsidiary to Offer Machine Learning Tools to Defense Customers - ExecutiveBiz - December 18th, 2024 [December 18th, 2024]
- How Powerful Will AI and Machine Learning Become? - International Policy Digest - December 18th, 2024 [December 18th, 2024]
- ChatGPT-Assisted Machine Learning for Chronic Disease Classification and Prediction: A Developmental and Validation Study - Cureus - December 18th, 2024 [December 18th, 2024]
- Blood Tests Are Far From Perfect But Machine Learning Could Change That - Inverse - December 18th, 2024 [December 18th, 2024]
- Amazons AGI boss: You dont need a PhD in machine learning to build with AI anymore - Fortune - December 18th, 2024 [December 18th, 2024]
- From Novice to Pro: A Roadmap for Your Machine Learning Career - KDnuggets - December 10th, 2024 [December 10th, 2024]
- Dimension nabs $500M second fund for 'still contrary' intersection of bio and machine learning - Endpoints News - December 10th, 2024 [December 10th, 2024]
- Using Machine Learning to Make A Really Big Detailed Simulation - Astrobites - December 10th, 2024 [December 10th, 2024]
- Driving Business Growth with GreenTomatos Data and Machine Learning Strategy on Generative AI - AWS Blog - December 10th, 2024 [December 10th, 2024]
- Unlocking the power of data analytics and machine learning to drive business performance - WTW - December 10th, 2024 [December 10th, 2024]
- AI and the Ethics of Machine Learning | by Abwahabanjum | Dec, 2024 - Medium - December 10th, 2024 [December 10th, 2024]
- Differentiating Cystic Lesions in the Sellar Region of the Brain Using Artificial Intelligence and Machine Learning for Early Diagnosis: A Prospective... - December 10th, 2024 [December 10th, 2024]
- New Amazon SageMaker AI Innovations Reimagine How Customers Build and Scale Generative AI and Machine Learning Models - Amazon Press Release - December 10th, 2024 [December 10th, 2024]
- What is Machine Learning? 18 Crucial Concepts in AI, ML, and LLMs - Netguru - December 5th, 2024 [December 5th, 2024]
- Machine learning-based prediction of antibiotic resistance in Mycobacterium tuberculosis clinical isolates from Uganda - BMC Infectious Diseases - December 5th, 2024 [December 5th, 2024]
- Interdisciplinary Team Needed to Apply Machine Learning in Epilepsy Surgery: Lara Jehi, MD, MHCDS - Neurology Live - December 5th, 2024 [December 5th, 2024]
- A multimodal machine learning model for the stratification of breast cancer risk - Nature.com - December 5th, 2024 [December 5th, 2024]
- Machine learning based intrusion detection framework for detecting security attacks in internet of things - Nature.com - December 5th, 2024 [December 5th, 2024]
- Machine learning evaluation of a hypertension screening program in a university workforce over five years - Nature.com - December 5th, 2024 [December 5th, 2024]
- Vaultree Introduces VENum Stack: Combining the Power of Machine Learning and Encrypted Data Processing for Secure Innovation - PR Newswire - December 5th, 2024 [December 5th, 2024]
- Direct simulation and machine learning structure identification unravel soft martensitic transformation and twinning dynamics - pnas.org - December 5th, 2024 [December 5th, 2024]
- AI and Machine Learning - Maryland to use AI technology to manage traffic flow - SmartCitiesWorld - December 5th, 2024 [December 5th, 2024]
- Researchers make machine learning breakthrough in lithium-ion tech here's how it could make aging batteries safer - Yahoo! Voices - December 5th, 2024 [December 5th, 2024]
- Integrating IoT and machine learning: Benefits and use cases - TechTarget - December 5th, 2024 [December 5th, 2024]
- Landsat asks industry for artificial intelligence (AI) and machine learning for satellite operations - Military & Aerospace Electronics - December 5th, 2024 [December 5th, 2024]
- Machine learning optimized efficient graphene-based ultra-broadband solar absorber for solar thermal applications - Nature.com - December 5th, 2024 [December 5th, 2024]
- Polymathic AI Releases The Well: 15TB of Machine Learning Datasets Containing Numerical Simulations of a Wide Variety of Spatiotemporal Physical... - December 5th, 2024 [December 5th, 2024]
- Prediction of preterm birth using machine learning: a comprehensive analysis based on large-scale preschool children survey data in Shenzhen of China... - December 5th, 2024 [December 5th, 2024]
- Application of machine learning algorithms to identify serological predictors of COVID-19 severity and outcomes - Nature.com - November 30th, 2024 [November 30th, 2024]
- Predicting the time to get back to work using statistical models and machine learning approaches - BMC Medical Research Methodology - November 30th, 2024 [November 30th, 2024]
- AI and Machine Learning - US releases recommendations for use of AI in critical infrastructure - SmartCitiesWorld - November 30th, 2024 [November 30th, 2024]
- Machine learning-based diagnostic model for stroke in non-neurological intensive care unit patients with acute neurological manifestations -... - November 28th, 2024 [November 28th, 2024]
- Analysis of four long non-coding RNAs for hepatocellular carcinoma screening and prognosis by the aid of machine learning techniques - Nature.com - November 28th, 2024 [November 28th, 2024]
- Evaluation and prediction of the physical properties and quality of Jatob-do-Cerrado seeds processed and stored in different conditions using machine... - November 28th, 2024 [November 28th, 2024]
- Researchers use fitness tracker data and machine learning to detect bipolar disorder mood swings - Medical Xpress - November 28th, 2024 [November 28th, 2024]
- Advances in AI and Machine Learning for Nuclear Applications - Frontiers - November 28th, 2024 [November 28th, 2024]
- Researchers make machine learning breakthrough in lithium-ion tech here's how it could make aging batteries safer - The Cool Down - November 28th, 2024 [November 28th, 2024]
- Svitla Systems Publishes Results of the Study on Machine Learning's Role in Credit Scoring - Newsfile - November 28th, 2024 [November 28th, 2024]
- Predicting poor performance on cognitive tests among older adults using wearable device data and machine learning: a feasibility study - Nature.com - November 28th, 2024 [November 28th, 2024]
- Quantum Machine Learning: Bridging the Future of AI and Quantum Computing - TechBullion - November 28th, 2024 [November 28th, 2024]
- AI and machine learning trends in healthcare - Healthcare Leader - November 28th, 2024 [November 28th, 2024]
- Identification of biomarkers for the diagnosis in colorectal polyps and metabolic dysfunction-associated steatohepatitis (MASH) by bioinformatics... - November 28th, 2024 [November 28th, 2024]
- Revolutionizing Business Systems with Machine Learning: Practical Innovations for the Modern Era - TechBullion - November 28th, 2024 [November 28th, 2024]
- Can AI improve plant-based meats? Using mechanical testing and machine learning to mimic the sensory experience - Phys.org - November 16th, 2024 [November 16th, 2024]
- Machine Learning Reveals Impact of Microbial Load on Gut Health and Disease - Genetic Engineering & Biotechnology News - November 16th, 2024 [November 16th, 2024]
- Machine learning for predicting in-hospital mortality in elderly patients with heart failure combined with hypertension: a multicenter retrospective... - November 16th, 2024 [November 16th, 2024]
- Apple Researchers Propose Cut Cross-Entropy (CCE): A Machine Learning Method that Computes the Cross-Entropy Loss without Materializing the Logits for... - November 16th, 2024 [November 16th, 2024]
- Exploring electron-beam induced modifications of materials with machine-learning assisted high temporal resolution electron microscopy - Nature.com - November 16th, 2024 [November 16th, 2024]
- Facilitated the discovery of new / Co-based superalloys by combining first-principles and machine learning - Nature.com - November 16th, 2024 [November 16th, 2024]
- Thwarting Phishing Attacks with Predictive Analytics and Machine Learning in 2024 - Petri.com - November 16th, 2024 [November 16th, 2024]
- Optoelectronic performance prediction of HgCdTe homojunction photodetector in long wave infrared spectral region using traditional simulations and... - November 16th, 2024 [November 16th, 2024]
- A new approach for sex prediction by evaluating mandibular arch and canine dimensions with machine-learning classifiers and intraoral scanners (a... - November 16th, 2024 [November 16th, 2024]
- AI and Machine Learning - Google and National League of Cities develop AI toolkit - SmartCitiesWorld - November 16th, 2024 [November 16th, 2024]
- Machine learning for the physics of climate - Nature.com - November 14th, 2024 [November 14th, 2024]
- Red Hat acquires tech to lower the cost of machine learning - ComputerWeekly.com - November 14th, 2024 [November 14th, 2024]
- SUU Professor Receives Grant to Develop Machine Learning Certificate - Southern Utah University - November 14th, 2024 [November 14th, 2024]
- Research on the timing for subsequent water flooding in Alkali-Surfactant-Polymer flooding in Daqing Oilfield based on automated machine learning -... - November 14th, 2024 [November 14th, 2024]
- SNPs and blood inflammatory marker featured machine learning for predicting the efficacy of fluorouracil-based chemotherapy in colorectal cancer -... - November 14th, 2024 [November 14th, 2024]
- Speech production under stress for machine learning: multimodal dataset of 79 cases and 8 signals - Nature.com - November 14th, 2024 [November 14th, 2024]
- Xbox Series X Machine Learning Hardware Has Some Use Cases, But Microsoft Never Showed Interest in Doing Anything With It - Wccftech - November 14th, 2024 [November 14th, 2024]
- Get An Introduction to Optimization: With Applications to Machine Learning, 5th Edition for FREE and save $106! - BetaNews - November 14th, 2024 [November 14th, 2024]
- New Study Uses fMRI and Machine Learning to Explore Brain Function - AZoRobotics - November 14th, 2024 [November 14th, 2024]
- Introduction to Machine Learning (ML) | by Venkat | Nov, 2024 - Medium - November 14th, 2024 [November 14th, 2024]
- The future of PC gaming will be AI-driven - AMD confirms machine learning FSR 4 for 2025, launching in Call of Duty: Black Ops 6 - TechRadar - November 4th, 2024 [November 4th, 2024]
- Machine-Learning Platform Gives DoD Ability To ID Threat Network Activity - Defense Innovation Unit - November 4th, 2024 [November 4th, 2024]
- Machine Learning Offers a Water Bill Discount to Wealthy Portlander - Willamette Week - November 4th, 2024 [November 4th, 2024]