High-performance, low-cost machine learning infrastructure is accelerating innovation in the cloud – MIT Technology Review
Artificial intelligence and machine learning (AI and ML) are key technologies that help organizations develop new ways to increase sales, reduce costs, streamline business processes, and understand their customers better. AWS helps customers accelerate their AI/ML adoption by delivering powerful compute, high-speed networking, and scalable high-performance storage options on demand for any machine learning project. This lowers the barrier to entry for organizations looking to adopt the cloud to scale their ML applications.
Developers and data scientists are pushing the boundaries of technology and increasingly adopting deep learning, which is a type of machine learning based on neural network algorithms. These deep learning models are larger and more sophisticated resulting in rising costs to run underlying infrastructure to train and deploy these models.
To enable customers to accelerate their AI/ML transformation, AWS is building high-performance and low-cost machine learning chips. AWS Inferentia is the first machine learning chip built from the ground up by AWS for the lowest cost machine learning inference in the cloud. In fact, Amazon EC2 Inf1 instances powered by Inferentia, deliver 2.3x higher performance and up to 70% lower cost for machine learning inference than current generation GPU-based EC2 instances. AWS Trainium is the second machine learning chip by AWS that is purpose-built for training deep learning models and will be available in late 2021.
Customers across industries have deployed their ML applications in production on Inferentia and seen significant performance improvements and cost savings. For example, AirBnBs customer support platform enables intelligent, scalable, and exceptional service experiences to its community of millions of hosts and guests across the globe. It used Inferentia-based EC2 Inf1 instances to deploy natural language processing (NLP) models that supported its chatbots. This led to a 2x improvement in performance out of the box over GPU-based instances.
With these innovations in silicon, AWS is enabling customers to train and execute their deep learning models in production easily with high performance and throughput at significantly lower costs.
Machine learning is an iterative process that requires teams to build, train, and deploy applications quickly, as well as train, retrain, and experiment frequently to increase the prediction accuracy of the models. When deploying trained models into their business applications, organizations need to also scale their applications to serve new users across the globe. They need to be able to serve multiple requests coming in at the same time with near real-time latency to ensure a superior user experience.
Emerging use cases such as object detection, natural language processing (NLP), image classification, conversational AI, and time series data rely on deep learning technology. Deep learning models are exponentially increasing in size and complexity, going from having millions of parameters to billions in a matter of a couple of years.
Training and deploying these complex and sophisticated models translates to significant infrastructure costs. Costs can quickly snowball to become prohibitively large as organizations scale their applications to deliver near real-time experiences to their users and customers.
This is where cloud-based machine learning infrastructure services can help. The cloud provides on-demand access to compute, high-performance networking, and large data storage, seamlessly combined with ML operations and higher level AI services, to enable organizations to get started immediately and scale their AI/ML initiatives.
AWS Inferentia and AWS Trainium aim to democratize machine learning and make it accessible to developers irrespective of experience and organization size. Inferentias design is optimized for high performance, throughput, and low latency, which makes it ideal for deploying ML inference at scale.
EachAWS Inferentiachip contains four NeuronCores that implement a high-performancesystolic arraymatrix multiply engine, which massively speeds up typical deep learning operations, such as convolution and transformers. NeuronCores are also equipped with a large on-chip cache, which helps to cut down on external memory accesses, reducing latency, and increasing throughput.
AWS Neuron, the software development kit for Inferentia, natively supports leading ML frameworks, likeTensorFlow andPyTorch. Developers can continue using the same frameworks and lifecycle developments tools they know and love. For many of their trained models, they can compile and deploy them on Inferentia by changing just a single line of code, with no additional application code changes.
The result is a high-performance inference deployment, that can easily scale while keeping costs under control.
Sprinklr, a software-as-a-service company, has an AI-driven unified customer experience management platform that enables companies to gather and translate real-time customer feedback across multiple channels into actionable insights. This results in proactive issue resolution, enhanced product development, improved content marketing, and better customer service. Sprinklr used Inferentia to deploy its NLP and some of its computer vision models and saw significant performance improvements.
Several Amazon services also deploy their machine learning models on Inferentia.
Amazon Prime Video uses computer vision ML models to analyze video quality of live events to ensure an optimal viewer experience for Prime Video members. It deployed its image classification ML models on EC2 Inf1 instances and saw a 4x improvement in performance and up to a 40% savings in cost as compared to GPU-based instances.
Another example is Amazon Alexas AI and ML-based intelligence, powered by Amazon Web Services, which is available on more than 100 million devices today. Alexas promise to customers is that it is always becoming smarter, more conversational, more proactive, and even more delightful. Delivering on that promise requires continuous improvements in response times and machine learning infrastructure costs. By deploying Alexas text-to-speech ML models on Inf1 instances, it was able to lower inference latency by 25% and cost-per-inference by 30% to enhance service experience for tens of millions of customers who use Alexa each month.
As companies race to future-proof their business by enabling the best digital products and services, no organization can fall behind on deploying sophisticated machine learning models to help innovate their customer experiences. Over the past few years, there has been an enormous increase in the applicability of machine learning for a variety of use cases, from personalization and churn prediction to fraud detection and supply chain forecasting.
Luckily, machine learning infrastructure in the cloud is unleashing new capabilities that were previously not possible, making it far more accessible to non-expert practitioners. Thats why AWS customers are already using Inferentia-powered Amazon EC2 Inf1 instances to provide the intelligence behind their recommendation engines and chatbots and to get actionable insights from customer feedback.
With AWS cloud-based machine learning infrastructure options suitable for various skill levels, its clear that any organization can accelerate innovation and embrace the entire machine learning lifecycle at scale. As machine learning continues to become more pervasive, organizations are now able to fundamentally transform the customer experienceand the way they do businesswith cost-effective, high-performance cloud-based machine learning infrastructure.
Learn more about how AWSs machine learning platform can help your company innovate here.
This content was produced by AWS. It was not written by MIT Technology Reviews editorial staff.
- Open source machine learning systems are highly vulnerable to security threats - TechRadar - December 22nd, 2024 [December 22nd, 2024]
- After the PS5 Pro's less dramatic changes, PlayStation architect Mark Cerny says the next-gen will focus more on CPUs, memory, and machine-learning -... - December 22nd, 2024 [December 22nd, 2024]
- Accelerating LLM Inference on NVIDIA GPUs with ReDrafter - Apple Machine Learning Research - December 22nd, 2024 [December 22nd, 2024]
- Machine learning for the prediction of mortality in patients with sepsis-associated acute kidney injury: a systematic review and meta-analysis - BMC... - December 22nd, 2024 [December 22nd, 2024]
- Machine learning uncovers three osteosarcoma subtypes for targeted treatment - Medical Xpress - December 22nd, 2024 [December 22nd, 2024]
- From Miniatures to Machine Learning: Crafting the VFX of Alien: Romulus - Animation World Network - December 22nd, 2024 [December 22nd, 2024]
- Identification of hub genes, diagnostic model, and immune infiltration in preeclampsia by integrated bioinformatics analysis and machine learning -... - December 22nd, 2024 [December 22nd, 2024]
- This AI Paper from Microsoft and Novartis Introduces Chimera: A Machine Learning Framework for Accurate and Scalable Retrosynthesis Prediction -... - December 18th, 2024 [December 18th, 2024]
- Benefits and Challenges of Integrating AI and Machine Learning into EHR Systems - Healthcare IT Today - December 18th, 2024 [December 18th, 2024]
- The History Of AI: How Machine Learning's Evolution Is Reshaping Everything Around Us - SlashGear - December 18th, 2024 [December 18th, 2024]
- AI and Machine Learning to Enhance Pension Plan Governance and the Investor Experience: New CFA Institute Research - Fintech Finance - December 18th, 2024 [December 18th, 2024]
- Address Common Machine Learning Challenges With Managed MLflow - The New Stack - December 18th, 2024 [December 18th, 2024]
- Machine Learning Used To Classify Fossils Of Extinct Pollen - Offworld Astrobiology Applications? - Astrobiology News - December 18th, 2024 [December 18th, 2024]
- Machine learning model predicts CDK4/6 inhibitor effectiveness in metastatic breast cancer - News-Medical.Net - December 18th, 2024 [December 18th, 2024]
- New Lockheed Martin Subsidiary to Offer Machine Learning Tools to Defense Customers - ExecutiveBiz - December 18th, 2024 [December 18th, 2024]
- How Powerful Will AI and Machine Learning Become? - International Policy Digest - December 18th, 2024 [December 18th, 2024]
- ChatGPT-Assisted Machine Learning for Chronic Disease Classification and Prediction: A Developmental and Validation Study - Cureus - December 18th, 2024 [December 18th, 2024]
- Blood Tests Are Far From Perfect But Machine Learning Could Change That - Inverse - December 18th, 2024 [December 18th, 2024]
- Amazons AGI boss: You dont need a PhD in machine learning to build with AI anymore - Fortune - December 18th, 2024 [December 18th, 2024]
- From Novice to Pro: A Roadmap for Your Machine Learning Career - KDnuggets - December 10th, 2024 [December 10th, 2024]
- Dimension nabs $500M second fund for 'still contrary' intersection of bio and machine learning - Endpoints News - December 10th, 2024 [December 10th, 2024]
- Using Machine Learning to Make A Really Big Detailed Simulation - Astrobites - December 10th, 2024 [December 10th, 2024]
- Driving Business Growth with GreenTomatos Data and Machine Learning Strategy on Generative AI - AWS Blog - December 10th, 2024 [December 10th, 2024]
- Unlocking the power of data analytics and machine learning to drive business performance - WTW - December 10th, 2024 [December 10th, 2024]
- AI and the Ethics of Machine Learning | by Abwahabanjum | Dec, 2024 - Medium - December 10th, 2024 [December 10th, 2024]
- Differentiating Cystic Lesions in the Sellar Region of the Brain Using Artificial Intelligence and Machine Learning for Early Diagnosis: A Prospective... - December 10th, 2024 [December 10th, 2024]
- New Amazon SageMaker AI Innovations Reimagine How Customers Build and Scale Generative AI and Machine Learning Models - Amazon Press Release - December 10th, 2024 [December 10th, 2024]
- What is Machine Learning? 18 Crucial Concepts in AI, ML, and LLMs - Netguru - December 5th, 2024 [December 5th, 2024]
- Machine learning-based prediction of antibiotic resistance in Mycobacterium tuberculosis clinical isolates from Uganda - BMC Infectious Diseases - December 5th, 2024 [December 5th, 2024]
- Interdisciplinary Team Needed to Apply Machine Learning in Epilepsy Surgery: Lara Jehi, MD, MHCDS - Neurology Live - December 5th, 2024 [December 5th, 2024]
- A multimodal machine learning model for the stratification of breast cancer risk - Nature.com - December 5th, 2024 [December 5th, 2024]
- Machine learning based intrusion detection framework for detecting security attacks in internet of things - Nature.com - December 5th, 2024 [December 5th, 2024]
- Machine learning evaluation of a hypertension screening program in a university workforce over five years - Nature.com - December 5th, 2024 [December 5th, 2024]
- Vaultree Introduces VENum Stack: Combining the Power of Machine Learning and Encrypted Data Processing for Secure Innovation - PR Newswire - December 5th, 2024 [December 5th, 2024]
- Direct simulation and machine learning structure identification unravel soft martensitic transformation and twinning dynamics - pnas.org - December 5th, 2024 [December 5th, 2024]
- AI and Machine Learning - Maryland to use AI technology to manage traffic flow - SmartCitiesWorld - December 5th, 2024 [December 5th, 2024]
- Researchers make machine learning breakthrough in lithium-ion tech here's how it could make aging batteries safer - Yahoo! Voices - December 5th, 2024 [December 5th, 2024]
- Integrating IoT and machine learning: Benefits and use cases - TechTarget - December 5th, 2024 [December 5th, 2024]
- Landsat asks industry for artificial intelligence (AI) and machine learning for satellite operations - Military & Aerospace Electronics - December 5th, 2024 [December 5th, 2024]
- Machine learning optimized efficient graphene-based ultra-broadband solar absorber for solar thermal applications - Nature.com - December 5th, 2024 [December 5th, 2024]
- Polymathic AI Releases The Well: 15TB of Machine Learning Datasets Containing Numerical Simulations of a Wide Variety of Spatiotemporal Physical... - December 5th, 2024 [December 5th, 2024]
- Prediction of preterm birth using machine learning: a comprehensive analysis based on large-scale preschool children survey data in Shenzhen of China... - December 5th, 2024 [December 5th, 2024]
- Application of machine learning algorithms to identify serological predictors of COVID-19 severity and outcomes - Nature.com - November 30th, 2024 [November 30th, 2024]
- Predicting the time to get back to work using statistical models and machine learning approaches - BMC Medical Research Methodology - November 30th, 2024 [November 30th, 2024]
- AI and Machine Learning - US releases recommendations for use of AI in critical infrastructure - SmartCitiesWorld - November 30th, 2024 [November 30th, 2024]
- Machine learning-based diagnostic model for stroke in non-neurological intensive care unit patients with acute neurological manifestations -... - November 28th, 2024 [November 28th, 2024]
- Analysis of four long non-coding RNAs for hepatocellular carcinoma screening and prognosis by the aid of machine learning techniques - Nature.com - November 28th, 2024 [November 28th, 2024]
- Evaluation and prediction of the physical properties and quality of Jatob-do-Cerrado seeds processed and stored in different conditions using machine... - November 28th, 2024 [November 28th, 2024]
- Researchers use fitness tracker data and machine learning to detect bipolar disorder mood swings - Medical Xpress - November 28th, 2024 [November 28th, 2024]
- Advances in AI and Machine Learning for Nuclear Applications - Frontiers - November 28th, 2024 [November 28th, 2024]
- Researchers make machine learning breakthrough in lithium-ion tech here's how it could make aging batteries safer - The Cool Down - November 28th, 2024 [November 28th, 2024]
- Svitla Systems Publishes Results of the Study on Machine Learning's Role in Credit Scoring - Newsfile - November 28th, 2024 [November 28th, 2024]
- Predicting poor performance on cognitive tests among older adults using wearable device data and machine learning: a feasibility study - Nature.com - November 28th, 2024 [November 28th, 2024]
- Quantum Machine Learning: Bridging the Future of AI and Quantum Computing - TechBullion - November 28th, 2024 [November 28th, 2024]
- AI and machine learning trends in healthcare - Healthcare Leader - November 28th, 2024 [November 28th, 2024]
- Identification of biomarkers for the diagnosis in colorectal polyps and metabolic dysfunction-associated steatohepatitis (MASH) by bioinformatics... - November 28th, 2024 [November 28th, 2024]
- Revolutionizing Business Systems with Machine Learning: Practical Innovations for the Modern Era - TechBullion - November 28th, 2024 [November 28th, 2024]
- Can AI improve plant-based meats? Using mechanical testing and machine learning to mimic the sensory experience - Phys.org - November 16th, 2024 [November 16th, 2024]
- Machine Learning Reveals Impact of Microbial Load on Gut Health and Disease - Genetic Engineering & Biotechnology News - November 16th, 2024 [November 16th, 2024]
- Machine learning for predicting in-hospital mortality in elderly patients with heart failure combined with hypertension: a multicenter retrospective... - November 16th, 2024 [November 16th, 2024]
- Apple Researchers Propose Cut Cross-Entropy (CCE): A Machine Learning Method that Computes the Cross-Entropy Loss without Materializing the Logits for... - November 16th, 2024 [November 16th, 2024]
- Exploring electron-beam induced modifications of materials with machine-learning assisted high temporal resolution electron microscopy - Nature.com - November 16th, 2024 [November 16th, 2024]
- Facilitated the discovery of new / Co-based superalloys by combining first-principles and machine learning - Nature.com - November 16th, 2024 [November 16th, 2024]
- Thwarting Phishing Attacks with Predictive Analytics and Machine Learning in 2024 - Petri.com - November 16th, 2024 [November 16th, 2024]
- Optoelectronic performance prediction of HgCdTe homojunction photodetector in long wave infrared spectral region using traditional simulations and... - November 16th, 2024 [November 16th, 2024]
- A new approach for sex prediction by evaluating mandibular arch and canine dimensions with machine-learning classifiers and intraoral scanners (a... - November 16th, 2024 [November 16th, 2024]
- AI and Machine Learning - Google and National League of Cities develop AI toolkit - SmartCitiesWorld - November 16th, 2024 [November 16th, 2024]
- Machine learning for the physics of climate - Nature.com - November 14th, 2024 [November 14th, 2024]
- Red Hat acquires tech to lower the cost of machine learning - ComputerWeekly.com - November 14th, 2024 [November 14th, 2024]
- SUU Professor Receives Grant to Develop Machine Learning Certificate - Southern Utah University - November 14th, 2024 [November 14th, 2024]
- Research on the timing for subsequent water flooding in Alkali-Surfactant-Polymer flooding in Daqing Oilfield based on automated machine learning -... - November 14th, 2024 [November 14th, 2024]
- SNPs and blood inflammatory marker featured machine learning for predicting the efficacy of fluorouracil-based chemotherapy in colorectal cancer -... - November 14th, 2024 [November 14th, 2024]
- Speech production under stress for machine learning: multimodal dataset of 79 cases and 8 signals - Nature.com - November 14th, 2024 [November 14th, 2024]
- Xbox Series X Machine Learning Hardware Has Some Use Cases, But Microsoft Never Showed Interest in Doing Anything With It - Wccftech - November 14th, 2024 [November 14th, 2024]
- Get An Introduction to Optimization: With Applications to Machine Learning, 5th Edition for FREE and save $106! - BetaNews - November 14th, 2024 [November 14th, 2024]
- New Study Uses fMRI and Machine Learning to Explore Brain Function - AZoRobotics - November 14th, 2024 [November 14th, 2024]
- Introduction to Machine Learning (ML) | by Venkat | Nov, 2024 - Medium - November 14th, 2024 [November 14th, 2024]
- The future of PC gaming will be AI-driven - AMD confirms machine learning FSR 4 for 2025, launching in Call of Duty: Black Ops 6 - TechRadar - November 4th, 2024 [November 4th, 2024]
- Machine-Learning Platform Gives DoD Ability To ID Threat Network Activity - Defense Innovation Unit - November 4th, 2024 [November 4th, 2024]
- Machine Learning Offers a Water Bill Discount to Wealthy Portlander - Willamette Week - November 4th, 2024 [November 4th, 2024]