Mediaboss Marketing

From protein engineering and 3D printing to detection of deepfake media, here are seven areas of technology that Nature will be watching in the year ahead.

Two decades ago, David Baker at the University of Washington in Seattle and his colleagues achieved a landmark feat: they used computational tools to design an entirely new protein from scratch. Top7 folded as predicted, but it was inert: it performed no meaningful biological functions. Today, de novo protein design has matured into a practical tool for generating made-to-order enzymes and other proteins. Its hugely empowering, says Neil King, a biochemist at the University of Washington who collaborates with Bakers team to design protein-based vaccines and vehicles for drug delivery. Things that were impossible a year and a half ago now you just do it.

Much of that progress comes down to increasingly massive data sets that link protein sequence to structure. But sophisticated methods of deep learning, a form of artificial intelligence (AI), have also been essential.

Sequence based strategies use the large language models (LLMs) that power tools such as the chatbot ChatGPT (see ChatGPT? Maybe next year). By treating protein sequences like documents comprising polypeptide words, these algorithms can discern the patterns that underlie the architectural playbook of real-world proteins. They really learn the hidden grammar, says Noelia Ferruz, a protein biochemist at the Molecular Biology Institute of Barcelona, Spain. In 2022, her team developed an algorithm called ProtGPT2 that consistently comes up with synthetic proteins that fold stably when produced in the laboratory1. Another tool co-developed by Ferruz, called ZymCTRL, draws on sequence and functional data to design members of naturally occurring enzyme families2.

Readers might detect a theme in this years technologies to watch: the outsized impact of deep-learning methods. But one such tool did not make the final cut: the much-hyped artificial-intelligence (AI)-powered chatbots. ChatGPT and its ilk seem poised to become part of many researchers daily routines and were feted as part of the 2023 Natures 10 round-up (see go.nature.com/3trp7rg). Respondents to a Nature survey in September (see go.nature.com/45232vd) cited ChatGPT as the most useful AI-based tool and were enthusiastic about its potential for coding, literature reviews and administrative tasks.

Such tools are also proving valuable from an equity perspective, helping those for whom English isnt their first language to refine their prose and thereby ease their paths to publication and career growth. However, many of these applications represent labour-saving gains rather than transformations of the research process. Furthermore, ChatGPTs persistent issuing of either misleading or fabricated responses was the leading concern of more than two-thirds of survey respondents. Although worth monitoring, these tools need time to mature and to establish their broader role in the scientific world.

Sequence-based approaches can build on and adapt existing protein features to form new frameworks, but theyre less effective for the bespoke design of structural elements or features, such as the ability to bind specific targets in a predictable fashion. Structure based approaches are better for this, and 2023 saw notable progress in this type of protein-design algorithm, too. Some of the most sophisticated of these use diffusion models, which also underlie image-generating tools such as DALL-E. These algorithms are initially trained to remove computer-generated noise from large numbers of real structures; by learning to discriminate realistic structural elements from noise, they gain the ability to form biologically plausible, user-defined structures.

RFdiffusion software3 developed by Bakers lab and the Chroma tool by Generate Biomedicines in Somerville, Massachusetts4, exploit this strategy to remarkable effect. For example, Bakers team is using RFdiffusion to engineer novel proteins that can form snug interfaces with targets of interest, yielding designs that just conform perfectly to the surface, Baker says. A newer all atom iteration of RFdiffusion5 allows designers to computationally shape proteins around non-protein targets such as DNA, small molecules and even metal ions. The resulting versatility opens new horizons for engineered enzymes, transcriptional regulators, functional biomaterials and more.

The explosion of publicly available generative AI algorithms has made it simple to synthesize convincing, but entirely artificial images, audio and video. The results can offer amusing distractions, but with multiple ongoing geopolitical conflicts and a US presidential election on the horizon, opportunities for weaponized media manipulation are rife.

Siwei Lyu, a computer scientist at the University at Buffalo in New York, says hes seen numerous AI-generated deepfake images and audio related to the IsraelHamas conflict, for instance. This is just the latest round in a high-stakes game of cat-and-mouse in which AI users produce deceptive content and Lyu and other media-forensics specialists work to detect and intercept it.

AI and science: what 1,600 researchers think

One solution is for generative-AI developers to embed hidden signals in the models output, producing watermarks of AI-generated content. Other strategies focus on the content itself. Some manipulated videos, for instance, replace the facial features of one public figure with those of another, and new algorithms can recognize artefacts at the boundaries of the substituted features, says Lyu. The distinctive folds of a persons outer ear can also reveal mismatches between a face and a head, whereas irregularities in the teeth can reveal edited lip-sync videos in which a persons mouth was digitally manipulated to say something that the subject didnt say. AI-generated photos also present a thorny challenge and a moving target. In 2019, Luisa Verdoliva, a media-forensics specialist at University Federico II of Naples, Italy, helped to develop FaceForensics++, a tool for spotting faces manipulated by several widely used software packages6. But image-forensic methods are subject- and software-specific, and generalization is a challenge. You cannot have one single universal detector its very difficult, she says.

And then theres the challenge of implementation. The US Defense Advanced Research Projects Agencys Semantic Forensics (SemaFor) programme has developed a useful toolbox for deepfake analysis, but, as reported in Nature (see Nature 621, 676679; 2023) major social-media sites are not routinely employing it. Broadening the access to such tools could help to fuel uptake, and to this end Lyus team has developed the DeepFake-O-Meter7, a centralized public repository of algorithms that can analyse video content from different angles to sniff out deepfake content. Such resources will be helpful, but it is likely that the battle against AI-generated misinformation will persist for years to come.

In late 2023, US and UK regulators approved the first-ever CRISPR-based gene-editing therapy for sickle-cell disease and transfusion-dependent -thalassaemia a major win for genome editing as a clinical tool.

CRISPR and its derivatives use a short programmable RNA to direct a DNA-cutting enzyme such as Cas9 to a specific genomic site. They are routinely used in the lab to disable defective genes and introduce small sequence changes. The precise and programmable insertion of larger DNA sequences spanning thousands of nucleotides is difficult, but emerging solutions could allow scientists to replace crucial segments of defective genes or insert fully functional gene sequences. Le Cong, a molecular geneticist at Stanford University in California and his colleagues are exploring single-stranded annealing proteins (SSAPs) virus-derived molecules that mediate DNA recombination. When combined with a CRISPRCas system in which the DNA-slicing function of Cas9 has been disabled, these SSAPs allow precisely targeted insertion of up to 2 kilobases of DNA into the human genome.

Seven technologies to watch in 2023

Other methods exploit a CRISPR-based method called prime editing to introduce short landing pad sequences that selectively recruit enzymes that in turn can precisely splice large DNA fragments into the genome. In 2022, for instance, genome engineers Omar Abudayyeh and Jonathan Gootenberg at the Massachusetts Institute of Technology, Cambridge and their colleagues first described programmable addition through site-specific targeting elements (PASTE), a method that can precisely insert up to 36 kilobases of DNA8. PASTE is especially promising for ex vivo modification of cultured, patient-derived cells, says Cong, and the underlying prime-editing technology is already on track for clinical studies. But for in vivo modification of human cells, SSAP might offer a more compact solution: the bulkier PASTE machinery requires three separate viral vectors for delivery, which could undermine editing efficiency relative to the two-component SSAP system. That said, even relatively inefficient gene-replacement strategies could be sufficient to mitigate the effects of many genetic diseases.

And such methods are not just relevant to human health. Researchers led by Caixia Gao at the Chinese Academy of Sciences in Beijing developed PrimeRoot, a method that uses prime editing to introduce specific target sites that enzymes can use to insert up to 20 kilobases of DNA in both rice and maize9. Gao thinks that the technique could be broadly useful for endowing crops with disease and pathogen resistance, continuing a wave of innovation in CRISPR-based plant genome engineering. I believe that this technology can be applied in any plant species, she says.

Pat Bennett has slower than average speech, and can sometimes use the wrong word. But given that motor neuron disease, also known as amyotrophic lateral sclerosis, had previously left her unable to express herself verbally, that is a remarkable achievement.

Bennetts recovery comes courtesy of a sophisticated braincomputer interface (BCI) device developed by Stanford University neuroscientist Francis Willett and his colleagues at the US-based BrainGate consortium10. Willett and his colleagues implanted electrodes in Bennetts brain to track neuronal activity and then trained deep-learning algorithms to translate those signals into speech. After a few weeks of training, Bennett was able to say as many as 62 words per minute from a vocabulary of 125,000 words more than twice the vocabulary of the average English speaker. Its really truly impressive, the rates at which theyre communicating, says bioengineer Jennifer Collinger, who develops BCI technologies at the University of Pittsburgh in Pennsylvania.

Braincomputer interface technology has allowed Pat Bennett (seated) to regain her speech.Credit: Steve Fisch/Stanford Medicine

BrainGates trial is just one of several studies from the past few years demonstrating how BCI technology can help people with severe neurological damage to regain lost skills and achieve greater independence. Some of that progress stems from the steady accumulation of knowledge about functional neuroanatomy in the brains of individuals with various neurological conditions, says Leigh Hochberg, a neurologist at Brown University in Providence, Rhode Island, and director of the BrainGate consortium. But that knowledge has been greatly amplified, he adds, by machine-learning-driven analytical methods that are revealing how to better place electrodes and decrypt the signals that they pick up.

Researchers are also applying AI-based language models to speed up the interpretation of what patients are trying to communicate essentially, autocomplete for the brain. This was a core component of Willetts study, as well as another11 from a team led by neurosurgeon Edward Chang at the University of California, San Francisco. In that work, a BCI neuroprosthesis allowed a woman who was unable to speak as a result of a stroke to communicate at 78 words per minute roughly half the average speed of English, but more than five times faster than the womans previous speech-assistance device. The field is seeing progress in other areas as well. In 2021, Collinger and biomedical engineer Robert Gaunt at the University of Pittsburgh implanted electrodes into the motor and somatosensory cortex of an individual who was paralysed in all four limbs to provide rapid and precise control over a robotic arm along with tactile sensory feedback12. Also under way are independent clinical studies from BrainGate and researchers at UMC Utrecht in the Netherlands, as well as a trial from BCI firm Synchron in Brooklyn, New York, to test a system that allows people who are paralysed to control a computer the first industry-sponsored trial of a BCI apparatus.

As an intensive-care specialist, Hochberg is eager to deliver these technologies to his patients with the most severe disabilities. But as BCI capabilities evolve, he sees potential to treat more-moderate cognitive impairments as well as mental-health conditions, such as mood disorders. Closed-loop neuromodulation systems informed by braincomputer interfaces could be of tremendous help to a lot of people, he says.

Stefan Hell, Eric Betzig and William Moerner were awarded the 2014 Nobel Prize in Chemistry for shattering the diffraction limit that constrained the spatial resolution of light microscopy. The resulting level of detail in the order of tens of nanometres opened a wide range of molecular-scale imaging experiments. Still, some researchers yearn for better and they are making swift progress. Were really trying to close the gap from super-resolution microscopy to structural-biology techniques like cryo-electron microscopy, says Ralf Jungmann, a nanotechnology researcher at the Max Planck Institute of Biochemistry in Planegg, Germany, referring to a method that can reconstruct protein structures with atomic-scale resolution.

Researchers led by Hell and his team at the Max Planck Institute for Multidisciplinary Sciences in Gttingen made an initial foray into this realm in late 2022 with a method called MINSTED that can resolve individual fluorescent labels with 2.3-ngstrm precision roughly one-quarter of a nanometre using a specialized optical microscope13.

Newer methods provide comparable resolution using conventional microscopes. Jungmann and his team, for instance, described a strategy in 2023 in which individual molecules are labelled with distinct DNA strands14. These molecules are then detected with dye-tagged complementary DNA strands that bind to their corresponding targets transiently but repeatedly, making it possible to discriminate individual fluorescent blinking points that would blur into a single blob if imaged simultaneously. This resolution enhancement by sequential imaging (RESI) approach could resolve individual base pairs on a DNA strand, demonstrating ngstrm-scale resolution with a standard fluorescence microscope.

The one-step nanoscale expansion (ONE) microscopy method, developed by a team led by neuroscientists Ali Shaib and Silvio Rizzoli at University Medical Center Gttingen, Germany, doesnt quite achieve this level of resolution. However, ONE microscopy offers an unprecedented opportunity to directly image fine structural details of individual proteins and multiprotein complexes, both in isolation and in cells15.

A form of imaging called RESI could allow the imaging of individual base pairs in DNA.Credit: Max Iglesias, Max Planck Institute of Biochemistry

ONE is an expansion-microscopy-based approach that involves chemically coupling proteins in the sample to a hydrogel matrix, breaking the proteins apart, and then allowing the hydrogel to expand 1,000-fold in volume. The fragments expand evenly in all directions, preserving the protein structure and enabling users to resolve features separated by a few nanometres with a standard confocal microscope. We took antibodies, put them in the gel, labelled them after expansion, and were like, Oh we see Y shapes! says Rizzoli, referring to the characteristic shape of the proteins.

ONE microscopy could provide insights into conformationally dynamic biomolecules or enable visual diagnosis of protein-misfolding disorders such as Parkinsons disease from blood samples, says Rizzoli. Jungmann is similarly enthusiastic about the potential for RESI to document reorganization of individual proteins in disease or in response to drug treatments. It might even be possible to zoom in more tightly. Maybe its not the end for the spatial resolution limits, Jungmann says. It might get better.

If youre looking for a convenient cafe, Google Maps can find nearby options and tell you how to get there. Theres no equivalent for navigating the much more complex landscape of the human body, but ongoing progress from various cell-atlas initiatives powered by advances in single-cell analysis and spatial omics methods could soon deliver the tissue-wide cellular maps that biologists crave.

The largest and perhaps the most ambitious of these initiatives is the Human Cell Atlas (HCA). The consortium was launched in 2016 by cell biologist Sarah Teichmann at the Wellcome Sanger Institute in Hinxton, UK, and Aviv Regev, now head of research and early development at biotechnology firm Genentech in South San Francisco, California. It encompasses some 3,000 scientists in nearly 100 countries, working with tissues from 10,000 donors. But HCA is also part of a broader ecosystem of intersecting cellular and molecular atlas efforts. These include the Human BioMolecular Atlas Program (HuBMAP) and the Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative Cell Census Network (BICCN), both funded by the US National Institutes of Health, as well as the Allen Brain Cell Atlas, funded by the Allen Institute in Seattle, Washington.

According to Michael Snyder, a genomicist at Stanford University and former co-chair of the HuBMAP steering committee, these efforts have been driven in part by the development and rapid commercialization of analytical tools that can decode molecular contents at the single-cell level. For example, Snyders team routinely uses the Xenium platform from 10X Genomics in Pleasanton, California, for its spatial transcriptomics analyses. The platform makes it possible to survey the expression of roughly 400 genes at once in 4 tissue samples every week. Multiplexed antibody-based methods such as the PhenoCycler platform by Akoya Biosciences in Marlborough, Massachusetts, allow the team to track large numbers of proteins with single-cell resolution in a format that enables 3D tissue reconstruction. Other multiomics methods allow scientists to profile multiple molecular classes in the same cell at once, including the expression of RNA, the structure of chromatin and the distribution of protein.

A cell atlas of the human lung describes different cell types and how they are regulated.Credit: Peng He

Last year saw dozens of studies showcasing progress in the generation of organ-specific atlases using these techniques. In June, for example, the HCA released an integrated analysis of 49 data sets from the human lung16. Having that very clear map of the lung informs the changes that happen in diseases like lung fibrosis, different tumours, even for COVID-19, says Teichmann. And in 2023, Nature released an article collection (see go.nature.com/3vbznk7) highlighting progress from HuBMAP and Science produced a collection detailing the work of the BICCN (see go.nature.com/3nsf4ys).

Considerable work remains Teichmann estimates that it will be at least five years before the HCA reaches completion. But the resulting maps will be invaluable when they arrive. Teichmann, for example, predicts using atlas data to guide tissue- and cell-specific drug targeting, while Snyder is eager to learn how cellular microenvironments inform the risk and aetiology of complex disorders such as cancer and irritable bowel syndrome. Will we solve that in 2024? I dont think so its a multiyear problem, Snyder says. But its a big driver for this whole field.

Weird and interesting things can happen at the nanometre scale. This can make materials-science predictions difficult, but it also means that nanoscale architects can manufacture lightweight materials with distinctive characteristics such as increased strength, tailored interactions with light or sound, and enhanced capacity for catalysis or energy storage.

Several strategies exist for precisely crafting such nanomaterials, most of which use lasers to induce patterned photopolymerization of light-sensitive materials, and over the past few years, scientists have made considerable headway in overcoming the limitations that have impeded broader adoption of these methods.

Researchers have crafted microscale metal structures using a hydrogel.Credit: Max Saccone/Greer Lab

One is speed. Sourabh Saha, an engineer at the Georgia Institute of Technology in Atlanta, says that the assembly of nanostructures using photopolymerization is roughly three orders of magnitude faster than other nanoscale 3D-printing methods. That might be good enough for lab use, but its too slow for large-scale production or industrial processes. In 2019, Saha and mechanical engineer Shih-Chi Chen at the Chinese University of Hong Kong and their colleagues showed that they could accelerate polymerization by using a patterned 2D light-sheet rather than a conventional pulsed laser17. That increases the rate by a thousand times, and you can still maintain those 100-nanometre features, says Saha. Subsequent work from researchers including Chen has identified other avenues for faster nanofabrication18.

Another challenge is that not all materials can be printed directly through photopolymerization such as metals. But Julia Greer, a materials scientist at the California Institute of Technology in Pasadena, has developed a clever workaround. In 2022, she and her colleagues described a method in which photopolymerized hydrogels serve as a microscale template; these are then infused with metal salts and processed in a way that induces the metal to assume the structure of the template while also shrinking19. Although the technique was initially developed for microscale structures, Greers team has also used this strategy for nanofabrication, and the researchers are enthusiastic about the potential to craft functional nanostructures from rugged, high-melting-point metals and alloys.

The final barrier economics could be the toughest to break. According to Saha, the pulsed-laser-based systems used in many photopolymerization methods cost upwards of US$500,000. But cheaper alternatives are emerging. For example, physicist Martin Wegener and his colleagues at the Karlsruhe Institute of Technology in Germany have explored continuous lasers that are cheaper, more compact, and consume less power than standard pulsed lasers20. And Greer has launched a start-up company to commercialize a process for fabricating nanoarchitected metal sheets that could be suitable for applications such as next-generation body armour or ultra-durable and impact-resistant outer layers for aircraft and other vehicles.

Read this article:
Seven technologies to watch in 2024 - Nature.com

In the hunt for new materials, scientists have traditionally relied on tinkering in the lab, guided by intuition, with a hefty serving of trial and error.

But now a new battery material has been discovered by combining two computing superpowers: artificial intelligence and supercomputing. Its a discovery that highlights the potential for using computers to help scientists discover materials suited to specific needs, from batteries to carbon capture technologies to catalysts.

Calculationswinnowed down more than 32 million candidate materialsto just 23 promising options, researchers from Microsoft and Pacific Northwest National Laboratory, or PNNL, report in a paper submitted January 8 to arXiv.org. The team then synthesized and tested one of those materials and created a working battery prototype.

While scientists have used AI to predict materials properties before, previous studies typically havent seen that process through to producing the new material. The nice thing about this paper is that it goes all the way from start to finish, says computational materials scientist Shyue Ping Ong of the University of California, San Diego, who was not involved with the research.

The researchers targeted a coveted type of battery material: a solid electrolyte. An electrolyte is a material that transfers ions electrically charged atoms back and forth between a batterys electrodes. In standard lithium-ion batteries, the electrolyte is a liquid. But that comes with hazards, like batteries leaking or causing fires. Developing batteries with solid electrolytes is a major aim of materials scientists.

The original 32 million candidates were generated via a game of mix-and-match, substituting different elements in crystal structures of known materials. Sorting through a list this large with traditional physics calculations would have taken decades, says computational chemist Nathan Baker of Microsoft. But with machine learning techniques, which can make quick predictions based on patterns learned from known materials, the calculation produced results in just 80 hours.

First, the researchers used AI to filter the materials based on stability, namely, whether they could actually exist in the real world. That pared the list down to fewer than 600,000 candidates. Further AI analysis selected candidates likely to have the electrical and chemical properties necessary for batteries. Because AI models are approximate, the researchers filtered this smaller list using tried-and-tested, computationally intensive methods based on physics. They also weeded out rare, toxic or expensive materials.

That left the researchers with 23 candidates, five of which were already known. Researchers at PNNL picked a material that looked promising it was related to other materials that the researchers knew how to make in the lab, and it had suitable stability and conductivity. Then they set to work synthesizing it, eventually fashioning it into a prototype battery. And it worked.

Thats when we got very excited, says materials scientist Vijay Murugesan of PNNL in Richland, Wash. Going from the synthesis stage to the functional battery took about six months. That is superfast.

The new electrolyte is similar to a known material containing lithium, yttrium and chlorine,but swaps some lithium for sodium an advantage aslithium is costly and in high demand(SN: 5/7/19).

Combining lithium and sodium is unconventional. In a usual approach we would not mix these two together, says materials scientist Yan Zeng of Florida State University in Tallahassee, who was not involved in the research. The typical practice is to use either lithium or sodium ions as a conductor, not both. The two types of ions might be expected to compete with one another, resulting in worse performance. The unorthodox material highlights one hope for AI in research, Zeng says: AI can sort of step out of the box.

In the new work, the researchers created a series of AI models that could predict different properties of a material, based on training data from known materials. The AI architecture is a type known as a graph neural network, in which a system is represented as a graph, a mathematical structure composed of edges and nodes. This type of model is particularly suited for describing materials, as the nodes can represent atoms, and the edges can represent bonds between the elements.

To perform both the AI and physics-based calculations, the team used Microsofts Azure Quantum Elements, which provides access to a cloud-based supercomputer tailored for chemistry and materials science research.

The project, Baker says, is an example of a practice known in tech circles as eating your own dog food, in which a company uses its own product to confirm that it works. In the future, he says he hopes others will pick up the tool and use it for a variety of scientific endeavors.

The study is one of many efforts to use AI to discover new materials. In November, researchers from Google DeepMind used graph neural networks to predict the existence ofhundreds of thousands of stable materials, they reported in the Dec. 7Nature. And in the same issue ofNature,Zeng and colleagues reported on alaboratoryoperated by AI,designed to produce new materials autonomously.

Read the rest here:
Artificial intelligence helped scientists create a new type of battery - Science News Magazine

The World Health Organization (WHO) is releasing new guidance on the ethics and governance of large multi-modal models (LMMs) a type of fast growing generative artificial intelligence (AI) technology with applications across health care.

The guidance outlines over 40 recommendations for consideration by governments, technology companies, and health care providers to ensure the appropriate use of LMMs to promote and protect the health of populations.

LMMs can accept one or more type of data inputs, such as text, videos, and images, and generate diverse outputs not limited to the type of data inputted. LMMs are unique in their mimicry of human communication and ability to carry out tasks they were not explicitly programmed to perform. LMMs have been adopted faster than any consumer application in history, with several platforms such as ChatGPT, Bard and Bert entering the public consciousness in 2023.

Generative AI technologies have the potential to improve health care but only if those who develop, regulate, and use these technologies identify and fully account for the associated risks, said Dr Jeremy Farrar, WHO Chief Scientist. We need transparent information and policies to manage the design, development, and use of LMMs to achieve better health outcomes and overcome persisting health inequities.

The new WHO guidance outlines five broad applications of LMMs for health:

While LMMs are starting to be used for specific health-related purposes, there are also documented risks of producing false, inaccurate, biased, or incomplete statements, which could harm people using such information in making health decisions. Furthermore, LMMs may be trained on data that are of poor quality or biased, whether by race, ethnicity, ancestry, sex, gender identity, or age.

The guidance also details broader risks to health systems, such as accessibility and affordability of the best-performing LMMs. LMMS can also encourage automation bias by health care professionals and patients, whereby errors are overlooked that would otherwise have been identified or difficult choices are improperly delegated to a LMM. LMMs, like other forms of AI, are also vulnerable to cybersecurity risks that could endanger patient information or the trustworthiness of these algorithms and the provision of health care more broadly.

To create safe and effective LMMs, WHO underlines the need for engagement of various stakeholders: governments, technology companies, healthcare providers, patients, and civil society, in all stages of development and deployment of such technologies, including their oversight and regulation.

Governments from all countries must cooperatively lead efforts to effectively regulate the development and use of AI technologies, such as LMMs, said Dr Alain Labrique, WHO Director for Digital Health and Innovation in the Science Division.

The new WHO guidance includes recommendations for governments, who have the primary responsibility to set standards for the development and deployment of LMMs, and their integration and use for public health and medical purposes. For example, governments should:

The guidance also includes the following key recommendations for developers of LMMs, who should ensure that:

The new document on Ethics and governance of AI for health Guidance on large multi-modal models is based on WHOs guidance published in June 2021. Access the publication here

See the original post:
WHO releases AI ethics and governance guidance for large multi-modal models - World Health Organization

In 1960, Herbert Simon, who went on to win both the Nobel Prize for economics and the Turing Award for computer science, wrote in his book The New Science of Management Decision that machines will be capable, within 20 years, of doing any work that a man can do.

History is filled with exuberant technological predictions that have failed to materialize. Within the field of artificial intelligence, the brashest predictions have concerned the arrival of systems that can perform any task a human can, often referred to as artificial general intelligence, or AGI.

So when Shane Legg, Google DeepMinds co-founder and chief AGI scientist, estimates that theres a 50% chance that AGI will be developed by 2028, it might be tempting to write him off as another AI pioneer who hasnt learnt the lessons of history.

Still, AI is certainly progressing rapidly. GPT-3.5, the language model that powers OpenAIs ChatGPT was developed in 2022, and scored 213 out of 400 on the Uniform Bar Exam, the standardized test that prospective lawyers must pass, putting it in the bottom 10% of human test-takers. GPT-4, developed just months later, scored 298, putting it in the top 10%. Many experts expect this progress to continue.

Read More: 4 Charts That Show Why AI Progress Is Unlikely to Slow Down

Leggs views are common among the leadership of the companies currently building the most powerful AI systems. In August, Dario Amodei, co-founder and CEO of Anthropic, said he expects a human-level AI could be developed in two to three years. Sam Altman, CEO of OpenAI, believes AGI could be reached sometime in the next four or five years.

But in a recent survey the majority of 1,712 AI experts who responded to the question of when they thought AI would be able to accomplish every task better and more cheaply than human workers were less bullish. A separate survey of elite forecasters with exceptional track records shows they are less bullish still.

The stakes for divining who is correct are high. Legg, like many other AI pioneers, has warned that powerful future AI systems could cause human extinction. And even for those less concerned by Terminator scenarios, some warn that an AI system that could replace humans at any task might replace human labor entirely.

Many of those working at the companies building the biggest and most powerful AI models believe that the arrival of AGI is imminent. They subscribe to a theory known as the scaling hypothesis: the idea that even if a few incremental technical advances are required along the way, continuing to train AI models using ever greater amounts of computational power and data will inevitably lead to AGI.

There is some evidence to back this theory up. Researchers have observed very neat and predictable relationships between how much computational power, also known as compute, is used to train an AI model and how well it performs a given task. In the case of large language models (LLM)the AI systems that power chatbots like ChatGPTscaling laws predict how well a model can predict a missing word in a sentence. OpenAI CEO Sam Altman recently told TIME that he realized in 2019 that AGI might be coming much sooner than most people think, after OpenAI researchers discovered the scaling laws.

Read More: 2023 CEO of the Year: Sam Altman

Even before the scaling laws were observed, researchers have long understood that training an AI system using more compute makes it more capable. The amount of compute being used to train AI models has increased relatively predictably for the last 70 years as costs have fallen.

Early predictions based on the expected growth in compute were used by experts to anticipate when AI might match (and then possibly surpass) humans. In 1997, computer scientist Hans Moravec argued that cheaply available hardware will match the human brain in terms of computing power in the 2020s. An Nvidia A100 semiconductor chip, widely used for AI training, costs around $10,000 and can perform roughly 20 trillion FLOPS, and chips developed later this decade will have higher performance still. However, estimates for the amount of compute used by the human brain vary widely from around one trillion floating point operations per second (FLOPS) to more than one quintillion FLOPS, making it hard to evaluate Moravecs prediction. Additionally, training modern AI systems requires a great deal more compute than running them, a fact that Moravecs prediction did not account for.

More recently, researchers at nonprofit Epoch have made a more sophisticated compute-based model. Instead of estimating when AI models will be trained with amounts of compute similar to the human brain, the Epoch approach makes direct use of scaling laws and makes a simplifying assumption: If an AI model trained with a given amount of compute can faithfully reproduce a given portion of textbased on whether the scaling laws predict such a model can repeatedly predict the next word almost flawlesslythen it can do the work of producing that text. For example, an AI system that can perfectly reproduce a book can substitute for authors, and an AI system that can reproduce scientific papers without fault can substitute for scientists.

Some would argue that just because AI systems can produce human-like outputs, that doesnt necessarily mean they will think like a human. After all, Russell Crowe plays Nobel Prize-winning mathematician John Nash in the 2001 film, A Beautiful Mind, but nobody would claim that the better his acting performance, the more impressive his mathematical skills must be. Researchers at Epoch argue that this analogy rests on a flawed understanding of how language models work. As they scale up, LLMs acquire the ability to reason like humans, rather than just superficially emulating human behavior. However, some researchers argue it's unclear whether current AI models are in fact reasoning.

Epochs approach is one way to quantitatively model the scaling hypothesis, says Tamay Besiroglu, Epochs associate director, who notes that researchers at Epoch tend to think AI will progress less rapidly than the model suggests. The model estimates a 10% chance of transformative AIdefined as AI that if deployed widely, would precipitate a change comparable to the industrial revolutionbeing developed by 2025, and a 50% chance of it being developed by 2033. The difference between the models forecast and those of people like Legg is probably largely down to transformative AI being harder to achieve than AGI, says Besiroglu.

Although many in leadership positions at the most prominent AI companies believe that the current path of AI progress will soon produce AGI, theyre outliers. In an effort to more systematically assess what the experts believe about the future of artificial intelligence, AI Impacts, an AI safety project at the nonprofit Machine Intelligence Research Institute, surveyed 2,778 experts in fall 2023, all of whom had published peer-reviewed research in prestigious AI journals and conferences in the last year.

Among other things, the experts were asked when they thought high-level machine intelligence, defined as machines that could accomplish every task better and more cheaply than human workers without help, would be feasible. Although the individual predictions varied greatly, the average of the predictions suggests a 50% chance that this would happen by 2047, and a 10% chance by 2027.

Like many people, the experts seemed to have been surprised by the rapid AI progress of the last year and have updated their forecasts accordinglywhen AI Impacts ran the same survey in 2022, researchers estimated a 50% chance of high-level machine intelligence arriving by 2060, and a 10% chance by 2029.

The researchers were also asked when they thought various individual tasks could be carried out by machines. They estimated a 50% chance that AI could compose a Top 40 hit by 2028 and write a book that would make the New York Times bestseller list by 2029.

Nonetheless, there is plenty of evidence to suggest that experts dont make good forecasters. Between 1984 and 2003, social scientist Philip Tetlock collected 82,361 forecasts from 284 experts, asking them questions such as: Will Soviet leader Mikhail Gorbachev be ousted in a coup? Will Canada survive as a political union? Tetlock found that the experts predictions were often no better than chance, and that the more famous an expert was, the less accurate their predictions tended to be.

Next, Tetlock and his collaborators set out to determine whether anyone could make accurate predictions. In a forecasting competition launched by the U.S. Intelligence Advanced Research Projects Activity in 2010, Tetlocks team, the Good Judgement Project (GJP), dominated the others, producing forecasts that were reportedly 30% more accurate than intelligence analysts who had access to classified information. As part of the competition, the GJP identified superforecastersindividuals who consistently made above-average accuracy forecasts. However, although superforecasters have been shown to be reasonably accurate for predictions with a time horizon of two years or less, it's unclear whether theyre also similarly accurate for longer-term questions such as when AGI might be developed, says Ezra Karger, an economist at the Federal Reserve Bank of Chicago and research director at Tetlocks Forecasting Research Institute.

When do the superforecasters think AGI will arrive? As part of a forecasting tournament run between June and October 2022 by the Forecasting Research Institute, 31 superforecasters were asked when they thought Nick Bostromthe controversial philosopher and author of the seminal AI existential risk treatise Superintelligencewould affirm the existence of AGI. The median superforecaster thought there was a 1% chance that this would happen by 2030, a 21% chance by 2050, and a 75% chance by 2100.

All three approaches to predicting when AGI might be developedEpochs model of the scaling hypothesis, and the expert and superforecaster surveyshave one thing in common: theres a lot of uncertainty. In particular, the experts are spread widely, with 10% thinking it's as likely as not that AGI is developed by 2030, and 18% thinking AGI wont be reached until after 2100.

Still, on average, the different approaches give different answers. Epochs model estimates a 50% chance that transformative AI arrives by 2033, the median expert estimates a 50% probability of AGI before 2048, and the superforecasters are much further out at 2070.

There are many points of disagreement that feed into debates over when AGI might be developed, says Katja Grace, who organized the expert survey as lead researcher at AI Impacts. First, will the current methods for building AI systems, bolstered by more compute and fed more data, with a few algorithmic tweaks, be sufficient? The answer to this question in part depends on how impressive you think recently developed AI systems are. Is GPT-4, in the words of researchers at Microsoft, the sparks of AGI? Or is this, in the words of philosopher Hubert Dreyfus, like claiming that the first monkey that climbed a tree was making progress towards landing on the moon?

Second, even if current methods are enough to achieve the goal of developing AGI, it's unclear how far away the finish line is, says Grace. Its also possible that something could obstruct progress on the way, for example a shortfall of training data.

Finally, looming in the background of these more technical debates are peoples more fundamental beliefs about how much and how quickly the world is likely to change, Grace says. Those working in AI are often steeped in technology and open to the idea that their creations could alter the world dramatically, whereas most people dismiss this as unrealistic.

The stakes of resolving this disagreement are high. In addition to asking experts how quickly they thought AI would reach certain milestones, AI Impacts asked them about the technologys societal implications. Of the 1,345 respondents who answered questions about AIs impact on society, 89% said they are substantially or extremely concerned about AI-generated deepfakes and 73% were similarly concerned that AI could empower dangerous groups, for example by enabling them to engineer viruses. The median respondent thought it was 5% likely that AGI leads to extremely bad, outcomes, such as human extinction.

Given these concerns, and the fact that 10% of the experts surveyed believe that AI might be able to do any task a human can by 2030, Grace argues that policymakers and companies should prepare now.

Preparations could include investment in safety research, mandatory safety testing, and coordination between companies and countries developing powerful AI systems, says Grace. Many of these measures were also recommended in a paper published by AI experts last year.

If governments act now, with determination, there is a chance that we will learn how to make AI systems safe before we learn how to make them so powerful that they become uncontrollable, Stuart Russell, professor of computer science at the University of California, Berkeley, and one of the papers authors, told TIME in October.

Link:
When Might AI Outsmart Us? It Depends Who You Ask - TIME

Policymakers around the world are grappling with the new opportunities and dangers that artificial intelligence presents. Of all the effects that AI can have on the world, among the most consequential would be integrating it into the command and control for nuclear weapons. Improperly used, AI in nuclear operations could have world-ending effects. If properly implemented, it could reduce nuclear risk by improving early warning and detection and enhancing the resilience of second-strike capabilities, both of which would strengthen deterrence. To take full advantage of these benefits, systems must take into account the strengths and limitations of humans and machines. Successful human-machine joint cognitive systems will harness the precision and speed of automation with the flexibility of human judgment and do so in a way that avoids automation bias and surrendering human judgment to machines. Because of the early state of AI implementation, the United States has the potential to make the world safer by more clearly outlining its policies, pushing for broad international agreement, and acting as a normative trendsetter.

The United States has been extremely transparent and forward-leaning in establishing and communicating its policies on military AI and autonomous systems, publishing its policy on autonomy in weapons in 2012, adopting ethical principles for military AI in 2020, and updating its policy on autonomy in weapons in 2023. The department stated formally and unequivocally in the 2022 Nuclear Posture Review that it will always maintain a human in the loop for nuclear weapons employment. In November 2023, over 40 nations joined the United States in endorsing a political declaration on responsible military use of AI. Endorsing states included not just U.S. allies but also nations in Africa, Southeast Asia, and Latin America.

[wotr_memer_button]

Building on this success, the United States should push for international agreements with other nuclear powers to mitigate the risks of integrating AI into nuclear systems or placing nuclear weapons onboard uncrewed vehicles. The United Kingdom and France released a joint statement with the United States in 2022 agreeing on the need to maintain human control of nuclear launches. Ideally, this could represent the beginning of a commitment by the permanent members of the United Nations Security Council if Russia and China could be convinced to join this principle. Even if they are not willing to agree, the United States should further mature its own policies to address critical gaps and work with other nuclear-armed states to strengthen their commitments as an interim measure and as a way to build international consensus on the issue.

The Dangers of Automation

As militaries increasingly adopt AI and automation, there is an urgent need to clarify how these technologies should be used in nuclear operations. Absent formal agreements, states risk an incremental trend of creeping automation that could undermine nuclear stability. While policymakers are understandably reluctant to adopt restrictions on emerging technologies lest they give up a valuable future capability, U.S. officials should not be complacent in assuming other states will approach AI and automation in nuclear operations responsibly. Examples such as Russias Perimeter dead hand system and Poseidon autonomous nuclear-armed underwater drone demonstrate that other nations might see these risks differently than the United States and might be willing to take risks that U.S. policymakers would find unacceptable.

Existing systems, such as Russias Perimeter, highlight the risks of states integrating automation into nuclear systems. Perimeter is reportedly a system created by the Soviet Union in the 1980s to act as a failsafe in case Soviet leadership was destroyed in a decapitation strike. Perimeter reportedly has a network of sensors to determine if a nuclear attack has occurred. If these sensors are triggered while Perimeter is activated, the system would wait a predetermined period of time for a signal from senior military commanders. If there is no signal from headquarters, presumably because Soviet/Russian leadership had been wiped out, then Perimeter would bypass the normal chain of command and pass nuclear launch authority to a relatively junior officer on duty. Senior Russian officials have stated the system is still functioning, noting in 2011 that the system was combat ready and in 2018 that it had been improved.

The system was designed to reduce the burden on Soviet leaders of hastily making a nuclear decision under time pressure and with incomplete information. In theory, Soviet/Russian leaders could take more time to deliberate knowing that there is a failsafe guaranteeing retaliation if the United States succeeded in a decapitation strike. The cost, however, is a system that risks easing pathways to nuclear annihilation in the event of an accident.

Allowing autonomous systems to participate in nuclear launch decisions risks degrading stability and increasing the dangers of nuclear accidents. The Stanislav Petrov incident is an illustrative example of the dangers of automation in nuclear decision-making. In 1983, a Soviet early warning system indicated that the United States had launched several intercontinental ballistic missiles. Lieutenant Colonel Stanislav Petrov, the duty officer at the time, suspected that the system was malfunctioning because the number of missiles launched was suspiciously low and the missiles were not picked up by early warning radars. Petrov reported it (correctly) as a malfunction instead of an attack. AI and autonomous systems often lack the contextual understanding that humans have and that Petrov used to recognize that the reported missile launch was a false alarm. Without human judgment at critical stages of nuclear operations, automated systems could make mistakes or elevate false alarms, heightening nuclear risk.

Moreover, merely having humans in the loop will not be enough to ensure effective human decision-making. Human operators frequently fall victim to automation bias, a condition in which humans overtrust automation and surrender their judgment to machines. Accidents with self-driving cars demonstrate the dangers of humans overtrusting automation, and military personnel are not immune to this phenomenon. To ensure humans remain cognitively engaged in their decision-making, militaries will need to take into account not only the automation itself but also human psychology and human-machine interfaces.

More broadly, when designing human-machine systems, it is essential to consciously determine the appropriate roles for humans and machines. Machines are often better at precision and speed, while humans are often better at understanding the broader context and applying judgment. Too often, human operators are left to fill in the gaps for what automation cant do, acting as backups or failsafes for the edge cases that autonomous systems cant handle. But this model often fails to take into account the realities of human psychology. Even if human operators dont fall victim to automation bias, to assume that a person can sit passively watching a machine perform a task for hours on end, whether a self-driving car or a military weapon system, and then suddenly and correctly identify a problem when the automation is not performing and leap into action to take control is not realistic. Human psychology doesnt work that way. And tragic accidents with complex highly automated systems, such as the Air France 447 crash in 2009 and the 737 MAX crashes in 2018 and 2019, demonstrate the importance of taking into account the dynamic interplay between automation and human operators.

The U.S. military has also suffered tragic accidents with automated systems, even when humans are in the loop. In 2003, U.S. Army Patriot air and missile defense systems shot down two friendly aircraft during the opening phases of the Iraq war. Humans were in the loop for both incidents. Yet a complex mix of human and technical failures meant that human operators did not fully understand the complex, highly automated systems they were in charge of and were not effectively in control.

The military will need to establish guidance to inform system design, operator training, doctrine, and operational procedures to ensure that humans in the loop arent merely unthinking cogs in a machine but actually exercise human judgment. Issuing this concrete guidance for weapons developers and operators is most critical in the nuclear domain, where the consequences of an accident could be grave.

Clarifying Department of Defense Guidance

Recent policies and statements on the role of autonomy and AI in nuclear operations are an important first step in establishing this much-needed guidance, but additional clarification is needed. The 2022 Nuclear Posture Review states: In all cases, the United States will maintain a human in the loop for all actions critical to informing and executing decisions by the President to initiate and terminate nuclear weapon employment. The United Kingdom adopted a similar policy in 2022, stating in their Defence Artificial Intelligence Strategy: We will ensure that regardless of any use of AI in our strategic systems human political control of our nuclear weapons is maintained at all times.

As the first official policies on AI in nuclear command and control, these are landmark statements. Senior U.S. military officers had previously emphasized the importance of human control over nuclear weapons, including statements by Lt. Gen. Jack Shanahan, then-director of the Joint Artificial Intelligence Center in 2019. Official policy statements are more significant, however, in signaling to audiences both internal and external to the military the importance of keeping humans firmly in charge of all nuclear use decisions. These high-level statements nevertheless leave many open questions about implementation.

The next step for Department of Defense is to translate what the high-level principle of human in the loop means for nuclear systems, doctrine, and training. Key questions include: Which actions are critical to informing and executing decisions by the president? Do those only consist of actions immediately surrounding the president, or do they also include actions further down the chain of command before and after a presidential decision? For example, would it be acceptable for a human to deliver an algorithm-based recommendation to the president to carry out a nuclear attack? Or does a human need to be involved in understanding the data and rendering their own human judgment?

The U.S. military already uses AI to process information, such as satellite images and drone video feeds. Presumably, AI would also be used to support intelligence analysis that could support decisions about nuclear use. Under what circumstances is AI appropriate and beneficial to nuclear stability? Are some applications and ways of using AI more valuable than others?

When AI is used, what safeguards should be put in place to guard against mistakes, malfunctions, or spoofing of AI systems? For example, the United States currently employs a dual phenomenology mechanism to ensure that a potential missile attack is confirmed by two independent sensing methods, such as satellites and ground-based radars. Should the United States adopt a dual algorithm approach to any use of AI in nuclear operations, ensuring that there are two independent AI systems trained on different data sets with different algorithms as a safeguard against spoofing attacks or unreliable AI systems?

When AI systems are used to process information, how should that information be presented to human operators? For example, if the military used an algorithm trained to detect signs of a missile being fueled, that information could be interpreted differently by humans if the AI system reported fueling versus preparing to launch. Fueling is a more precise and accurate description of what the AI system is actually detecting and might lead a human analyst to seek more information, whereas preparing to launch is a conclusion that might or might not be appropriate depending on the broader context.

When algorithmic recommendation systems are used, how much of the underlying data should humans have to directly review? Is it sufficient for human operators to only see the algorithms conclusion, or should they also have access to the raw data that supports the algorithms recommendation?

Finally, what degree of engagement is expected from a human in the loop? Is the human merely there as a failsafe in case the AI malfunctions? Or must the human be engaged in the process of analyzing information, generating courses of actions, and making recommendations? Are some of these steps more important than others for human involvement?

These are critical questions that the United States will need to address as it seeks to harness the benefits of AI in nuclear operations while meeting the human in the loop policy. The sooner the Department of Defense can clarify answers to these questions, the more that it can accelerate AI adoption in ways that are trustworthy and meet the necessary reliability standards for nuclear operations. Nor does clarifying these questions overly constrain how the United States approaches AI. Guidance can always be changed over time as the technology evolves. But a lack of clear guidance risks forgoing valuable opportunities to use AI or, even worse, adopting AI in ways that might undermine nuclear surety and deterrence.

Dead Hand Systems

In clarifying its human-in-the-loop policy, the United States should make a firm commitment to reject dead hand nuclear launch systems or a system with a standing order to launch that incorporates algorithmic components. Dead hand systems akin to Russias Perimeter would appear to be prohibited by current Department of Defense policy. However, the United States should explicitly state that it will not build such systems given their risk.

Despite their danger, some U.S. analysts have suggested that the United States should adopt a dead hand system to respond to emerging technologies such as AI, hypersonics, and advanced cruise missiles. There are safer methods for responding to these threats, however. Rather than gambling humanitys future on an algorithm, the United States should strengthen its second-strike deterrent in response to new threats.

Some members of the U.S. Congress have even expressed a desire for writing this requirement into law. In April 2023, a bipartisan group of representatives introduced the Block Nuclear Launch by Autonomous Artificial Intelligence Act, which would prohibit funding for any system that launches nuclear weapons without meaningful human control. There is precedent for a legal requirement to maintain a human in the loop for strategic systems. In the 1980s, during development of the Strategic Defense Initiative (also known as Star Wars), Congress passed a law requiring affirmative human decision at an appropriate level of authority for strategic missile defense systems. This legislation could serve as a blueprint for a similar legislative requirement for nuclear use. One benefit of a legal requirement is that it ensures that such an important policy could not be overturned by a future administration or Pentagon leadership that is more risk-accepting without Congressional authorization.

Nuclear Weapons and Uncrewed Vehicles

The United States should similarly clarify its policy for nuclear weapons on uncrewed vehicles. The United States is producing a new nuclear-capable strategic bomber, the B-21, that will be able to perform uncrewed missions in the future, and is developing large undersea uncrewed vehicles that could carry weapons payloads. U.S. military officers have stated a strong reticence for placing nuclear weapons aboard uncrewed platforms. In 2016, then-Commander of Air Force Global Strike Command Gen. Robin Rand noted that the B-21 would always be crewed when carrying nuclear weapons: If you had to pin me down, I like the man in the loop; the pilot, the woman in the loop, very much, particularly as we do the dual-capable mission with nuclear weapons. General Rands sentiment may be shared among senior military officers, but it is not official policy. The United States should adopt an official policy that nuclear weapons will not be placed aboard recoverable uncrewed platforms. Establishing this policy could help provide guidance to weapons developers and the services about the appropriate role for uncrewed platforms in nuclear operations as the Department of Defense fields larger uncrewed and optionally crewed platforms.

Nuclear weapons have long been placed on uncrewed delivery vehicles, such as ballistic and cruise missiles, but placing nuclear weapons on a recoverable uncrewed platform such as a bomber is fundamentally different. A human decision to launch a nuclear missile is a decision to carry out a nuclear strike. Humans could send a recoverable, two-way uncrewed platform, such as a drone bomber or undersea autonomous vehicle, out on patrol. In that case, the human decision to launch the nuclear-armed drone would not yet be a decision to carry out a nuclear strike. Instead, the drone could be sent on patrol as an escalation signal or to preposition in case of a later decision to launch a nuclear attack. Doing so would put enormous faith in the drones communications links and on-board automation, both of which may be unreliable.

The U.S. military has lost control of drones before. In 2017, a small tactical Army drone flew over 600 miles from southern Arizona to Denver after Army operators lost communications. In 2011, a highly sensitive U.S. RQ-170 stealth drone ended up in Iranian hands after U.S. operators lost contact with it over Afghanistan. Losing control of a nuclear-armed drone could cause nuclear weapons to fall into the wrong hands or, in the worst case, escalate a nuclear crisis. The only way to maintain nuclear surety is direct, physical human control over nuclear weapons up until the point of a decision to carry out a nuclear strike.

While the U.S. military would likely be extremely reluctant to place nuclear weapons onboard a drone aircraft or undersea vehicle, Russia is already developing such a system. The Poseidon, or Status-6, undersea autonomous uncrewed vehicle is reportedly intended as a second- or third-strike weapon to deliver a nuclear attack against the United States. How Russia intends to use the weapon is unclear and could evolve over time but an uncrewed platform like the Poseidon in principle could be sent on patrol, risking dangerous accidents. Other nuclear powers could see value in nuclear-armed drone aircraft or undersea vehicles as these technologies mature.

The United States should build on its current momentum in shaping global norms on military AI use and work with other nations to clarify the dangers of nuclear-armed drones. As a first step, the U.S. Defense Department should clearly state as a matter of official policy that it will not place nuclear weapons on two-way, recoverable uncrewed platforms, such as bombers or undersea vehicles. The United States has at times foresworn dangerous weapons in other areas, such as debris-causing antisatellite weapons, and publicly articulated their dangers. Similarly explaining the dangers of nuclear-armed drones could help shape the behavior of other nuclear powers, potentially forestalling their adoption.

Conclusion

It is imperative that nuclear powers approach the integration of AI and autonomy in their nuclear operations thoughtfully and deliberately. Some applications, such as using AI to help reduce the risk of a surprise attack, could improve stability. Other applications, such as dead hand systems, could be dangerous and destabilizing. Russias Perimeter and Poseidon systems demonstrate that other nations might be willing to take risks with automation and autonomy that U.S. leaders would see as irresponsible. It is essential for the United States to build on its current momentum to clarify its own policies and work with other nuclear-armed states to seek international agreement on responsible guardrails for AI in nuclear operations. Rumors of a U.S.-Chinese agreement on AI in nuclear command and control at the meeting between President Joseph Biden and General Secretary Xi Jinping offer a tantalizing hint of the possibilities for nuclear powers to come together to guard against the risks of AI integrated into humanitys most dangerous weapons. The United States should seize this moment and not let this opportunity pass to build a safer, more stable future.

Michael Depp is a research associate with the AI safety and stability project at the Center for a New American Security (CNAS).

Paul Scharre is the executive vice president and director of studies at CNAS and the author of Four Battlegrounds: Power in the Age of Artificial Intelligence.

Image: U.S. Air Force photo by Senior Airman Jason Wiese

Read the rest here:
Artificial Intelligence and Nuclear Stability - War On The Rocks