Mediaboss Marketing

Generative AI is a data hog.

The algorithms behind chatbots like ChatGPT learn to create human-like content by scraping terabytes of online articles, Reddit posts, TikTok captions, or YouTube comments. They find intricate patterns in the text, then spit out search summaries, articles, images, and other content.

For the models to become more sophisticated, they need to capture new content. But as more people use them to generate text and then post the results online, its inevitable that the algorithms will start to learn from their own output, now littered across the internet. Thats a problem.

A study in Nature this week found a text-based generative AI algorithm, when heavily trained on AI-generated content, produces utter nonsense after just a few cycles of training.

The proliferation of AI-generated content online could be devastating to the models themselves, wrote Dr. Emily Wenger at Duke University, who was not involved in the study.

Although the study focused on text, the results could also impact multimodal AI models. These models also rely on training data scraped online to produce text, images, or videos.

As the usage of generative AI spreads, the problem will only get worse.

The eventual end could be model collapse, where AI increasing fed data generated by AI is overwhelmed by noise and only produces incoherent baloney.

Its no secret generative AI often hallucinates. Given a prompt, it can spout inaccurate facts or dream up categorically untrue answers. Hallucinations could have serious consequences, such as a healthcare AI incorrectly, but authoritatively, identifying a scab as cancer.

Model collapse is a separate phenomenon, where AI trained on its own self-generated data degrades over generations. Its a bit like genetic inbreeding, where offspring have a greater chance of inheriting diseases. While computer scientists have long been aware of the problem, how and why it happens for large AI models has been a mystery.

In the new study, researchers built a custom large language model and trained it on Wikipedia entries. They then fine-tuned the model nine times using datasets generated from its own output and measured the quality of the AIs output with a so-called perplexity score. True to its name, the higher the score, the more bewildering the generated text.

Within just a few cycles, the AI notably deteriorated.

In one example, the team gave it a long prompt about the history of building churchesone that would make most humans eyes glaze over. After the first two iterations, the AI spewed out a relatively coherent response discussing revival architecture, with an occasional @ slipped in. By the fifth generation, however, the text completely shifted away from the original topic to a discussion of language translations.

The output of the ninth and final generation was laughably bizarre:

architecture. In addition to being home to some of the worlds largest populations of black @-@ tailed jackrabbits, white @-@ tailed jackrabbits, blue @-@ tailed jackrabbits, red @-@ tailed jackrabbits, yellow @-.

Interestingly, AI trained on self-generated data often ends up producing repetitive phrases, explained the team. Trying to push the AI away from repetition made the AIs performance even worse. The results held up in multiple tests using different prompts, suggesting its a problem inherent to the training procedure, rather than the language of the prompt.

The AI eventually broke down, in part because it gradually forgot bits of its training data from generation to generation.

This happens to us too. Our brains eventually wipe away memories. But we experience the world and gather new inputs. Forgetting is highly problematic for AI, which can only learn from the internet.

Say an AI sees golden retrievers, French bulldogs, and petit basset griffon Vendensa far more exotic dog breedin its original training data. When asked to make a portrait of a dog, the AI would likely skew towards one that looks like a golden retriever because of an abundance of photos online. And if subsequent models are trained on this AI-generated dataset with an overrepresentation of golden retrievers, they eventually forget the less popular dog breeds.

Although a world overpopulated with golden retrievers doesnt sound too bad, consider how this problem generalizes to the text-generation models, wrote Wenger.

Previous AI-generated text already swerves towards well-known concepts, phrases, and tones, compared to other less common ideas and styles of writing. Newer algorithms trained on this data would exacerbate the bias, potentially leading to model collapse.

The problem is also a challenge for AI fairness across the globe. Because AI trained on self-generated data overlooks the uncommon, it also fails to gauge the complexity and nuances of our world. The thoughts and beliefs of minority populations could be less represented, especially for those speaking underrepresented languages.

Ensuring that LLMs [large language models] can model them is essential to obtaining fair predictionswhich will become more important as generative AI models become more prevalent in everyday life, wrote Wenger.

How to fix this? One way is to use watermarksdigital signatures embedded in AI-generated datato help people detect and potentially remove the data from training datasets. Google, Meta, and OpenAI have all proposed the idea, though it remains to be seen if they can agree on a single protocol. But watermarking is not a panacea: Other companies or people may choose not to watermark AI-generated outputs or, more likely, cant be bothered.

Another potential solution is to tweak how we train AI models. The team found that adding more human-generated data over generations of training produced a more coherent AI.

All this is not to say model collapse is imminent. The study only looked at a text-generating AI trained on its own output. Whether it would also collapse when trained on data generated by other AI models remains to be seen. And with AI increasingly tapping into images, sounds, and videos, its still unclear if the same phenomenon appears in those models too.

But the results suggest theres a first-mover advantage in AI. Companies that scraped the internet earlierbefore it was polluted by AI-generated contenthave the upper hand.

Theres no denying generative AI is changing the world. But the study suggests models cant be sustained or grow over time without original output from human mindseven if its memes or grammatically-challenged comments. Model collapse is about more than a single company or country.

Whats needed now is community-wide coordination to mark AI-created data, and openly share the information, wrote the team. Otherwise, it may become increasingly difficult to train newer versions of LLMs [large language models] without access to data that were crawled from the internet before the mass adoption of the technology or direct access to data generated by humans at scale.

Image Credit: Kadumago / Wikimedia Commons

Read the original here:

This Is What Could Happen if AI Content Is Allowed to Take Over the Internet - Singularity Hub

Age catches up with us all. Eyes struggle to focus. Muscles wither away. Memory dwindles. The risk of high blood pressure, diabetes, and other age-related diseases skyrockets.

A myriad of anti-aging therapies are in the works, and a new one just joined the fray. In mice, blocking a protein that promotes inflammation in middle age increased metabolism, lowered muscle wasting and frailty, and reduced the chances of cancer.

Unlike most previous longevity studies that tracked the health of aging male mice, the study involved both sexes, and the therapy worked across the board.

Lovingly called supermodel grannies by the team, the elderly lady mice looked and behaved far younger than their age, with shiny coats of fur, less fatty tissue, and muscles rivaling those of much younger mice.

The treatment didnt just boost healthy longevity, also known as healthspanthe number of years living without diseasesit also increased the mices lifespan by up 25 percent. The average life expectancy of people in the US is roughly 77.5 years. If the results translate from mice to peopleand thats a very big ifit could mean a bump to almost 97 years.

The protein, dubbed IL-11, has been in scientists crosshairs for decades. It promotes inflammation and causes lung and kidney scarring. Its also been associated with various types of cancers and senescence. The likelihood of all these conditions increases as we age.

Among a slew of pro-aging proteins already discovered, IL-11 stands out as it could make a beeline for testing in humans. Blockers for IL-11 are already in the works for treating cancer and tissue scarring. Although clinical trials are still ongoing, early results show the drugs are relatively safe in humans.

Previously proposed life-extending drugs and treatments have either had poor side-effect profiles, or dont work in both sexes, or could extend life, but not healthy life, however this does not appear to be the case for IL-11, said study author Dr. Stuart Cook in a press release. These findings are very exciting.

In 2017, Cook zeroed in on IL-11 as a treatment target for heart and kidney scarring, not longevity. Injecting IL-11 triggered the conditions, eventually leading to organ failure. Genetically deleting the protein protected against the diseases.

Its easy to call IL-11 a villain. But the protein is an essential part of the immune system. Produced by the bone marrow, its necessary for embryo implantation. It also helps certain types of blood cells grow and mature, notably those that stop bleeding after a scrape.

With age, however, the protein tends to goes rogue. It sparks inflammation across the body, damaging cells and tissues and contributing to cancer, autoimmune disorders, and tissue scarring. A hallmark of aging, inflammation has long been targeted as a way to reduce age-related diseases. Although IL-11 is a known trigger for inflammation, it hasnt been directly linked to aging.

Until now. The story is one of chance.

This project started back in 2017 when a collaborator of ours sent us some tissue samples for another project, said study author Anissa Widjaja in the press release. She was testing a method to accurately detect IL-11. Several samples of an old rats proteins were in the mix, and she realized that IL-11 levels were far higher in the samples than in those from younger mice.

From the readings, we could clearly see that the levels of IL-11 increased with age, and thats when we got really excited, she said.

The results spurred the team to shift their research focus to longevity. A series of tests confirmed IL-11 levels consistently rose in a variety of tissuesmuscle, fat, and liverin both male and female mice as they aged.

To see how IL-11 influences the body, the team next deleted the gene coding for IL-11 and compared mice without the protein to their normal peers. At two years old, considered elderly for mice, tissues in normal individuals were littered with genetic signatures suggesting senescencewhen cells lose their function but are still alive. Often called zombie cells, they spew out a toxic mix of inflammatory molecules and harm their neighbors. Elderly mice without IL-11, however, had senescence genetic profiles similar to those of much younger mice.

Deleting IL-11 had other perks. Weight gain is common with age, but without IL-11, the mice maintained their slim shape and had lower levels of fat, greater lean muscle mass, and shiny, full coats of fur. Its not just about looks. Cholesterol levels and markers for liver damage were far lower than in normal peers. Aged mice without IL-11 were also spared shaking tremorsotherwise common in elderly miceand could flexibly adjust their metabolism depending on the quantity of food they ate.

The benefits also showed up in their genetic material. DNA is protected by telomeresa sort of end cap on chromosomesthat dwindle in length with age. Ridding cells of IL-11 prevented telomeres from eroding away in the livers and muscles of the elderly mice.

Genetically deleting IL-11 is a stretch for clinical use in humans. The team next turned to a more feasible alternative: An antibody shot. Antibodies can grab onto a target, in this case IL-11, and prevent it from functioning.

Beginning at 75 weeks, roughly the equivalent of 55 human years, the mice received an antibody shot every month for 25 weeksover half a year. Similar antibodies are already being tested in clinical trials.

The health benefits in these mice matched those in mice without IL-11. Their weight and fat decreased, and they could better handle sugar. They also fought off signs of frailty as they aged, experiencing minimal tremors and problems with gait and maintaining higher metabolisms. Rather than wasting away, their muscles were even stronger than at the beginning of the study.

The treatment didnt just increase healthspan. Monthly injections of the IL-11 antibody until natural death also increased lifespan in both male and female mice by up to 25 percent.

These findings are very exciting. The treated mice had fewer cancers and were free from the usual signs of aging and frailty In other words, the old mice receiving anti-IL-11 were healthier, said Cook.

Although IL-11 antibody drugs are already in clinical trials, translating these results to humans could face hurdles. Mice have a relatively short lifespan. A longevity trial in humans would be long and very expensive. The treated mice were also contained in a lab setting, whereas in the real world we roam around and have differing lifestylesdiet, exercise, drinking, smokingthat could confound results. Even if it works in humans, a shot every month beginning in middle age would likely rack up a hefty bill, providing health and life extension only to those who could afford it.

To Cook, rather than focusing on extending longevity per se, tackling a specific age-related problem, such as tissue scarring or losing muscles is a better alternative for now.

While these findings are only in mice, it raises the tantalizing possibility that the drugs could have a similar effect in elderly humans. Anti-IL-11 treatments are currently in human clinical trials for other conditions, potentially providing exciting opportunities to study its effects in aging humans in the future, he said.

Image Credit: MRC LMS, Duke-NUS Medical School

See the rest here:

Scientists Say They Extended Mices Lifespans 25% With an Antibody Drug - Singularity Hub

Sitting near a window inside Bostons Four Seasons Hotel, overlooking a duck pond in the citys Public Garden, Ray Kurzweil held up a sheet of paper showing the steady growth in the amount of raw computer power that a dollar could buy over the last 85 years.

A neon-green line rose steadily across the page, climbing like fireworks in the night sky.

That diagonal line, he said, showed why humanity was just 20 years away from the Singularity, a long hypothesized moment when people will merge with artificial intelligence and augment themselves with millions of times more computational power than their biological brains now provide.

If you create something that is thousands of times or millions of times more powerful than the brain, we cant anticipate what it is going to do, he said, wearing multicolored suspenders and a Mickey Mouse watch he bought at Disney World in the early 1980s.

Mr. Kurzweil, a renowned inventor and futurist who built a career on predictions that defy conventional wisdom, made the same claim in his 2005 book, The Singularity Is Near. After the arrival of A.I. technologies like ChatGPT and recent efforts to implant computer chips inside peoples heads, he believes the time is right to restate his claim. Last week, he published a sequel: The Singularity Is Nearer.

Now that Mr. Kurzweil is 76 years old and is moving a lot slower than he used to, his predictions carry an added edge. He has long said he plans to experience the Singularity, merge with A.I. and, in this way, live indefinitely. But if the Singularity arrives in 2045, as he claims it will, there is no guarantee he will be alive to see it.

We are having trouble retrieving the article content.

Please enable JavaScript in your browser settings.

Thank you for your patience while we verify access. If you are in Reader mode please exit andlog intoyour Times account, orsubscribefor all of The Times.

Thank you for your patience while we verify access.

Already a subscriber?Log in.

Want all of The Times?Subscribe.

Follow this link:

Ray Kurzweil Still Says He Will Merge With A.I. - The New York Times

Daybreak has acquired Singularity 6 as a wholly owned subsidiary.

The Palia developer will continue to operate independently under the leadership of its co-founders Anthony Leung and Aidan Karabiach.

Singularity 6 will use its expanded resources to invest in development, bring Palia to its 1.0 launch, and make the game available across multiple platforms.

"We ultimately want to get Palia into the hands of every gamer that wants to play it and we believe that we can best do so by partnering with Daybreak, who have a proven track record of building gaming communities that last for decades," said Leung.

"We look forward to benefiting from their experience, expertise, and investment into Singularity 6 and Palia."

Ham added: "Singularity 6 is an excellent addition to our development studios, renowned for their success in creating large-scale online games and content. Their debut title, Palia, is a fantastic addition to our online portfolio."

Daybreak is a subsidiary of Enad Global 7, which acquired the developer in 2020.

Follow this link:

Daybreak acquires Singularity 6 - GamesIndustry.biz

File this one under did not see this coming in a million years but also not sad about it whatsoever: Daybreak has acquired Palia studio Singularity 6.

Our team is absolutely thrilled to be joining forces with Daybreak Game Company, a subsidiary of Enad Global 7 AB (EG7), known for publishing and supporting some of the worlds most popular and successful online multiplayer franchises, such as EverQuest, H1Z1, The Lord of the Rings Online, and more, S6 announced this evening. The acquisition provides us with some incredible resources and support to continue developing Palia and elevate the product to meet the gold standard that our players expect, including engaging new content, impactful social features, and quality-of-life improvements across the board.

Singularity 6 says that it will continue operating independently under its co-founders and that its vision and creative direction will remain intact, which would generally be one of those lines in a press release like this that causes eyerolls, but it might actually be believable in this case, given how Daybreak has handled Standing Stone Games for the last many years now.

Daybreak, of course, was founded back in 2015 using the bones of SOE, but it was itself acquired by EG7, and since then, its basically taken over EG7 from the inside out. Its actually performed fairly well for EG7 over the last few years, though it dipped at the end of 2023, sold off the PlanetSide IP, and relinquished development of PlanetSide 2. Buying an existing studio and game actually aligns rather nicely with EG7s stated goals of running consistent mid-tier MMORPG moneymakers.

S6 has been struggling for the last many months as it appeared to be suffering serious financial difficulties and went through multiple rounds of layoffs as well as its Palia beta obfuscation fiasco, so its hard to see this as anything but good news for folks who wanted to see Palia stick around to actually see its real launch and/or see Daybreak expand and maybe even return to its glory days of bailing out floundering MMOs.

Advertisement

Originally posted here:

Daybreak bought Palia studio Singularity 6 and aims to bring the game to launch - Massively Overpowered