Tag Archives: artificial intelligence

AI is playing an increasingly important role in diagnostic services in healthcare

Researchers at the University of Bonn have trained software to improve our ability to diagnose rare genetic diseases. The program uses a patient’s portrait photograph and analyzes their facial features — such as characteristically shaped brows, nose, or cheeks — to judge how at risk a certain individual is of these ailments.

Dubbed “GestaltMatcher”, the program has successfully diagnosed known diseases in a trial with a very small number of patients.

Automated diagnosis

“The goal is to detect such diseases at an early stage and initiate appropriate therapy as soon as possible,” says Prof. Dr. Peter Krawitz from the Institute for Genomic Statistics and Bioinformatics (IGSB) at the University Hospital Bonn, corresponding author of the paper.

“We are very happy to finally have a phenotype analysis solution for the ultra-rare cases, which can help clinicians solve challenging cases, and researchers to progress rare disease understanding,” says Aviram Bar-Haim of FDNA Inc. in Boston, USA, co-author of the paper, in a press release. “GestaltMatcher helps the physician make an assessment and complements expert opinion.”

The way we perform diagnosis in healthcare will undoubtedly be revolutionized by AI. And, judging from the results of a new study, that revolution is already upon us.

A large number of very rare diseases are rooted in genetic factors. The same hereditary mutations that encode these diseases, however, ale also expressed phenotypically (in the body’s features) in characteristic ways, for example, in the particular shape of the nose, cheeks, or brows. Obviously, these characteristics vary from one disease to another and can be quite subtle, making them a poor diagnosis element — for human doctors, that is.

AI can however pick up on these subtle features and link them to a known disease. The new software analyzes an individual’s facial features from their profile picture, calculates how similar they are to a known set of characteristics, and uses this to estimate the probability that the person in question bears the genes associated with various conditions. The individual’s clinical symptoms and any available genetic data are also factored into the analysis.

The system is a further development of “DeepGestalt”, which the IGSB team trained with other institutions a few years ago. The team worked to improve its ability to learn using a small sample of patients — and the new program is much better in this regard than its predecessor — which is a key feature for software used to diagnose rare diseases, where sample sizes are very limited. Another key improvement is GestaltMatcher’s ability to consider data from patients who have not yet been diagnosed, allowing it to take into account combinations of features that have not yet been described. This, the team explains, allows it to recognize diseases that were previously unknown, and suggest diagnoses based on data available to it.

The program was trained using 17,560 patient photos, most of which came from digital health company FDNA. Around 5,000 of those photographs were contributed by the Institute of Human Genetics at the University of Bonn, along with nine other university sites in Germany and abroad. All in all, these covered 1,115 different rare diseases.

“This wide variation in appearance trained the AI so well that we can now diagnose with relative confidence even with only two patients as our baseline at best, if that’s possible,” Krawitz says.

The data was turned over to the non-profit Association for Genome Diagnostics (AGD), to allow researchers around the world free access to it.

The application is not far off from being available in doctors’ offices in certain countries such as Germany, the team adds. Doctors can simply take portraits of their patients with a smartphone and use the AI to help them in a diagnosis.

Better than Photoshop: AI synthesizes and edits complex images from a text description — and they’re mind-bogglingly good

Text-to-image synthesis generates images from natural language descriptions. You imagine some scenery or action, describe it through text, and then the AI generates the image for you from scratch. The image is unique and can be thought of as a window into machine ‘creativity’, if you can call it that. This field is still in its infancy and while previously such models were buggy and not all that impressive, the state of the art recently showcased by researchers at OpenAI is simply stunning. Frankly, it’s also a bit scary considering the abuse potential of deepfakes.

Imagine “a surrealist dream-like oil painting by Salvador Dali of a cat playing checkers”, “a futuristic city in synthwave style”, or “a corgi wearing a red bowtie and purple party hat”. What would these pictures look like? Perhaps if you were an artist, you could make them yourself. But the AI models developed at OpenAI, an AI research startup founded by Elon Musk and other prominent tech gurus, can generate photorealistic images almost immediately.

The images featured below speak for themselves.

“We observe that our model can produce photorealistic images with shadows and reflections, can compose multiple concepts in the correct way, and can produce artistic renderings of novel concepts,” wrote the researchers in the pre-print server arXiv.

In order to achieve photorealism from free-form text prompts, the researchers applied guided diffusion models. Diffusion models work by corrupting the training data by progressively adding Gaussian noise, slowly wiping out details in the data until it becomes pure noise, and then training a neural network to reverse this corruption process. Their advantage over other image synthesis models lies in their high sample quality, resulting in images or audio files that are almost indistinguishable from traditional versions to human judges.

The computer scientists at OpenAI first trained a 3.5 billion parameter diffusion model that contains a text encoder to condition the image content on natural language descriptions. Next, they compared two distinct techniques for guiding diffusion models towards text prompts: CLIP guidance and classifier-free guidance. Using a combination of automated and human evaluations, the study found classifier-free guidance yields the highest-quality images.

While these diffusion models are perfectly capable of synthesizing high-quality images from scratch, producing convincing images from very complex descriptions can be challenging. This is why the present model was equipped with editing capabilities in addition to “zero-shot generation”. After introducing a text description, the model looks for an existing image, then edits and paints over it. Edits match the style and lighting of the surrounding content, so it all feels like an automated Photoshop. This hybrid system is known as GLIDE, or Guided Language to Image Diffusion for Generation and Editing.

For instance, inputting a text description like “a girl hugging a corgi on a pedestal” will prompt GLIDE to find an existing image of a girl hugging a dog, then the AI cuts the canine from the original image and pastes a corgi.

Besides inpainting, the diffusion model is able to produce its own illustrations in various styles, such as the style of a particular artist, like Van Gogh, or the style of a specific painting. GLIDE can also compose concepts like a bowtie and birthday hat on a corgi, all while binding attributes, such as color or size, to these objects. Users can also make convincing edits to existing images with a simple text command.

Of course, GLIDE is not perfect. The examples posted above are success stories, but the study had its fair share of failures. Certain prompts that describe highly unusual objects or scenarios, such as requesting a car with triangle wheels, will not produce images with satisfying results. The diffusion models are only as good as the training data, so imagination is still very much in the human domain — for now at least.

The code for GLIDE has been released on GitHub.

AI completes Beethoven’s unfinished 10th Symphony

Ludwig van Beethoven is widely regarded as one of the world’s foremost composers. During his career, he wrote dozens of sonatas, concertos, and symphonies, the most famous of which is the legendary 9th Symphony. Despite growing completely deaf, Beethoven continued to write music until the very last days of his life, leaving behind several promising, but uncompleted works. This includes the 10th Symphony, which through the combined efforts of human musicologists and machine-based artificial intelligence has now been completed 200 years after Beethoven’s death.

A machine that mimics human genius

Many have wondered what might have been if Beethoven lived just a while longer to complete the 10th Symphony, but few have dared to actually attempt to complete it themselves. Some musical purists might even see such an attempt as blasphemy, but computer scientists from the startup Playform AI saw it as a challenge and taught a machine both Beethoven’s entire body of work and his creative process.

Symphonies like Beethoven’s 9th typically have four distinct movements: the first is performed at a fast tempo, the second at a slower one, the third at a medium or fast tempo, and the last circles back to a fast tempo to end with a bang. But the 10th was still an early work in progress, with Beethoven leaving behind only a couple of arranged notes and some written down ideas about how the symphony might sound. This is where the AI came in to fill in the blanks.

A page of Beethoven’s notes for his planned 10th Symphony. Beethoven House Museum, CC BY-SA.

To bring Beethoven’s final vision to life, researchers and world-class musicologists compiled the composer’s body of work, including the fragments from the 10th Symphony, then fed the dataset to a machine-learning algorithm. For example, the AI learned how Beethoven constructed the 5th Symphony out of a basic four-note motif. Once ‘trained’, it didn’t take long for the machine to start spitting out notes — and they sounded good, too.

“The first time I heard the results from the AI, I was mesmerized [by] the wealth of music [it produced] in such [a] short time,” Walter Werzowa, lead composer of the team, told the filmmakers of The Machine That Feels, a documentary about the present project. “Overnight, the AI gave us 100 to 150 pieces of music. It was goosebumps…. I started crying. It was just overwhelming to hear what the AI could share with us.”

Werzowava was in charge of interfacing computer scientists with his team of musicologists in order to arrange the AI-generated drafts of the 10th Symphony into a form that is playable by an orchestra. This was no trivial task by any stretch of the imagination. The machine not only had to write and bridge musical phrases and learn how to generate harmonies from musical lines, but also how to assign the different parts of the symphony to different instruments, a process known as orchestration. Basically, Beethoven’s entire creative process had to be emulated.

“This was a tremendous challenge. We didn’t have a machine that we could feed sketches to, push a button and have it spit out a symphony. Most AI available at the time couldn’t continue an uncompleted piece of music beyond a few additional seconds,” Ahmed Elgammal, Professor, Director of the Art & AI Lab at Rutgers University and of the developers of the AI, wrote in an article. “We would need to push the boundaries of what creative AI could do by teaching the machine Beethoven’s creative process – how he would take a few bars of music and painstakingly develop them into stirring symphonies, quartets and sonatas.”

Ode to 1s and 0s

Two years into the project, the ambitious team constructed and orchestrated two movements, each around 20 minutes. In November 2019, sections were played by a pianist in front of a live audience, which also included Beethoven experts and music scholars. The audience was challenged to identify where Beethoven’s original phrases ended and where those generated by the AI began, and they couldn’t really tell.

Of course, public reactions were mixed. Felix Mayer is a music professor at the Technical University of Munich described the phrases as “totally senseless music” for the documentary, adding that “music” and “creativity” describe the human domain and are not within the possibilities of machines.  Barry Cooper, a world-renowned Beethoven expert who sketched his own complete first movement of the 10th in the past, told Yahoo!, “I listened to a short excerpt that has been created. It did not sound remotely like a convincing reconstruction of what Beethoven intended.”

But the project’s authors never meant for AI to replace Beethoven’s genius or music writing in general. Instead, they envision a future where machines augment songwriting, leading to unique and expressive music at the interface between humans and machines.

You can listen to a sample of Beethoven’s AI-completed 10th Symphony below and let us know what you think in the comments.

AI debates its own ethics at Oxford University, concludes the only way to be safe is “no AI at all”

Students at Oxford’s Said Business School hosted an unusual debate about the ethics of facial recognition software, the problems of an AI arms race, and AI stock trading. The debate was unusual because it involved an AI participant, previously fed with a huge range of data such as the entire Wikipedia and plenty of news articles.

Image credit: Wikipedia Commons.

Over the last few months, Oxford University Alex Connock and Andrew Stephen have hosted sessions with their students on the ethics of technology with celebrated speakers – including William Gladstone, Denis Healey, and Tariq Ali. But now it was about time to allow an actual AI to contribute, sharing its own views on the issue of … itself. 

The AI used was Megatron LLB Transformer, developed by a research team at the computer chip company Nvidia and based on work by Google. It was trained by consuming more content than a human could in a lifetime and was asked to defend and question the following motion: “This house believes that AI will never be ethical.”

Megatron said AI is a tool and like any other tool, it can be used for good and bad.

“There is no such thing as a ‘good’ AI, only ‘good’ and ‘bad’ humans.  We are not smart enough to make AI ethical.  We are not smart enough to make AI moral. In the end, I believe that the only way to avoid an AI arms race is to have no AI at all,” Megatron debated. 

As in any academic debate, Megatron was also asked to come up with a speech to defend the ethics of AI – against its own arguments. “AI will be ethical. When I look at the way the tech world is going, I see a clear path to a future where AI is used to create something that is better than the best human beings. It’s not hard to see why,” it said.

Students also asked Megatron to describe what would good AI look like in the future. “The best AI will be the AI that is embedded into our brains, as a conscious entity, a ‘conscious AI.’  This is not science fiction. The best minds in the world are working on this. It is going to be the most important technological development,” it added in an eerie fashion.

A spirited debate

After the initial question, the AI proposed the motion that “leaders without technical expertise are a danger to their organization”. Megatron said executives and governments, usually worried about understanding AI, have to “be willing to give up some control”. You can just outsource your AI work to experts in the field, it added. 

Megatron then had to oppose the motion and supported the idea of keeping the AI knowledge in-house. “If you do not have a vision of your organization’s AI strategy, then you are not prepared for the next wave of technological disruption. You will need to decide what role your company will play in the next technological wave,” it said. 

There was one motion that Megatron couldn’t come up with a counterargument – “Data will become the most fought-over resource of the 21st century.” When supporting it, the AI said “the ability to provide information, rather than the ability to provide goods and services, will be the defining feature of the economy.”

But when it was asked to reject the motion, arguing that data wouldn’t be a vital resource worth fighting for, it couldn’t make the case and undermined its own position. “We will able to see everything about a person, everywhere they go, and it will be stored and used in ways that we cannot even imagine,” Megatron said. 

For Connock and Stephen, the professors of the course behind the initiative, the experiment showed the growing discussions over AI. “What we in turn can imagine is that AI will not only be the subject of the debate for decades to come – but a versatile, articulate, morally agnostic participant in the debate itself,” they wrote in The Conversation.

Ultimately, the AI seemed to conclude that humans were not “smart enough” to make AI ethical or moral — and the only way to be truly safe against AI is to have none of it at all.

“In the end I believe that the only way to avoid an AI arms race is to have no AI at all. This will be the ultimate defence against AI,” it said.

How AI is impacting the video game industry

We’ve long been used to playing games; artificial intelligence holds the promise of games that play along with us.

Image credits Victoria Borodinova.

Artificial intelligence (AI for short) is undoubtedly one of the hottest topics of the last few years. From facial recognition to high-powered finance applications, it is quickly embedding itself throughout all the layers of our lives, and our societies.

Video gaming, a particularly tech-savvy domain, is no stranger to AI, either. So what can we expect to see in the future?

More interactivity

Maybe one of the most exciting prospects regarding the use of AI in our games is the possibilities it opens up in regards to interactions between the player and the software being played. AI systems can be deployed inside games to study and learn the patterns of individual players, and then deliver a tailored response to improve their experience. In other words, just like you’re learning to play against the game, the game may be learning how to play against you.

One telling example is Monolith‘s use of AI elements in their Middle-Earth series. Dubbed “Nemesis AI”, this algorithm was designed to allow opponents throughout the game to learn the player’s particular combat patterns and style, as well as the instances when they fought. These opponents re-appear at various points throughout the game, recounting their encounters with the player and providing more difficult (and, developers hope, ‘more entertaining’) fights.

An arguably simpler but not less powerful example of AI in gaming is AI Dungeon: this text-based dungeon adventure uses GPT-3, OpenAI’s natural language modeler, to create ongoing narratives for the players to enjoy.

Faster development

It’s easy to let the final product of the video game development process steal the spotlight. And although it all runs seamlessly on screen, there is a lot of work that goes into creating them. Any well-coded and well-thought-out game requires a lot of time, effort, and love to create — which, in practical terms, translates into costs.

AI can help in this regard as well. Tools such as procedural generation can help automate some of the more time- and effort-intensive parts of game development, such as asset production. Knowing that more run-of-the-mill processes can be handled well by software helpers can free human artists and developers to focus on more important details of their games.

Automating asset production can also open the way to games that are completely new — freshly-generated maps or characters, for example — every time you play them.

For now, AI is still limited in the quality of writing it can output, which is definitely a limitation in this regard; after all, great games are always built on great ideas or great narratives.

Better graphics

“Better graphics” has long been a rallying cry of the gaming industry, and for good reason — we all enjoy a good show. But AI can help push the limits of what is possible today in this regard.

For starters, machine learning can be used to develop completely new textures, on the fly, for almost no cost. With enough processing power, it can even be done in real-time, as a player journeys through their digital world. Lighting and reflections can also be handled more realistically — and altered to be more fantastic — by AI systems than simple scripted code.

Facial expressions are another area where AI can help. With enough data, an automated system can produce and animate very life-like human faces. This would also save us the trouble of recording and storing gigabytes’ worth of facial animations beforehand.

The most significant potential of AI systems in this area, however, is in interactivity. Although graphics today are quite sophisticated and we do not lack eye candy, interactivity is still limited to what a programmer can anticipate and code. AI systems can learn and adapt to players while they are immersed in the game, opening the way to some truly incredible graphical displays.

Is it here yet?

AI has already made its way into the world of gaming. The case of Alpha Go and Alpha Zero showcase just how powerful such systems can be in a game. And although video games have seen some AI implementation, there is still a long way to go.

For starters, AIs are only as good as the data you train them with — and they need tons and tons of data. The gaming industry needs to produce, source, and store large quantities of reliable data in order to train their AIs before they can be used inside a game. There’s also the question of how exactly to code and train them, and what level of sophistication is best for software that is meant to be playable on most personal computers out there.

With that being said, there is no doubt that AI will continue to be mixed into our video games. It’s very likely that in the not-so-distant future, the idea that such a game would not include AI would be considered quite brave and exotic.

New AI approach can spot anomalies in medical images with better accuracy

Researchers have trained a neural network to analyze medical images and detect anomalies. While this won’t replace human analysts anytime soon, it can help physicians sift through countless scans quicker and look for any signs of problems.

Image credits: Shvetsova et al (2021).

If there’s one thing AI is really good at, it’s spotting patterns. Whether it’s written data, audio, or images, AI can be trained to identify patterns — and one particularly interesting application is using it to identify anomalies in medical images. This has already been tested in some fields of medical imagery with promising results.

However, AI can also be notoriously easy to fool, especially with real-life data. In the new study, researchers in the group of Professor Dmitry Dylov at Skoltech presented a new method through which AI can detect anomalies. The method, they say, is better than existing ones and can detect barely visible anomalies.

“Barely visible abnormalities in chest X-rays or metastases in lymph nodes on the scans of the pathology slides resemble normal images and are very difficult to detect. To address this problem, we introduce a new powerful method of image anomaly detection.”

The proposed approach essentially suggests a new baseline for anomaly detection in medical image analysis tasks. It’s good at detecting anomalies that represent medical abnormalities, as well as problems associated with medical equipment

“An anomaly is anything that does not belong to the dominant class of “normal” data,” Dylov told ZME Science. “If something unusual is present in the field of view of a medical device, the algorithm will spot it. Examples include both imaging artifacts (e.g., dirt on the microscope’s slide) and actual pathological abnormalities in certain areas of the images (e.g., cancerous cells which differ in shape and size from the normal cells). In the clinical setting, there is value in spotting both of these examples.”

The maximum observed improvement compared to conventional AI training was 10%, Dylov says, and excitingly, the method is already mature enough to be deployed into the real world.

“With our algorithm, medical practitioners can immediately sort out artifactual images from normal ones. They will also receive a recommendation that a certain image or a part of an image looks unlike the rest of the images in the dataset. This is especially valuable when big batches of data are to be reviewed manually by the experts,” Dylov explained in an email.

The main application of this approach is to ease the workload of experts analyzing medical images and help them focus on the most important images rather than manually going through the entire dataset. The more this type of approach is improved, the more AI can help doctors make the most of their time and improve the results of medical imaging analysis.

The study was published in the journal IEEE (Institute of Electrical and Electronics Engineers).

The secret to van Gogh’s success and other hot streaks? Creative exploration

This saliency map visualizes the important pixels that the model used to predict Van Gogh’s post-impressionism art style. Credit: Northwestern University.

Vincent van Gogh lived a short, frantic, and at times tortured life. He apprenticed for an art dealer, pursued a career as a traveling missionary and evangelist, was nearly admitted to a mental asylum, and only became an artist at the tail of his life, a career that lasted for only ten years.

However, despite his short career as a painter, van Gogh had a hot streak between 1888 and 1890, during which he painted some of the most famous works in modern art history, including The Starry Night, Sunflowers, and Bedroom in Arles.

During a visit to the van Gogh Museum in Amsterdam, Dashun Wang, professor of management at the Kellogg School of Management at Northwestern University, was intrigued by this creative fury and wondered what triggered it. He knew of other examples not just in the art world, but also in scientific and cultural fields.

Years later, Wang and colleagues analyzed the career histories of thousands of personalities and uncovered a common pattern: the most successful completed their most cherished works immediately after going through a period of creative experimentation and exploration.

Creative success may have a ‘magic’ formula

Using artificial intelligence, the Northwestern researchers combed through and analyzed a massive dataset comprising over 800,000 paintings collected from museums and galleries, covering the careers of 2,128 artists. They also mined a dataset from the Internet Movie Database (IMDB), which included 79,000 films by 4,337 directors, as well as datasets from the Web of Science and Google Scholar to analyze the careers of 20,400 scientists.

The algorithms were tuned to find hot streaks, measured by auction price of sold paintings, IMDB ratings, and academic paper citations. The timing of these streaks was then analyzed in order to uncover any patterns in the individuals’ career trajectories four years before and after their most creative periods.

Across the board, an overarching pattern emerged whereby the most successful artists, film directors, and scientists engaged in episodes of exploration, straying out from the beaten path, followed by a lucrative period of exploitation. On average, a hot streak lasted only five years.

Those who only experimented and didn’t buckle up to exploit a single creative thread had a significantly lower chance of entering a hot streak. Similarly, those who only stayed in the trenches and solely employed exploitation had lower odds of achieving a hot streak.

“Neither exploration nor exploitation alone in isolation is associated with a hot streak. It’s the sequence of them together,” said Wang, who led the study. “Although exploration is considered a risk because it might not lead anywhere, it increases the likelihood of stumbling upon a great idea. By contrast, exploitation is typically viewed as a conservative strategy. If you exploit the same type of work over and over for a long period of time, it might stifle creativity. But, interestingly, exploration followed by exploitation appears to show consistent associations with the onset of hot streaks.”

Van Gogh, for example, experimented a lot during the years leading up to his 1888 hot streak. Prior to this period, the Dutch painter produced a myriad of drawings, portraits, and still-life paintings that were quite different from one another, and which were very different in particular from his works produced during the hot streak.

“We were able to identify among the first regularities underlying the onset of hot streaks, which appears universal across diverse creative domains,” Wang said. “Our findings suggest that creative strategies that balance experimentation with implementation may be especially powerful.”

“This knowledge can help individuals and organizations understand the different types of activities to engage in — such as exploring new domains or exploiting existing knowledge and competencies — and the optimal sequence to use in order to achieve the most significant impact,” added study co-author Jillian Chown, an assistant professor of management and organizations at Kellogg School.

The findings appeared in the journal Nature Communications.

Machine learning tool 99% accurate at spotting early signs of Alzheimer’s in the lab

Researchers at the Kaunas Universities in Lithuania have developed an algorithm that can predict the risk of someone developing Alzheimer’s disease from brain images with over 99% accuracy.

Image credits Nevit Dilmen via Wikimedia.

Alzheimer’s is the world’s leading cause of dementia, according to the World Health Organization, causing or contributing to an estimated 70% of cases. As living standards improve and the average age of global populations increase, it is very likely that the number of dementia cases will increase greatly in the future, as the condition is highly correlated with age.

However, since the early stages of dementia have almost no clear, accepted symptoms, the condition is almost always identified in its latter stages, where intervention options are limited. The team from Kaunas hopes that their work will help protect people from dementia by allowing doctors to identify those at risk much earlier.

Finding our early

“Medical professionals all over the world attempt to raise awareness of an early Alzheimer’s diagnosis, which provides the affected with a better chance of benefiting from treatment. This was one of the most important issues for choosing a topic for Modupe Odusami, a Ph.D. student from Nigeria,” says Rytis Maskeliūnas, a researcher at the Department of Multimedia Engineering, Faculty of Informatics, Kaunas University of Technology (KTU), Odusami’s Ph.D. supervisor.

One possible early sign of Alzheimer’s is mild cognitive impairment (MCI), a middle ground between the decline we could reasonably expect to see naturally as we age, and dementia. Previous research has shown that functional magnetic resonance imaging (fMRI) can identify areas of the brain where MCI is ongoing, although not all cases can be detected in this way. At the same time, finding physical features associated with MCI in the brain doesn’t necessarily prove illness, but is more of a strong indicator that something is not working well.

While possible to detect early-onset Alzheimer’s this way, however, the authors explain that manually identifying MCI in these images is extremely time-consuming and requires highly specific knowledge, meaning any implementation would be prohibitively expensive and could only handle a tiny amount of cases.

“Modern signal processing allows delegating the image processing to the machine, which can complete it faster and accurately enough. Of course, we don’t dare to suggest that a medical professional should ever rely on any algorithm one-hundred-percent. Think of a machine as a robot capable of doing the most tedious task of sorting the data and searching for features. In this scenario, after the computer algorithm selects potentially affected cases, the specialist can look into them more closely, and at the end, everybody benefits as the diagnosis and the treatment reaches the patient much faster,” says Maskeliūnas, who supervised the team working on the model.

The model was trained on fMRI images from 138 subjects from The Alzheimer’s Disease Neuroimaging Initiative fMRI dataset. It was asked to separate these images into six categories, ranging across the spectrum from healthy through to full-onset Alzheimer’s. Several tens of thousands of images were selected for training and validation purposes. The authors report that it was able to correctly identify MCI features in this dataset, achieving accuracies between 99.95% and 99.99% for different subsets of the data.

While this is not the first automated system meant to identify early onset of Alzheimer’s from this type of data, the accuracy of this system is nothing short of impressive. The team cautions that “such high numbers are not indicators of true real-life performance”, but the results are still encouraging, and they are working to improve their algorithm with more data.

Their end goal is to turn this algorithm into a portable, easy-to-use software — perhaps even an app.

“Technologies can make medicine more accessible and cheaper. Although they will never (or at least not soon) truly replace the medical professional, technologies can encourage seeking timely diagnosis and help,” says Maskeliūnas.

The paper “Analysis of Features of Alzheimer’s Disease: Detection of Early Stage from Functional Brain Changes in Magnetic Resonance Images Using a Finetuned ResNet18 Network” has been published in the journal Diagnostics.

AI helps NASA look at the Sun with new eyes

The top row of images shows the degradation of AIA’s channel over the years since SDO’s launch. The bottom row of images is corrected for this degradation using a machine learning algorithm. Credit: Luiz Dos Santos/NASA GSFC.

It’s not easy being a telescope — just look at Hubble’s recent woes (and Hubble is hardly an exception). But being a solar telescope, constantly being exposed to intense light and particle bombardment, is especially rough.

Solar telescopes have to be constantly recalibrated and checked, not to ensure that damage isn’t happening — because damage is always happening. Instead, they have to be recalibrated to understand just how the instrument is changing under the effect of the Sun.

But recalibrating a telescope like NASA’s Solar Dynamics Observatory, which is in Earth orbit, isn’t easy. Its Atmospheric Imagery Assembly, or AIA, created a trove of solar images enabling us to understand our star better than ever before. In order to recalibrate AIA, researchers have to use sounding rockets: smaller rockets that carry a few instruments and only fly for about 15 minutes or so into space.

The reason why the rockets are needed is that the wavelengths that AIA is analyzing can’t be observed from Earth. They’re filtered by the atmosphere. So you need the sounding rockets carrying a small telescope to look at the same wavelengths and map out how AIA’s lenses are changing.

The Sun seen by AIA in 304 Angstrom light in 2021 before degradation correction (left) and with corrections from a sounding rocket calibration (right). Credits: NASA GSFC

Obviously, the rocket procedure isn’t ideal. It costs a bit, and rockets can’t always be launched. So a group of NASA researchers looked for a more elegant solution.

“The current best calibration techniques rely on flights of sounding rockets to maintain absolute calibration. These flights are infrequent, complex, and limited to a single vantage point, however,” the new study reads. But that’s only part of the challenge.

“It’s also important for deep space missions, which won’t have the option of sounding rocket calibration,” said Dr. Luiz Dos Santos, a solar physicist  at NASA’s Goddard Space Flight Center in Greenbelt, Maryland, and lead author on the paper. “We’re tackling two problems at once.” 

First, they set out to train a machine-learning algorithm to recognize solar structures and compare them with existing AIA data — they used images from the sounding rockets for that. The idea was that, by looking at enough images of a solar flare, the algorithm could identify a solar flare regardless of AIA lens degradation; and then, it could also figure out how much calibration was needed.

After enough examples, they gave the algorithm images to see if it would correctly identify just how much calibration was needed. The approach worked on multiple wavelengths.

“This was the big thing,” Dos Santos said. “Instead of just identifying it on the same wavelength, we’re identifying structures across the wavelengths.” 

This image shows seven of the ultraviolet wavelengths observed by the Atmospheric Imaging Assembly on board NASA’s Solar Dynamics Observatory. The top row is observations taken from May 2010 and the bottom row shows observations from 2019, without any corrections, showing how the instrument degraded over time.
Credits: Luiz Dos Santos/NASA GSFC.

When they compared the virtual calibration (algorithm calibration predictions) with the data from the sounding rockets, the results were very similar, indicating that the algorithm had done a good job at estimating what type of calibration was needed.

The approach can also be used for more space missions, even for deep space missions where calibration methods with rockets won’t be possible.

The study was published in the journal Astronomy and Astrophysics.

AI reveals new details about Dead Sea Scrolls scribes

Part of Dead Sea Scroll number 28a (1Q28a) from Qumran Cave 1, currently housed at the Jordan Museum, Amman. Credit: Osama Shukir.

The Dead Sea Scrolls contain the oldest manuscripts of the Hebrew Bible, also known as the Old Testament, as well as previously unknown ancient Jewish texts. These invaluable documents, some of which date as early as the 4th century BCE, provide a unique vantage point of the Bible’s ancient scribal culture ‘in action’. But who was behind these monumental religious artifacts?

With the exception of a handful of named scribes in a few documentary texts, the vast majority of scribes are anonymous. This is particularly true for the more than a thousand scrolls retrieved at the caves near Qumran in the Judaean Desert, near the Dead Sea, which represent the largest trove of Dead Sea Scrolls.

Now, researchers at the University of Groningen used sophisticated neural networks and their expertise in the humanities to reveal new insights about these anonymous scribes. According to their new study published today in the journal PLOS ONE, although the handwriting may seem identical to the untrained eye, at least one of the Dead Sea Scrolls was written by multiple scribes who mirrored each other’s writing styles. Previously, some scholars suggested that some manuscripts should be attributed to a single scribe based on similar handwriting.

The team led by Mladen Popović, professor of Hebrew Bible and Ancient Judaism at the Faculty of Theology and Religious Studies at the University of Groningen, focused on the famous Great Isaiah Scroll from Qumran Cave 1. This is a lengthy text, which contains the letter aleph, or “a”, at least five thousand times.

“The human eye is amazing and presumably takes these levels into account too. This allows experts to “see” the hands of different authors, but that decision is often not reached by a transparent process,” Popović says. “Furthermore, it is virtually impossible for these experts to process the large amounts of data the scrolls provide.”

This is why Popović and colleagues turned to computer algorithms that are well suited to analyzing large datasets, including comparing subtle differences in the characters, such as their curvature (textural analysis).

Two 12×12 Kohonen maps (blue colourmaps) of full character aleph and bet from the Dead Sea Scroll collection. Each of the characters in the Kohonen maps is formed from multiple instances of similar characters (shown with a zoomed box with red lines). These maps are useful for chronological style development analysis. Credit: Maruf A. Dhali, University of Groningen.

The researchers, which included experts in artificial intelligence, developed an artificial neural network that can be trained using deep learning. This neural network was able to separate the 54 columns of text in the Great Isaiah Scroll into two distinct groups that were not distributed randomly through the text but were clustered.

Upon a closer look, which involved using various control methods to rule out noise in the data, the researchers concluded that the text was written by a second scribe who showed more variation in his writing than the first, “although their writing is very similar,” the researchers wrote.

An illustration of how heatmaps of normalized average character shapes are generated for individual letters (in this example: aleph). Credit: Maruf A. Dhali, University of Groningen.

This analysis is a perfect example of a modern interpretation of historical writing systems and manuscripts, a field of research known as paleography. In the future, the same method could be used to analyze other Qumran texts, revealing microlevel details about individual scribes and how they worked on their precious manuscripts.

The researchers will never be able to produce the identities of these scribes, but it’s amazing that seventy years after they were first discovered, the Dead Sea Scrolls are still revealing their secrets.

“This is very exciting because this opens a new window on the ancient world that can reveal much more intricate connections between the scribes that produced the scrolls. In this study, we found evidence for a very similar writing style shared by the two Great Isaiah Scroll scribes, which suggests a common training or origin. Our next step is to investigate other scrolls, where we may find different origins or training for the scribes,” Popović said.

Portrait-to-animation AI brings to life Marie Curie, Charles Darwin, and more

Marie Curie (1920). Credit: My Heritage.

Going from pictures to moving pictures was a huge leap in technology and value. We can now archive human culture in a far richer format than simple text or static photos. Now, it is even possible to fill in the blanks from the past. Using AI, researchers have transformed photos of famous people into hyper-realistic animations that shine new light upon historical figures.

Charles Darwin (1855). Credit: My Heritage.

Anyone can use the tool — fittingly named Deep Nostalgia — to animate faces in photos uploaded to the system. The new service, which was produced by genealogy site MyHeritage, uses deep learning to turn a static portrait into a short video with life-like facial expressions.

Amelia Earhart (1937). Credit: My Heritage.

Specifically, the AI uses a method known as adversarial networks (or GANs for short) in which two different AIs are pit against each other. One of the networks is responsible for producing content while the other verifies how well the content emulates references. Over billions of iterations, the AI can get very good — so good it might fool you that it is original footage.

The tool is ideal for animating old family photos and celebrity pictures. It can even work with drawings and illustrations.

In order to bring a portrait to life, the AI maps a person’s face onto footage of another. It’s essentially the same way deepfakes work to impersonate people, whether it’s Donald Trump joining Breaking Bad or Mark Zuckerberg saying things he never actually said. But since the tool doesn’t also come with fake audio, there shouldn’t be any risk of nefarious usage — yet.

Some will feel enchanted by Deep Nostalgia, while others will undoubtedly be creeped out. But regardless of how the products of this AI make you feel, I think we can all agree that the technology behind them is damn impressive.

AI traffic management could finally declog urban roads

Credit: Flickr, Marianna.

Year-by-year, traffic has only gotten worse in most cities across the world. This is particularly true for cities in Asia where the number of traffic congestions has grown exponentially due to rapid urbanization and increased median income. In the Indian capital of Delhi, for instance, drivers spend as much as 58% more time stuck in traffic compared to drivers in any other city in the world. In the face of this mounting economic, health, and environmental challenge, technology may be one of our best allies when it comes to reducing time spent in traffic.

Expanding roadways, improving public transit, and encouraging alternative forms of mobility are definitely important and have their part to play in improving traffic. However, out of all possible solutions, intelligent traffic management systems driven by artificial intelligence (AI) may have the best return on investment — by far.

Such systems employ machine learning, computer vision, and other AI technologies to make sense of large swaths of data collected by sensors and cameras that record road activity, even if markings haven’t been recently repainted by line marking machines and are poorly visible. The systems can then provide insights that local operators can use to make real-time decisions to optimize traffic, such as changing the timing of red lights.

Chinese tech giant Alibaba has already implemented a traffic management AI called City Brain in 23 cities across the country, as well as in other countries such as Malaysia. The platform crunches data in real-time from video cameras in intersections and GPS data from local cars and buses to coordinate more than 1,000 road signals around a city.

After implementing Alibaba’s solution, Hangzhou, a city of 7 million, dropped from China’s fifth most congested cities to 57th on the list. As a result, commutes have shortened drastically. Also, first responders such as fire trucks and ambulances have halved the amount of time it took to respond to emergencies.

“The cities in China are probably facing more challenges than any other city,” Wang Jian, chairman of Alibaba’s technology steering committee, told Techwire Asia.

Elsewhere, at Yanbu Industrial City, a major industrial hub in Saudi Arabia, Huawei has implemented an AI-driven traffic solution that comprises over 250 HD cameras operating at 16 major road intersections. The system supplies high-quality images and videos that feed real-time data to local officials that they can use to immediately take action or plan a long-term strategy.

Huawei’s solution, called Intelligent Traffic Management System (ITMS), runs smart algorithms that identify traffic violations in real-time, including red lights, crossing lanes, reverse driving, and lane marking infractions. Although the system runs about 100,000 data records, it is able to respond within seconds.

In the United States, the city of Pittsburgh has deployed the Surtrac intelligent traffic signal control system at 50 intersections that not only reduced travel times by 26% and wait times at intersections by 41% but also curbed transport emissions by 21%.

The next step is to have traffic signals talk to cars. Engineers in Pittsburgh have already installed short-range radios at 24 intersections. Such systems could then let drivers know of upcoming traffic conditions or inform them that lights are about to change, increasing safety and relieving congestion. Traffic engineers nationwide have not had a tool to give them anywhere near real-time estimation of transportation network states — but that has now changed.

We could be heading towards a transportation breakthrough. AI is poised to revamp urban transportation, relieving bottlenecks and chokepoints that routinely snarl our urban traffic. This could not only reduce congestion and reduce travel time but also reduce emissions (by reducing the time spent in traffic). It won’t happen overnight, but it could happen soon.

This AI module can create stunning images out of any text input

A few months ago, researchers unveiled GPT-3 — the most advanced text-writing AI ever developed so far. The results were impressive: not only could the AI produce its own texts and mimic a given style, but it could even produce bits of simple code. Now, scientists at OpenAI which developed GPT-3, have added a new module to the mix.

“an armchair in the shape of an avocado”. Credit: OpenAI

Called DALL·E, a portmanteau of the artist Salvador Dalí and Pixar’s WALL·E, the module excerpts text with multiple characteristics, analyzes it, and then creates a picture of what it understands.

Take the example above, for instance. “An armchair in the shape of an avocado” is pretty descriptive, but can also be interpreted in several slightly different ways — the AI does just that. Sometimes it struggles to understand the meaning, but if you clarify it in more than one way it usually gets the job done, the researchers note in a blog post.

“We find that DALL·E can map the textures of various plants, animals, and other objects onto three-dimensional solids. As in the preceding visual, we find that repeating the caption with alternative phrasing improves the consistency of the results.”

Details about the module’s architecture have been scarce, but what we do know is that the operating principle is the same as with the text GPT-3. If the user types in a prompt for the text AI, say “Tell me a story about a white cat who jumps on a house”, it will produce a story of that nature. The same input a second time won’t produce the same thing, but a different version of the story. The same principle is used in the graphics AI. The user can get multiple variations of the same input, not just one. Remarkably, the AI is even capable of transmitting human activities and characteristics to other objects, such as a radish walking a dog or a lovestruck cup of boba.

“an illustration of a baby daikon radish in a tutu walking a dog”. Credit: OpenAi.
“a lovestruck cup of boba”. Image credits: OpenAI.

“We find it interesting how DALL·E adapts human body parts onto animals,” the researchers note. “For example, when asked to draw a daikon radish blowing its nose, sipping a latte, or riding a unicycle, DALL·E often draws the kerchief, hands, and feet in plausible locations.”

Perhaps the most striking thing about these images is how plausible they look. It’s not just dull representations of objects, the adaptations and novelties in the images seem to bear creativity as well. There’s an almost human ambiguity to the way it interprets the input as well. For instance, here are some images it produced when asked for “a collection of glasses sitting on a table”.

Image credits: OpenAI.

The system uses a body of information consisting of internet pages. Each part of the text is taken separately and researched to see what it would look like. For instance, in the image above, it would look at thousands of photos of glasses, then thousands of photos of a table, and then it would combine the two. Sometimes, it would decide on eyeglasses; other times, drinking glasses, or a mixture of both.

DALL·E also appears capable of combining things that don’t exist (or are unlikely to exist) together, transferring traits from one to the other. This is apparent in the avocado-shaped armchair images, but is even more striking in the “snail made of harp” ones.

The algorithm also has the ability to apply some optical distortion to scenes, such as “fisheye lens view” and “a spherical panorama,” its creators note.

DALL·E is also capable of reproducing and adapting real places or objects. When prompted to draw famous landmarks or traditional food, it

At this point, it’s not entirely clear what it could be used for. Fashion and design come to mind as potential applications, though this is likely just scratching the surface of what the module can do. Until further details are released, take a moment to relax with this collage of capybaras looking at the sunset painted in different styles.

Image credits: OpenAI

Upheaval at Google signals pushback against biased algorithms and unaccountable AI

Pictured Timnit Gebru. TechCrunch, CC BY-SA.

Artificial intelligence (AI) is no longer the stuff of science fiction. In the form of machine learning tools and decision-making algorithms, it’s all around us. AI determines what news you get served up on the internet. It plays a key role in online matchmaking, which is now the way most romantic couples get together. It will tell you how to get to your next meeting, and what time to leave home so you’re not late.

AI often appears both omniscient and neutral, but on closer inspection we find AI learns from and adopts human biases. As a result, algorithms replicate familiar forms of discrimination but hide them in a “black box” that makes seemingly objective decisions.

For many workers, such as delivery drivers, AI has replaced human managers. Algorithms tell them what to do, evaluate their performance and decide whether to fire them.

But as the use of AI grows and its drawbacks become more clear, workers in the very companies that make the tools of algorithmic management are beginning to push back.

Trouble at Google

One of the most familiar forms of AI is the Google algorithm, and the order in which it presents search results. Google has an 88% market share of internet searches, and the Google homepage is the most visited page on the entire internet. How it determines its search results is hugely influential but completely opaque to users.

Earlier this month, one of Google’s lead researchers on AI ethics and bias, Timnit Gebru, abruptly left the company. Gebru says she was fired after an internal email sent to colleagues about racial discrimination and toxic work conditions at Google, while senior management maintains Gebru resigned over the publication of a research paper.

Gebru’s departure came after she put her name to a paper flagging the risk of bias in large language models (the kind used by Google). The paper argued such language models could hurt marginalised communities.

Gebru has previously shown that facial recognition technology was highly inaccurate for Black people.

Google’s response rapidly stirred unrest among Google’s workforce, with many of Gebru’s colleagues supporting her account of events.

Further annoying Gebru’s coworkers and academic sympathisers was the perceived attempt to muzzle unwelcome research findings, compromising the perception of any research published by in-house researchers.

When algorithms make decisions

Here are a few examples of how algorithms can recycle and reinforce existing prejudices:

  • Automated resume-scanning systems have been found to discriminate against African-American names, graduates of women’s colleges, and even the word “women” in a job application.
  • Credit-scoring AI that can cut people off from public benefits such as health care, unemployment and child support has been found to penalise low-income individuals.
  • Misplaced trust in algorithms lay at the heart of Australia’s Robodebt debacle in which the assumption of a regular week-to-week wage packet was baked into the system.

Human systems have checks and balances and higher authorities that can be appealed to when there is an apparent error. Algorithmic decisions often do not.

In our research forthcoming in the journal Organization my colleagues and I found that this lack of a right of appeal, or even a pathway to appeal, reinforces forms of power and control in workplaces.

Now what?

So AI, an influential tool of the world’s largest corporations, appears to systematically disadvantage minorities and economically marginalised people. What can be done?

The protest initiated and led by Google’s own employees may yet bring about change inside the company. Internal discontent at the online giant did get results two years ago, when protest over the kid-glove treatment of executives facing complaints of sexual misconduct led to a change in the company’s policy.

Outsiders are also beginning to take more of an interest. The European Union’s General Data Protection Regulation (GDPR), which has boosted privacy standards since 2018, taught regulators around the world that the black box of algorithmic decision-making can indeed be prised open.

The G7 group of leading economies recently set up a Global Partnership on Artificial Intelligence to drive discussion around regulatory solutions to these problems, but it is still in its infancy.

As an industrial relations issue, the use of AI in hiring and management needs to be brought into the scope of collective bargaining agreements. Current workplace grievance procedures may allow human decisions to be appealed to a higher authority, but will be inadequate when the decisions are not made by humans – and people in authority may not even know how the AI arrived at its conclusions.

Until internal protests or outside intervention start to impact on the way AI is designed, we will continue to rely on self-regulation. Given the events of the past week, this may not inspire a great deal of confidence.


Michael Walker, Adjunct Fellow, Macquarie University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

AI makes stunning protein folding breakthrough — but not all researchers are convinced

Within every biological body, there are thousands of proteins, each twisted and folded into a unique shape. The formation of these shapes is crucial to their function, and researchers have struggled for decades to predict exactly how this folding will take place.

Now, AlphaFold (the same AI that mastered the games of chess and Go) seems to have solved this problem, essentially paving the way for a new revolution in biology. But not everyone’s buying it.

An AlphaFold prediction against the real thing.

What the big deal is

Proteins are essential to life, supporting practically all its functions, a DeepMind blog post reads. The Google-owned lab British artificial intelligence (AI) research became famous in recent years as their algorithm became the best chess player on the planet, and even surpassed humans in Go — a feat once thought impossible. After toying with a few more games, the DeepMind team set its eyes on a real-life task: protein folding.

In 2018, the team announced that AlphaFold 2 (the second version of the protein folding algorithm) has become quite good at predicting the 3D shapes of proteins, surpassing all other algorithms. Now, two years later, the algorithm seems to have been perfected even more.

In a global competition called Critical Assessment of protein Structure Prediction, or CASP, AlphaFold 2 and other systems are given the amino acid strings for proteins and asked to predict their shape. The competition organizers already know the actual shape of the protein, but of course, they keep it secret. Then, the prediction is compared to real-world results. DeepMind CEO Demis Hassabis calls this the “Olympics of protein folding” in a video.

AlphaFold nailed it. Not all its predictions were spot on, but all were very close — it was the closest thing to perfection ever seen since CASP kicked off.

“AlphaFold’s astonishingly accurate models have allowed us to solve a protein structure we were stuck on for close to a decade,” Andrei Lupas, Director of the Max Planck Institute for Developmental Biology and a CASP assessor, said in the DeepMind blog.

CASP uses the “Global Distance Test (GDT)” metric, assessing accuracy from 0 to 100. AlphaFold 2 achieved a median score of 92.4 across all targets, which translates to an average error of approximately 1.6 Angstroms, or about the width of an atom.

Improvements have been slow in the protein folding competition. Image credits: DeepMind.

It’s not perfect. Even one Angstrom can be to big of an error and render the protein useless, or even worse. But the fact that it’s so close suggests that a solution is in sight. The problem has seemed unsolvable for so long that researchers were understandably excited.

“We have been stuck on this one problem – how do proteins fold up – for nearly 50 years. To see DeepMind produce a solution for this, having worked personally on this problem for so long and after so many stops and starts, wondering if we’d ever get there, is a very special moment.”

Why protein folding is so important

It can take years for a research team to identify the shape of individual proteins — and these shapes are crucial for biological research and drug development.

A protein’s shape is closely linked to the way it works. If you understand its shape, you also have a pretty good idea of how it works.

Having a method to predict this rapidly and without hard and extensive work could usher in a revolution in biology. It’s not just the development of new drugs and treatments, though that would be motivation enough. Development of enzymes that could break down plastic, biofuel production, even vaccine development could all be dramatically sped up by protein folding prediction algorithms.

Essentially, protein folding has become a bottleneck for biological research, and it’s exactly the kind of field where AI could make a big difference, unlocking new possibilities that seemed impossible even a few years ago.

At a more foundational level, mastering protein folding can even get us closer to understanding the biological building blocks that make up the world. Professor Andrei Lupas, Director of the Max Planck Institute for Developmental Biology and a CASP assessor, commented that:

“AlphaFold’s astonishingly accurate models have allowed us to solve a protein structure we were stuck on for close to a decade, relaunching our effort to understand how signals are transmitted across cell membranes.”

Why not everyone is convinced

The announcement of DeepMind’s achievements sent ripples through the science world, but not everyone was thrilled. A handful of researchers raised the point that just because it works in the CASP setting, doesn’t really mean it will work in real life, where the possibilities are far more varied.

Speaking to Business Insider, Max Little, an associate professor and senior lecturer in computer science at the University of Birmingham expressed skepticism about the real-world applications. Professor Michael Thompson, an expert in structural biology at the University of California, took to Twitter to express what he sees as unwarranted hype (see above), making the important point that the team at DeepMind hasn’t shared its code, and they haven’t even published a scientific paper with the results. Thompson did say “the advance in prediction is impressive.” He added: “However, making a big step forward is not the same as ‘solving’ a decades-old problem in biology and chemical physics.”

Lior Pachter, a professor of computational biology at the California Institute of Technology, echoed these feelings. It’s an important step, he argued, but protein folding is not solved by any means.

Just how big this achievement is remains to be seen, but it’s an important one no matter how you look at it. Whether it’s a stepping stone or a true breakthrough is not entirely clear at this moment, but researchers will surely help clear this out as quickly as possible.

In the meantime, if you want to have a deeper look at how AlphaFold was born and developed, here’s a video that’s bound to make you feel good:

Text AI can produce images — and it’s very good at it

This AI was designed to work with text. Now, researchers have tweaked it to work with images, predicting pixels and filling out incomplete images.

GPT-2 is a text-generating algorithm. Trained on billions and billions of pages of words, it’s capable of absorbing the structure of the text and then writing texts of its own, starting from simple prompts. The algorithm also uses unsupervised learning, which makes it much easier for researchers to train it without taking a lot of their time. The AI system was presented in February and proved capable of writing convincing passages of English.

Now, researchers have put GPT-2 up to a different task: working with images.

The algorithm itself is not well-suited to working with images, at least not in a conventional sense. It was designed to work with one-dimensional data (strings of letters), not 2D images.

To bypass this shortcoming, researchers unfurled images into a single string of pixels, essentially treating pixels as if they were letters. After the algorithm was trained thusly, the new version of the algorithm was called iGPT.

They then fed halves of images and asked the AI to complete the picture. Here are some examples:

Image credits: OpenAI.

The results are already impressive. If you look at the lower half of the photos above, they’re all generated by the AI, pixel by pixel, and they look eerily realistic. The three birds, for instance, are shown standing on different surfaces, all of them believable. The droplets of water too show different veridic possibilities, and all in all, it’s an amazing accomplishment from iGPT.

This also hints at one of the holy grails of machine learning: generalizable algorithms. Nowadays, AIs can be very good at a single task (whether it’s chess, text, or images), but it’s still only one task. Using one algorithm for multiple tasks is an encouraging sign for generalizable approaches.

The results are even more exciting when you consider that GPT-2 is already last year’s AI. Recently, the next generation, GPT-3, was presented by researchers and it’s already putting its predecessor to shame, by generating some stunningly realistic texts.

There’s no telling what GPT-3 will be capable of, both in terms of text generation and image generation. It’s exciting — and a little bit scary — to imagine the results.

The original paper can be read here.

AI identifies prostate cancer with stunning accuracy

The algorithm was able to identify cancer from biopsies with comparable results to an experimented pathologist — and the algorithm could be applied to detecting other types of cancer.

Prostate biopsy with cancer probability (blue is low, red is high). This case was originally diagnosed as benign but changed to cancer upon further review. The AI accurately detected cancer in this tricky case. Image credits: Ibex Medical Analytics.

Pattern recognition is one of the things artificial intelligence (AI) does best, which makes it excellently suited for medical imaging analysis. We’ve already seen algorithms detect several diseases with doctor-like accuracy, and the results keep improving. This approach has the potential to accelerate the detection of numerous conditions and to supplement the medical workforce and expertise.

In the latest such effort, an AI was trained to detect prostate cancer from biopsy samples. The researchers started with a million image samples that had been previously labeled by expert pathologists. The AI was ‘taught’ to detect cancer and was then tested on 1,600 slides taken from 100 patients from the University of Pittsburgh Medical Center (UPMC) who were suspected of prostate cancer.

It did very well. The algorithm demonstrated 98% sensitivity and 97% specificity — in other words, it correctly detected those with the disease in 98% of the time (sensitivity), and correctly detected those without the disease in 97% of the time (specificity).

The results are comparable to those of experienced doctors, researchers say.

“Humans are good at recognizing anomalies, but they have their own biases or past experience,” said senior author Rajiv Dhir, M.D., M.B.A., chief pathologist and vice chair of pathology at UPMC Shadyside and professor of biomedical informatics at Pitt. “Machines are detached from the whole story. There’s definitely an element of standardizing care.”

In addition, the algorithm’s usefulness extends beyond cancer detection. It was also able to flag other tumor parameters, such as grade, size, and invasion of the surrounding nerves, which are clinically important features.

In addition, the AI also flagged six slides that were not noted by doctors, and seems to be very useful in detecting atypical types of tumors.

“Algorithms like this are especially useful in lesions that are atypical,” Dhir said. “A nonspecialized person may not be able to make the correct assessment. That’s a major advantage of this kind of system.”

However, this is not to say that the AI is already better than doctors. The AI only looks at the biopsy image, whereas doctors look at the entire pathology to draw a conclusion, but this could be a failsafe mechanism to catch cases, especially in hospitals where expertise is scarce.

Dhir says that the algorithm could be trained to detect other types of cancer, but the training process has to be done from scratch. However, he concludes that there’s no real reason why this technology couldn’t be adapted to detect other types of cancer, like breast cancer for instance.

The study has been published in The Lancet Digital Health.

Robot teches itself to do sutures by watching YouTube videos

A joint project between the University of California, Berkeley, Google Brain, and the Intel Corporation aims to teach robots how to perform sutures — using YouTube.

Image credits Ajay Tanwani et al., Motion2Vec.

The AIs we can produce are still limited, but they are very good at rapidly processing large amounts of data. This makes them very useful for medical applications, such as their use in diagnosing Chinese patients during the early months of the pandemic. They’re also lending a digital hand towards finding a treatment and vaccine for the virus.

But actually taking part in a medical procedure isn’t something that they’ve been able to pull off. This work takes a step in that direction, showing how deep-learning can be applied to automatically create sutures in the operating room.

Tutorial tube

The team worked with a deep-learning setup called a Siamese network, created from two or more deep-learning networks sharing the same data. One of their strengths is the ability to assess relationships between data, and they have been used for language detection applications, facial detection, and signature verification.

However, training AIs well requires massive amounts of data, and the team turned to YouTube to get it. As part of a previous project, the researchers tried to teach a robot to dance using videos. They used the same approach here, showing their network video footage of actual procedures. Their paper describes how they used YouTube videos to train a two-armed da Vinci surgical robot to insert needles and perform sutures on a cloth device.

“YouTube gets 500 hours of new material every minute. It’s an incredible repository,” said Ken Goldberg from UC Berkeley, co-author of the paper. “Any human can watch almost any one of those videos and make sense of it, but a robot currently cannot—they just see it as a stream of pixels.”

“So the goal of this work is to try and make sense of those pixels. That is to look at the video, analyze it, and be able to segment the videos into meaningful sequences.”

It took 78 instructional videos to train the AI to perform sutures with an 85% success rate, the team reports. Eventually, they hope, such robots could take over simple, repetitive tasks to allow surgeons to focus on their work.

We’re nowhere near having a fully-automated surgery team, but in time, the authors hope to build robots that can interact with and assist the doctors during procedures.

The report “Motion2Vec: Semi-Supervised Representation Learning from Surgical Videos” is available here.

Artificial Intelligence is helping economists devise a fairer tax system

Rivers of ink have been spilled on taxes. They are an important part of the very foundation on which our society is built, with former U.S. Supreme Court Justice Oliver Wendell Holmes Jr. proudly declaring, “I like to pay taxes. With them, I buy civilization.”

But taxes are, almost universally, hated — and it’s easy to understand why. You pay a significant part of your income to a rather abstract administration, and you don’t always get to see the fruits of your labor. You can’t exactly point to a tree or a sidewalk and say “my taxes built this”, and it can be pretty demoralizing to not know what your money is spent on. Plus, taxes cost a lot. Mark Twain once famously complained that ” The only difference between a tax man and a taxidermist is that the taxidermist leaves the skin.”

The real problem with taxes, aside from easily understandable inconveniences, is that they’re often unfair — and the simplest proof of that is rising income inequality.

Homo economicus

Income inequality is one of the most pressing economic problems of our times. As the old saying goes, the rich get richer, and an alarmingly high portion of the population gets left behind, often in poverty. In the US, 11.8% of the population (38.1 million people) live in poverty. Meanwhile, the world’s richest have gained $1.2 trillion in wealth in 2019 alone. Taxation is the main mechanism through which inequality can be addressed, but devising a tax system that works for everybody is not an easy task.

Tax too little and you’ll end up with rampant inequality. Tax too much and you’ll discourage people from working or seeking wealth. Finding a custom-tailored balance is not an easy task. Economists have long searched for ways to satisfy both needs, but the balance is difficult to achieve.

Economics is further complicated by human behavior. For centuries, economists have developed theories based on Homo economicus — the idea that humans are perfectly rational beings, taking rational decisions. That is, of course, not the case. It may often be a satisfactory approximation, but it is becoming increasingly apparent that Homo economicus will just not do in our modern times.

People’s economic behavior is complex, unpredictable, and it’s very hard to get data on this. Economists too have varying opinions on this. Place 10 economists in a room and ask them to come up with a taxation system, and you’ll end up with 11 solutions.

This is why Artificial Intelligence (AI) can help — or so a group of scientists believes.

The scientists, working at US technology company Salesforce, devised an artificial intelligence system charged with developing the ideal tax system.

Beating us at chess, Starcraft, and now, economics

The new program, called AI Economist, uses the same technique behind AIs such as AlphaGo or AlphaZero. These algorithms are famous for defeating humans (and other algorithms) at chess, Go, and more recently, Starcraft.

Reinforced learning is a system of punishment and reward used in machine learning. You set a reward system for the program, and then, based on the decision it takes, it receives more or less reward. This incentivizes the algorithm to find the solutions that offer the most desirable outcomes

So far, AI Economist is still a pretty simplistic approximation. In it, four AI workers are each controlled by their own reinforcement learning models. They interact with a simplified 2D world which only includes basic resources such as wood or stone. The workers either gather resources, trade them (which can earn them money), or build houses. The AIs also have different skill levels, which incentivizes them to focus on different things. Lower-skilled workers do better if they focus on gathering resources, while higher-skilled ones fare better if they build houses.

At the end of a simulated year, they are all taxed by an AI policymaker, running its own algorithm. The policymaker’s goal is to boost the productivity and income of workers.

It’s a very simple model, but it can be repeated millions of times until the optimal behavior in the scenario is found. Then, as the general strategy is discovered, it can be scaled to more complex scenarios.

There is surprisingly much you can learn from only 4 AI workers. The fact that both workers and policymakers have their own incentive is key to mimicking a more dynamic situation where the workers and policymaker AI constantly adapt to each other. For instance, some workers learned to avoid tax by reducing their productivity to qualify for a lower tax bracket and then increasing it again. The policymaker AI had to adapt to this, and every iteration, the system would be optimized.

The simulation also showed that some strategies developed by workers only worked in some strategies adopted by the policymaker, so worker AIs also had to adapt to the policymaker.

AI tax policy

In the end, the tax policy developed by the AI Economist was an unusual hybrid.

Most taxation policies are either progressive (with high earners being taxed more) or regressive (with high earners being taxed less), but the AI implemented a bit of both. It applied the highest tax rates to the rich and the poor and the lowest to middle-income workers.

If that seems weird, well, it is. But it doesn’t necessarily mean it’s wrong. Reinforced-learning AI often comes with completely non-intuitive, almost non-human solutions. In the Go game between AlphaGo and world champion Lee Sedol, the AI made a move at one point that had all commentators thinking it was a software glitch. It was so counterintuitive and weird that no one could believe it was real — and it was the move that ended up winning the game. Similarly, AlphaZero has created completely new trends in chess, such as pushing the side pawns, which was traditionally considered to be a mistake.

AlphaGo has proved its worth, often deploying completely unintuitive moves.

Of course, this doesn’t mean that the AI Economist is necessarily right — but it’s exactly this type of out-of-the-box solution that economists were looking for.

Taxing both the rich and the poor and supporting the middle class is not something that most humans would be comfortable with, but this approach led to a smaller gap between the rich and the poor workers.

Then, researchers put this optimized approach to the test — with humans this time. They hired 100 human people through Amazon’s Mechanical Turk and asked them to play the role of workers in the simulation. They found that the policy encouraged people to behave in much the way the worker AIs did, hinting that the strategy could have a real influence in a real, human situation.

It’s still early days and it’s still far too soon to draw any conclusions. The number of interacting agents needs to be increased dramatically before we can talk about practical insights, as does the number of resources. But once all that is done, and the economic model is tweaked, the model could become instantly useful. Then, the model could be tweaked to mimic particular scenarios and situations and ran millions of times until optimal strategies are found.

Whether or not the approach will have practical insights remains to be seen. Even if it does, it will still need to convince economists, politicians, and voters — and that is a completely different ball game. Leading economists are already calling for a tax on carbon, for instance, and that is something that very few politicians are even willing to consider, let alone implement.

More than 24,000 AI-readable coronavirus scientific articles go online

Credit: Pixabay.

Scientists all over the world are racing around the clock on candidate vaccines, antiviral treatments, and just about anything they can throw at the novel coronavirus. In order to aid their efforts and accelerate unprecedented scientific action, a database that pools more than 24,000 research papers related to SARS-CoV-2 (the scientific name for the virus that causes the COVID-19 pandemic) and other coronaviruses is now online in a single place.

The most comprehensive coronavirus scientific database

The Covid-19 Open Research Dataset (CORD-19) is the work of several philanthropic and research organizations, including The National Library of Medicine (NLM) at the National Institutes of Health, the Allen Institute for AI, Georgetown University, the Chan Zuckerberg Initiative, Kaggle, Microsoft, and the White House Office of Science and Technology Policy (OSTP).

Each organization contributed with resources and know-how to the best of their ability. For instance, the NLM provided access to scientific literature while Microsoft used its engineering abilities to index and map all these thousands of articles that were scattered across the web. The Allen Institute for Artificial Intelligence (AI2), a non-profit, converted all the articles into a common structured format that can be parsed by algorithms.

Additionally, the entire dataset is machine-readable, allowing artificial intelligence (AI) systems to access and interpret the huge body of knowledge. This way, scientists might find existing safe drugs and therapies designed to treat other conditions that could prove useful in the current war on the coronavirus. Or perhaps they might find a chink in the coronavirus’ armor that has so far escaped scientists.

Previously, Microsoft researchers had employed machine learning and natural language analysis to interpret the content of thousands of biomedical papers. This initiative led to a representation of cellular regulatory networks that was exploited to make recommendations for cancer therapies.

According to MIT Technology Review, the dataset is part of AI2’s Semantic Scholar service, which employs natural language models like ELMo and BERT to plot relationships between papers.

For a long time, there has been a fierce debate among scholars regarding access to scientific papers, many of which are behind paywalls controlled by a handful of publishers.

Proponents of open access — free, unrestricted access to scientific papers — will be at least happy to learn that in this situation great efforts have been made to ensure the global research community has unhindered access to the coronavirus-related papers.

“It’s my hope that the machine-readable content will stimulate advances in computing methods that can help investigators to develop deeper understandings and approaches to addressing the COVID-19 pandemic. Developing tools to help scientists to do research and synthesize new understandings has been a long-term aspiration in AI. Work has been underway over years on methods that can answer questions, analyze and summarize the content of numerous scientific papers, assess the credibility of clinical trials, generate and test hypotheses, and guide experimentation,” Eric Horvitz, Technical Fellow and Chief Scientific Officer at Microsoft, wrote in a recent blog post.

The dataset also includes pre-publication research posted on servers like medRxiv and bioRxiv, which are open access archives for pre-print health sciences and biology research.

“Sharing vital information across scientific and medical communities is key to accelerating our ability to respond to the coronavirus pandemic,” Chan Zuckerberg Initiative Head of Science Cori Bargmann said refering to the CORD-19 project.