Tag Archives: neural network

A neural network has learned to identify tree species from satellite

A detailed land-cover map showing forest in Chiapas state in southern Mexico. The map was produced using Copernicus Sentinel-2 optical data from 14 April 2016. The image is not part of the discussed study.

Much of what we know about forest management comes from aerial photos nowadays. Whether it’s drones, helicopters, or satellites, bird’s-eye views of forests are crucial for understanding how our forests are faring — especially in remote areas that are hard to monitor on the ground.

Satellite imagery, in particular, offers a cheap and effective tool for monitoring. But the problem with satellite data is that oftentimes, the resolution is pretty low, and it can be hard to tell what you’re looking at.

But a new study using neural networks to distinguish between satellite imagery may help with that.

Hierarchical model structure/Svetlana Illarionova et al., IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

“Commercial forest taxation providers and their end-users, including timber procurers and processors, as well as the forest industry entities can use the new technology for quantitative and qualitative assessment of wood resources in leased areas. Also, our solution enables quick evaluations of underdeveloped forest areas in terms of investment appeal,” explains Svetlana Illarionova, the first author of the paper and a Skoltech PhD student.

Illarionova and her colleagues from the Skoltech Center for Computational and Data-Intensive Science and Engineering (CDISE) and Skoltech Space Center used a neural network to automate dominant tree species’ identification in high and medium resolution images.

Classes markup of the study area. Image credits: Illarionova et al.

After training, the neural networks were able to identify the dominant tree species in the test site from Leningrad Oblast, Russia. The data was confirmed with ground-based observations during the year 2018. A hierarchical classification model and additional data, such as vegetation height, helped further enhance the predictions’ quality while improving the algorithm’s stability to facilitate its practical application.

The study focused on identifying the dominant species. Of course, among the forests with different compositions, there will be forests where the distribution is roughly equal between two or even more species, but the compositions of these mixed forests was outside the scope of the study.

“It is worth noting that the “dominant species” in forestry does not exactly match the biological term “species” and is connected mostly with the timber class and quality,” the researchers write in the paper.

Overall, the algorithm appeared capable of identifying the dominant species, although the researchers note that the outcome can be improved by a better training markup, which they plan on doing in future research

“However, in future research, we are going to cover mixed forest cases, which will fall entirely into the hierarchical segmentation scheme. The other goal is to add more forest inventory characteristics, which can also be estimated from the satellite imagery,” the study concludes.

AI reveals new details about Dead Sea Scrolls scribes

Part of Dead Sea Scroll number 28a (1Q28a) from Qumran Cave 1, currently housed at the Jordan Museum, Amman. Credit: Osama Shukir.

The Dead Sea Scrolls contain the oldest manuscripts of the Hebrew Bible, also known as the Old Testament, as well as previously unknown ancient Jewish texts. These invaluable documents, some of which date as early as the 4th century BCE, provide a unique vantage point of the Bible’s ancient scribal culture ‘in action’. But who was behind these monumental religious artifacts?

With the exception of a handful of named scribes in a few documentary texts, the vast majority of scribes are anonymous. This is particularly true for the more than a thousand scrolls retrieved at the caves near Qumran in the Judaean Desert, near the Dead Sea, which represent the largest trove of Dead Sea Scrolls.

Now, researchers at the University of Groningen used sophisticated neural networks and their expertise in the humanities to reveal new insights about these anonymous scribes. According to their new study published today in the journal PLOS ONE, although the handwriting may seem identical to the untrained eye, at least one of the Dead Sea Scrolls was written by multiple scribes who mirrored each other’s writing styles. Previously, some scholars suggested that some manuscripts should be attributed to a single scribe based on similar handwriting.

The team led by Mladen Popović, professor of Hebrew Bible and Ancient Judaism at the Faculty of Theology and Religious Studies at the University of Groningen, focused on the famous Great Isaiah Scroll from Qumran Cave 1. This is a lengthy text, which contains the letter aleph, or “a”, at least five thousand times.

“The human eye is amazing and presumably takes these levels into account too. This allows experts to “see” the hands of different authors, but that decision is often not reached by a transparent process,” Popović says. “Furthermore, it is virtually impossible for these experts to process the large amounts of data the scrolls provide.”

This is why Popović and colleagues turned to computer algorithms that are well suited to analyzing large datasets, including comparing subtle differences in the characters, such as their curvature (textural analysis).

Two 12×12 Kohonen maps (blue colourmaps) of full character aleph and bet from the Dead Sea Scroll collection. Each of the characters in the Kohonen maps is formed from multiple instances of similar characters (shown with a zoomed box with red lines). These maps are useful for chronological style development analysis. Credit: Maruf A. Dhali, University of Groningen.

The researchers, which included experts in artificial intelligence, developed an artificial neural network that can be trained using deep learning. This neural network was able to separate the 54 columns of text in the Great Isaiah Scroll into two distinct groups that were not distributed randomly through the text but were clustered.

Upon a closer look, which involved using various control methods to rule out noise in the data, the researchers concluded that the text was written by a second scribe who showed more variation in his writing than the first, “although their writing is very similar,” the researchers wrote.

An illustration of how heatmaps of normalized average character shapes are generated for individual letters (in this example: aleph). Credit: Maruf A. Dhali, University of Groningen.

This analysis is a perfect example of a modern interpretation of historical writing systems and manuscripts, a field of research known as paleography. In the future, the same method could be used to analyze other Qumran texts, revealing microlevel details about individual scribes and how they worked on their precious manuscripts.

The researchers will never be able to produce the identities of these scribes, but it’s amazing that seventy years after they were first discovered, the Dead Sea Scrolls are still revealing their secrets.

“This is very exciting because this opens a new window on the ancient world that can reveal much more intricate connections between the scribes that produced the scrolls. In this study, we found evidence for a very similar writing style shared by the two Great Isaiah Scroll scribes, which suggests a common training or origin. Our next step is to investigate other scrolls, where we may find different origins or training for the scribes,” Popović said.

Text bubble.

AI spots depression by looking at your patterns of speech

A new algorithm developed at MIT can help spot signs of depression from a simple sample (text of audio) of conversation.

Text bubble.

Image credits Maxpixel.

Depression has often been referred to as the hidden depression of modern times, and the figures seem to support this view: 300 million people around the world have depression, according to the World Health Organization. The worst part about it is that many people live and struggle with undiagnosed depression day after day for years, and it has profoundly negative effects on their quality of life.

Our quest to root out depression in our midst has brought artificial intelligence to the fray. Machine learning has seen increased use as a diagnostics aid against the disorder in recent years. Such applications are trained to pick up on words and intonations of speech that may indicate depression. However, they’re of limited use as the software draws on an individual’s answers to specific questions.

In a bid to bring the full might of the silicon brain to bear on the matter, MIT researchers have developed a neural network that can look for signs of depression in any type of conversation. The software can accurately predict if an individual is depressed without needing any other information about the questions and answers.

Hidden in plain sight

“The first hints we have that a person is happy, excited, sad, or has some serious cognitive condition, such as depression, is through their speech,” says first author Tuka Alhanai, a researcher in the Computer Science and Artificial Intelligence Laboratory (CSAIL).

“If you want to deploy [depression-detection] models in scalable way […] you want to minimize the amount of constraints you have on the data you’re using. You want to deploy it in any regular conversation and have the model pick up, from the natural interaction, the state of the individual.”

The team based their algorithm on a technique called sequence modeling, which sees use mostly in speech-processing applications. They fed the neural network samples of text and audio recordings of questions and answers used in diagnostics, from both depressed and non-depressed individuals, one by one. The samples were obtained from a dataset of 142 interactions from the Distress Analysis Interview Corpus (DAIC).

The DAIC contains clinical interviews designed to support the diagnosis of psychological distress conditions such as anxiety, depression, and post-traumatic stress disorder. Each subject is rated ,in terms of depression, on a scale between 0 to 27, using the Personal Health Questionnaire. Scores between moderate (10 to 14) and moderately severe (15 to 19) are considered depressed, while all others below that threshold are considered not depressed. Out of all the subjects in the dataset, 28 (20 percent) were labeled as depressed.

Simple diagram of the network. LSTM stands for Long Short-Term Memory.
Image credits Tuka Alhanai, Mohammad Ghassemi, James Glass, (2018), Interspeech.

The model drew on this wealth of data to uncover speech patterns for people with or without depression. For example, past research has shown that words such as “sad,” “low,” or “down,” may be paired with audio signals that are flatter and more monotone in depressed individuals. Individuals with depression may also speak more slowly and use longer pauses between words.

The model’s job was to determine whether any patterns of speech from an individual were predictive of depression or not.

“The model sees sequences of words or speaking style, and determines that these patterns are more likely to be seen in people who are depressed or not depressed,” Alhanai says. “Then, if it sees the same sequences in new subjects, it can predict if they’re depressed too.”

Samples from the DAIC were also used to test the network’s efficiency. It was measured on its precision (whether the individuals it identified as depressed had been diagnosed as depressed) and recall (whether it could identify all subjects who were diagnosed as depressed in the entire dataset). It scored 71% on precision and 83% on recall for an averaged combined score of 77%, the team writes. While it may not sound that impressive, the authors write that this outperforms similar models in the majority of tests.

The model had a much harder time spotting depression from audio than text. For the latter, the model needed an average of seven question-answer sequences to accurately diagnose depression. With audio, it needed around 30 sequences. The team says this “implies that the patterns in words people use that are predictive of depression happen in a shorter time span in text than in audio,” a surprising insight that should help tailor further research into the disorder.

The results are significant as the model can detect patterns indicative of depression, and then map those patterns to new individuals, with no additional information. It can run on virtually any kind of conversation. Other models, by contrast, only work with specific questions — for example, a straightforward inquiry, “Do you have a history of depression?”. The models then compare a subject’s response to standard ones hard-wired into their code to determine if they are depressed.

“But that’s not how natural conversations work,” Alhanai says.

“We call [the new model] ‘context-free,’ because you’re not putting any constraints into the types of questions you’re looking for and the type of responses to those questions.”

The team hopes their model will be used to detect signs of depression in natural conversation. It could, for instance, be remade into a phone app that monitors its user’s texts and voice communication for signs of depression, and alert them to it. This could be very useful for those who can’t get to a clinician for an initial diagnosis, due to distance, cost, or a lack of awareness that something may be wrong, the team writes.

However, in a post-Cambridge-Analytica-scandal world, that may be just outside of the comfort zone of many. Time will tell. Still, the model can still be used as a diagnosis aid in clinical offices, says co-author James Glass, a senior research scientist in CSAIL.

“Every patient will talk differently, and if the model sees changes maybe it will be a flag to the doctors,” he says. “This is a step forward in seeing if we can do something assistive to help clinicians.”

Truth be told, while the model does seem very good at spotting depression, the team doesn’t really understand what crumbs it follows to do so. “The next challenge is finding out what data it’s seized upon,” Glass concludes.

Apart from this, the team also plans to expand their model with data from many more subjects — both for depression and other cognitive conditions.

The paper “Detecting Depression with Audio/Text Sequence Modeling of Interviews” has been published in the journal Interspeech.

Google AI can now look at your retina and predict the risk of heart disease

Google researchers are extremely intuitive: just by looking into people’s eyes they can see their problems — cardiovascular problems, to be precise. The scientists trained artificial intelligence (AI) to predict cardiovascular hazards, such as strokes, based on the analysis of retina shots.

The way the human eye sees the retina vs the way the AI sees it. The green traces are the pixels used to predict the risk factors. Photo Credit: UK Biobank/Google

After analyzing data from over a quarter million patients, the neural network can predict the patient’s age (within a 4-year range), gender, smoking status, blood pressure, body mass index, and risk of cardiovascular disease.

“Cardiovascular disease is the leading cause of death globally. There’s a strong body of research that helps us understand what puts people at risk: Daily behaviors including exercise and diet in combination with genetic factors, age, ethnicity, and biological sex all contribute. However, we don’t precisely know in a particular individual how these factors add up, so in some patients, we may perform sophisticated tests … to help better stratify an individual’s risk for having a cardiovascular event such as a heart attack or stroke”, declared study co-author Dr. Michael McConnell, a medical researcher at Verily.

Even though you might think that the number of patients the AI was trained on is large, AI networks typically work with much larger sample sizes. In order for neural networks to be more accurate in their predictions, they must analyze as much data as possible. The results of this study show that, until now, the predictions made by AI cannot outperform specialized medical diagnostic methods, such as blood tests.

“The caveat to this is that it’s early, (and) we trained this on a small data set,” says Google’s Lily Peng, a doctor and lead researcher on the project. “We think that the accuracy of this prediction will go up a little bit more as we kind of get more comprehensive data. Discovering that we could do this is a good first step. But we need to validate.”

The deep learning applied to photos of the retina and medical data works like this: the network is presented with the patient’s retinal shot, and then with some medical data, such as age, and blood pressure. After seeing hundreds of thousands of these kinds of images, the machine will start to see patterns correlated with the medical data inserted. So, for example, if most patients that have high blood pressure have more enlarged retinal vessels, the pattern will be learned and then applied when presented just the retinal shot of a prospective patient. The algorithms correctly discovered patients who had great cardiovascular risks within a 5-year window 70 percent of the time.

“In summary, we have provided evidence that deep learning may uncover additional signals in retinal images that will allow for better cardiovascular risk stratification. In particular, they could enable cardiovascular assessment at the population level by leveraging the existing infrastructure used to screen for diabetic eye disease. Our work also suggests avenues of future research into the source of these associations, and whether they can be used to better understand and prevent cardiovascular disease,” conclude the authors of the study.

The paper, published in the journal Nature Biomedical Engineering, is truly remarkable. In the future, doctors will be able to screen for the number one killer worldwide much more easily, and they will be doing it without causing us any physical discomfort. Imagine that!

Duo of neural networks get within a pixel of reading our mind and re-creating what’s there

Machines are starting to peer into the brain, see what we’re thinking, and re-create it.


Image credits Nathan Sawaya / PxHere.

Full disclosure here, but I’m not the hardest worker out there. I also have a frustrating habit of timing my bouts of inspiration to a few minutes after my head hits the pillow.

In other words, I wave most of those bouts goodbye on my way to dream town.

But the work of researchers from Japan’s Advanced Telecommunications Research Institute (ATR) and Kyoto University could finally let me sleep my bouts away and also make the most of them — at the same time. The team has created a first-of-its-kind algorithm that can interpret and accurately reproduce images seen or imagined by a person.

Despite still being “decades” away from practical use, the technology brings us one step closer to systems that can read and understand what’s going on in our minds.

Eyes on the mind

Trying to tame a computer to decode mental images isn’t a new idea. It’s actually been in the works for a few years now — researchers have been recreating movie clips, photos, and even dream imagery from brains since 2011. However, all previous systems have been limited in scope and ability. Some can only handle narrow domains like facial shape, while others can only rebuild images from preprogrammed images or categories (‘bird’, ‘cake’, ‘person’, so on). Until now, all technologies needed pre-existing data; they worked by matching a subject’s brain activity to that recorded earlier while the human was viewing images.

According to researchers, their new algorithm can generate new, recognizable images from scratch. It works even with shapes that aren’t seen but imagined.

It all starts with functional magnetic resonance imaging (fMRI), a technique that measures blood flow in the brain and uses that to gauge neural activity. The team mapped out 3 subjects’ visual processing areas down to a resolution of 2 millimeters. This scan was performed several times. During every scan, each of the three subjects was asked to look at over 1000 pictures. These included a fish, an airplane, and simple colored shapes.

A new algorithm uses brain activity to create reconstructions (bottom two rows) of observed photos (top row). Image credits: Kamitani Lab.

The team’s goal here was to understand the activity that comes as a response to seeing an image, and eventually have a computer program generate an image that would stir a similar response in the brain.

However, there’s where the team started flexing their muscles. Instead of showing their subjects image after image until the computer got it right, the researchers used a deep neural network (DNN) with several layers of simple processing elements.

“We believe that a deep neural network is good proxy for the brain’s hierarchical processing,” says Yukiyasu Kamitani, senior author of the study.

“By using a DNN we can extract information from different levels of the brain’s visual system [from simple light contrast up to more meaningful content such as faces]”.

Through the use of a “decoder”, the team created representations of the brain’s responses to the images in the DNN. From then on, they no longer needed the fMRI measurements and worked with the DNN translations alone as templates.

Software teaching software

Lego man.

“We are the humans now.”
Image credits Elisa Riva.

Next came a reiterative process in which the system created images in an attempt to get the DNN to respond similarly to the desired templates — be they of an animal or stained-glass window. It was a trial and error process in which the program started with neutral images (think TV static) and slowly refined them over the course of 200 rounds. To get an idea of how close it was to the desired image, the system compared the difference between the template and the DNN’s response to the generated picture. Such calculations allowed it to improve, pixel by pixel, towards the desired image.

To increase the accuracy of the final images, the team included a “deep generator network” (DGN), an algorithm that had been pre-trained to create realistic images from raw input. The DGN was, in essence, the one that put the finishing details on the images to make them look more natural.

After the DGN touched up the pictures, a neutral human observer was asked to rate the work. He was presented with two images to choose from and asked which was meant to recreate a given picture. The authors report that the human observer was able to pick the system’s generated image 99% of the time.

Next was to integrate all the work with the ‘mind-reading’ bit of the process. They asked three subjects to recall the images that had been previously displayed to them and scanned their brains as they did so. It got a bit tricky at this point, but the results are still exciting — the method didn’t work well for photos, but for the shapes, the generator created a recognizable image 83% of the time.

It’s important to note that the team’s work seems very tidy and carefully executed. It’s possible that their system actually works really well, and the bottleneck isn’t in the software but in our ability to measure brain activity. We’ll have to wait for better fMRI and other brain imaging techniques to come along before we can tell, however.

In the meantime, I get to enjoy my long-seeded dream of having a pen that can write or draw anything in my drowsy mind as I’m lying half-asleep in bed. And, to an equal extent, ponder the immense consequences such tech will have on humanity — both for good and evil.

The paper “Deep image reconstruction from human brain activity” has been published in the pre-print server biorXiv.

Artificial Intelligence can tell you your blood pressure, age, and smoking status — just by looking at your eye

Eyes are said to be the window to the soul, but according to Google engineers, they’re also the window to your health.

The engineers wanted to see if they could determine some cardiovascular risks simply by looking a picture of someone’s retina. They developed a convolutional neural network — a feed-forward algorithm inspired by biological processes, especially pattern between neurons, commonly used in image analysis.

This type of artificial intelligence (AI) analyzes images holistically, without splitting them into smaller pieces, based on their shared similarities and symmetrical parts.

The approach became quite popular in recent years, especially as Facebook and other tech giants began developing their face-recognition software. Scientists have long proposed that this type of network can be used in other fields, but due to the innate processing complexity, progress has been slow. The fact that such algorithms can be applied to biology (and human biology, at that) is astonishing.

“It was unrealistic to apply machine learning to many areas of biology before,” says Philip Nelson, a director of engineering at Google Research in Mountain View, California. “Now you can — but even more exciting, machines can now see things that humans might not have seen before.”

Observing and quantifying associations in images can be difficult because of the wide variety of features, patterns, colors, values, and shapes in real data. In this case, Ryan Poplin, Machine Learning Technical Lead at Google, used AI trained on data from 284,335 patients. He and his colleagues then tested their neural network on two independent datasets of 12,026 and 999 photos respectively. They were able to predict age (within 3.26 years), and within an acceptable margin, gender, smoking status, systolic blood pressure as well as major adverse cardiac events. Researchers say results were similar to the European SCORE system, a test which relies on a blood test.

To make things even more interesting, the algorithm uses distinct aspects of the anatomy to generate each prediction, such as the optic disc or blood vessels. This means that, in time, each individual detection pattern can be improved and tailored for a specific purpose. Also, a data set of almost 300,000 models is relatively small for a neural network, so feeding more data into the algorithm can almost certainly improve it.

Doctors today rely heavily on blood tests to determine cardiovascular risks, so having a non-invasive alternative could save a lot of costs and time, while making visits to the doctor less unpleasant. Of course, for Google (or rather Google’s parent company, Alphabet), developing such an algorithm would be a significant development and a potentially profitable one at that.

It’s not the first time Google engineers have dipped their feet into this type of technology — one of the authors, Lily Peng, published another paper last year in which she used AI to detect blindness associated with diabetes.

Journal Reference: Ryan Poplin et al. Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning.  arXiv:1708.09843

Scientist trains AI to generate Halloween costumes ideas, and some are pretty good

If you’re having problems deciding on Halloween costumes, you might find inspiration in an unexpected place: artificial intelligence (AI). Professor Panda, Strawberry shark, and Pirate firefighter are my favorites.

Image credits: Yasin Erdal.

Janelle Shane is a researcher who likes to explore the weirder side of AI. She felt that there’s often too little creativity when it comes to Halloween costumes, so she employed the help of neural networks (something she’s done several times in the past) to come up with some spooky-fresh ideas.

“I train neural networks, a type of machine learning algorithm, to write humor by giving them datasets that they have to teach themselves to mimic. They can sometimes do a surprisingly good job, coming up with a metal band called Chaosruga craft beer called Yamquak and another called The Fine Stranger (which now exists!), and a My Little Pony called Blue Cuss.”

However, it wasn’t an easy process. For starters, she didn’t have a large enough dataset to start training the AI. So she crowdsourced it by asking readers to list awesome Halloween costumes, receiving over 4,500 suggestions. There were no big surprises in the datasets. The classics dominated the list — with 42 witches, 32 ghosts, 30 pirates, 22 Batmans, 21 cats (30 including sexy cats), 19 vampires, and 17 each of pumpkins and sexy nurses. Overall, some 300 costumes (around 6%) were “sexy.” This is a bit surprising to me, since going to Halloween parties you get the feeling that much more than 6% of them focus on sexiness.

The submissions were certainly creative, and it was clear that the AI would have a tough job surpassing its human counterparts. She used a version of AI which learns words from scratch, letter by letter, with no knowledge of their meaning. Early in the training, the AI made many missteps, but as it learned and learned, it became better and better at generating costume ideas. Janelle herself took to Twitter to present some of the results:

Credits: Twitter – @JanelleCShane, via BI.

Some of the costume ideas were seriously awesome:

  • Punk Tree
  • Disco Monster
  • Spartan Gandalf
  • Starfleet Shark
  • A masked box
  • Martian Devil
  • Panda Clam
  • Potato man
  • Shark Cow
  • Space Batman
  • The shark knight
  • Snape Scarecrow
  • Gandalf the Good Witch
  • Professor Panda
  • Strawberry shark
  • Vampire big bird
  • Samurai Angel
  • Lady Garbage
  • Pirate firefighter
  • Fairy Batman

I’m telling you, shark knight, Space Batman, and Pirate firefighter are gonna be massive. Spartan Gandalf sounds like he’s just too powerful. There are many more ideas, go read them here.

The AI also came up with what could very well be Marvel’s next cast of superheroes (or spoofs).

  • The Bunnizer
  • Ladybog
  • Light man
  • Bearley Quinn
  • Glad woman
  • Robot Werewolf
  • Super Pun
  • Super of a bog
  • Space Pants
  • Barfer
  • Buster pirate
  • Skull Skywolk lady
  • Skynation the Goddess
  • Fred of Lizard

While this is still an easy-going use of AI, it raises an interesting question. After all, looking at some of the ideas on the list, you could easily mistake it for creativity. Will future, more refined AIs be… creative?

Red Laser Diffraction.

Computer chip can mimic human neurons using only beams of light

Researchers at the MIT have constructed a brain-mimicking chip that uses light instead of electricity, which could provide a significant boost in processing power and enable the wide-scale use of artificial neural networks.

Red Laser Diffraction.

Image via Wikimedia.

As far as processing power goes, nature’s designs still beat ours fair and square. Thankfully, we’re not above copying our betters, so designing a computer that uses the same architecture and functions similarly to the human brain has been a long-standing goal of the computer industry.

We have made some headway on these types of computers using algorithms known as artificial neural networks. They’ve proven themselves on tasks that would swamp traditional computers, such as detecting lies, recognizing faces, even predicting heart attacks. The catch is that such algorithms require solid processing power to work, and most computers can’t run them very well, if at all.

Follow the light

To address this shortcoming, one team of researchers has swapped the ubiquitous transistor for beams of light which mimic the activity of neurons on a chip. These devices can process information faster and use less energy than traditional chips and could be used to put together “optical neural networks” making deep learning applications many times faster and more efficient than today.

That’s because computers today rely on transistors, tiny devices that allow or cut off the flow of electricity through a circuit. They’re massively better than the vacuum tubes of yore, but still limited in what they can do. Scientists have figured out for some time now that light could speed up certain processes that computers have to perform since light waves can travel and interact in parallel so they can perform several functions at the same time. Another advantage is that once you generate light, it keeps going by itself, whereas transistors require a constant flow of energy to operate — meaning higher energy costs and the need for greater heat dispersal.

Still, one issue in particular stemmed research into optical neural networks. The first photonic processors put together by scientists using optical equipment were massive, requiring tabletops full of mirrors and precision lenses to do the same job a modest computer processor could pull off. So for a long time, light processors were considered to be a nice idea but impractical for real applications.

But in the classic MIT fashion, a team of researchers from the Institute has managed to prove everyone wrong and condense all that equipment into a modest-sized computer chip just a few millimeters across.

Thinking with lasers

Artificial neural network.

Artificial neural networks layer neurons and have the first group do a preliminary analysis, pass their results on to the next layer and so on until the data is fully crunched.,br /> Image via Wikimedia.

The device is made of silicon and simulates a network of 16 neurons in a 4 neuron by 4 layer configuration. Information is fed into the device using a laser beam split into four smaller beams. Each beam’s brightness can be altered to encode a different number or information, and the brightness of each exiting beam represents the problem’s solution (be it a number or other type of information.)

Data processing is performed by crossing different light beams inside the chip, making them interact — either by amplifying or tuning each other out. These crossings points simulate how a signal from one neuron to another can be intensified or dampened in the brain depending on the strength of the connection between them. The beams also pass through simulated neurons that further adjust their intensities.

The team then went to work testing the optical network against a traditional counterpart in vowel sound recognition. After training on recordings of 90 people making four vowel sounds, transistor-powered computers simulating a 16-neuron network got it right 92% of the time. The optical network had a success rate of just 77%, but performed the task much faster and with greater efficiency — however, the team reckons that they can get the device’s performance up to speed after they solve all the teething problems.

One of the best parts about the new network is that it relies on components made of silicon, which is already massively employed in making computer components. In other words, the optical chips could be implemented for very low costs since there’s already an infrastructure in place to allow for their production. So once the team gets works out all the kinks and upgrades it with some more neurons, we may be poised to supply very fast, very energy efficient neural networks to for a wide variety of applications — from data centers, autonomous cars, to national security services.

The study’s primary authors, Yichen Shen, a physicist, and Nicholas Harris, an electrical engineer, are starting a new company towards that end and hope to have a product ready in two years.

The paper “Neuromorphic Silicon Photonic Networks” has been published in the e-print archive ArXiv.

DeepMind can now learn how to use its memories, apply knowledge to new tasks

DeepMind is one step closer to emulating the human mind. Google engineers claim their artificial neural network can now use store data similarly to how humans access memory.

But we’re one step closer to giving it one.
Image credits Pierre-Olivier Carles / Flickr.

The AI developed by Alphabet, Google’s parent company, just received a new and powerful update. By pairing up the neural network’s ability to learn with the huge data stores of conventional computers, the programmers have created the first Differential Neural Computer, or DNC — allowing DeepMind to navigate and learn from the data on its own.

This brings AIs one step closer to working as a human brain, as the neural network simulates the brain’s processing patterns and external data banks supplying vast amounts of information, just like our memory.

“These models… can learn from examples like neural networks, but they can also store complex data like computers,” write DeepMind researchers Alexander Graves and Greg Wayne in a blog post.

Traditional neural networks are really good at learning to do one task — sorting cucumbers, for example. But they all share a drawback in learning to do something new. Aptly called “catastrophic forgetting”, such a network has to erase and re-write everything it knows before being able to learn something else.

Learn like a human, work like a robot

Our brains don’t have this problem because they can store past experience as memories. Your computer doesn’t have this problem either, as it can store data on external banks for future use. So Alphabet paired up the later with a neural network to make it behave like a brain.

The DNC is underpinned by a controller that constantly optimizes the system’s responses, comparing its results with the desired or correct answers. Over time, this allows it to solve tasks more and more accurately while learning how to apply the data it has access to at the same time.

At the heart of the DNC is a controller that constantly optimizes its responses, comparing its results with the desired and correct ones. Over time, it’s able to get more and more accurate, figuring out how to use its memory data banks at the same time. The results are quite impressive.

After feeding the London subway network into the system, it was able to answer questions which require deductive reasoning — which computers are not good at.

For example here’s one question the DNC could answer: “Starting at Bond street, and taking the Central line in a direction one stop, the Circle line in a direction for four stops, and the Jubilee line in a direction for two stops, at what stop do you wind up?”

While that may not seem like much — a simple navigation app can tell you that in a few seconds — what’s groundbreaking here is that the DNC isn’t just executing lines of code — it’s working out the answers on its own, working with the information it has in its memory banks.

The cherry on top, the DeepMind team stated, is that DNCs are able to store learned facts and techniques, and then call upon them when needed. So once it learns how to deal with the London underground, it can very easily handle another transport network, say, the one in New York.

This is still early work, but it’s not hard to see how this could grow into something immensely powerful in the future — just imagine having a Siri that can look at and understand the data on the Internet just like you or me. This could very well prove to be the groundwork for producing AI that’s able to reason independently.

And I, for one, am excited to welcome our future digital overlords.

The team published a paper titled “Hybrid computing using a neural network with dynamic external memory” describing the research in the journal Nature.

Neuroscientists read the mind of a fruit fly

Do flies dream of flying sheep? We might soon have the answer to that question, as Northwestern University neuroscientists have developed a method that allows them to pinpoint communicating neurons in a living fly’s brain — effectively paving the way for mind-reading. Their mapping of specific neural connection patterns could provide insight into the computational processes that underlie the workings of the human brain.

Image via naturetrib

Neurons rely on points of communication known as synapses to share information. As crunching sensory data is a collective effort, involving a large number or neurons, synapses are a good indicator of a brain’s processing power — and they’re the focus of Northwestern’s study.

“Much of the brain’s computation happens at the level of synapses, where neurons are talking to each other,” said Marco Gallio, assistant professor of neurobiology in Northwestern’s Weinberg College of Arts and Sciences and lead scientist of the study. “Our technique gives us a window of opportunity to see which synapses were engaged in communication during a particular behavior or sensory experience. It is a unique retrospective label.”

The team chose Drosphila melanogaster (the common fruit fly) for the study, as the insect’s brain and its communication channels are well documented, but they ran into a little problem: it’s impossible to look at a fly’s brain under the microscope as it’s performing any kind of complex activity, so they had to come up with a solution.

Their answer was to use fluorescent molecules to mark neurons, focusing on the neural networks of three of the fly’s sensory systems — smell, sight and themrosensory system.

Starting with the gene for a green fluorescent protein found in jellyfish, they were able to genetically engineer three differently-colored proteins to serve as labels. Then, to make the molecules light up only when synapses started firing, they split the molecules in two and attached one half to the neuron sending and the other to the neuron receiving the information.

As the insect was exposed to sensory triggers or performed an activity, neurons would touch briefly to communicate and the now-whole molecules started lighting up the parts of the fly’s brain that were involved in processing the information or directing movement. Even better, the fluorescent signal persists for hours after the communication event, allowing researchers to study the brain’s activity after the fact, under a microscope.

“Our results show we can detect a specific pattern of activity between neurons in the brain, recording instantaneous exchanges between them as persistent signals that can later be visualized under a microscope,” Gallio said.

By “reading” the brains they could tell if a fly had been in either heat or cold (at least for 10 minutes) an entire hour after the sensory event had happened, for example. They also could see that exposure to the scent of a banana activated neural connections in the olfactory system that were different from those activated when the fly smelled jasmine.

“Different synapses are active during different behaviors, and we can see that in the same animal with our three distinct labels,” said Gallio.

This is the kind of new technology scientists discuss in the context of President Obama’s BRAIN (Brain Research Through Advancing Innovative Neurotechnologies) Initiative, Gallio said. Such a tool will help researchers better understand how brain circuits process information, and this knowledge then can be applied to humans.

Their paper, titled “Dynamic labelling of neural connections in multiple colours by trans-synaptic fluorescence complementation” has been published in the journal Nature Communications.





Neural network image processor tells you what’s going in your pictures

Facial recognition and motion tracking is already old news. The next level is describing what you do or what’s going on – for now only in still pictures. Meet NeuralTalk, a deep learning image processing algorithm developed by Stanford engineers which uses processes similar to those used by the human brain to decipher and interpret photos. The software can easily describe, for instance, a band of people dressed up as zombies. It’s remarkably effective and freaking creepy at the same time.



A while ago ZME Science wrote about Google’s amazing neural networks and its inner workings. The network uses stacks of 10 to 30 layers of artificial neurons to dissect images and interpret them at a seemingly cognitive level. Like a child, the neural network first learns, for instance, what a book looks like and what it means, then uses this information to identify books, no matter its shape, size or colour, in other pictures. It’s next level image processing, and with each Google image query the software gets better.


Working in a similar vein, NeuralTalk also employs a neural network to analyze images, only it also returns a description covering the gist of the image. It’s eerily accurate to boast.


In the published study, lead author Fei-Fei Li, director of the Stanford Artificial Intelligence Laboratory, says NeuralTalk works similarly to the human brain. “I consider the pixel data in images and video to be the dark matter of the Internet,” Li toldThe New York Times last year. “We are now starting to illuminate it.

It’s not quite perfect though. According to Verge, a fully-grown woman gingerly holding a huge donut is tagged as “a little girl holding a blow dryer next to her head,” while an inquisitive giraffe is mislabeled as a dog looking out of a window. But we’re only seeing the first steps of an infant technology with an incredible transformative potential. Tasks that would require the attention of humans could be easily replaced by an equally effective algorithm. In effect hundreds of thousands of collective man hours could be saved. For instance, previously Google Maps had to rely on teams of employees would check every address for accuracy. When Google Brain came online, it transcribed Street View data from France in under an hour.

What the heck is this? This just how a computer "dreams", of course. Image: Google Research

Google’s AI on LSD: what a robot’s dreams look like

What the heck is this?  This just how a computer "dreams", of course. Image:  Google Research

What the heck is this? This just how a computer “dreams”, of course. Image: Google Research

In his book “Do Androids Dream of Electric Sheep”, one of my favorite writers Philip K. Dick explores what sets apart humans from androids. The theme is more valid today than it ever was, considering the great leaps in artificial intelligence we’re seeing coming off major tech labs around the world, like Google’s. Take for instance how the company employs advanced artificial neural networks to zap through a gazillion images, interpret them and return the right one you’re looking for when you make a query using the search engine. Though nothing like a human brain, the networks uses 10-30 stacked layers of artificial neurons with each layer doing its job in incremental order to come to an “answer” by the final output layer is finished. While not dead-on, the network seems to return results better than anything we’ve seen before and as a by-product, it can also “dream.” These artificial dreams output some fascinating images to say the least, going from virtually nothing (white noise) to something that looks out of a surrealist painting. Who says computers can’t be creative?

Engineers at Google were gracious enough to tell us how the AI works, but also shared some images that show how the Google’s neural network “sees” or “dreams” in a recent blog post.

Before it can identify objects on its own, the neural network is trained to understand what those objects are. By feeding millions of photos, researchers teach the network what a fork is, for instance. The network knows that an object that has a handle and 2-4 tines is likely a fork, without getting too involved in details like colour, shape or size. Once it learns, it goes through images through its neural network layer by layer, each time analyzing something different or amplifying a definite feature, until it can identify that fork. Along the way, the AI can fail though. So, to know if it does a good job or to learn which layer isn’t working as it supposed to, it helps to visualize the network’s representation of a fork. In the image below, you can see what the AI thought a dumbbell looked like.


Image: Google Research

As you can see, the AI can’t extract the essence of what a dumbbell is without attaching some human biceps or muscles in the picture as well. Probably, because most of the images of dumbbell it’s been spoon-fed featured people lifting the weights. By visualizing these mishaps, the engineers at their own learn about how their system works (it’s so complex that even though they designed it, the AI can sometimes seem to have a mind of its own) and build it better to fit their expectations.

Typically, the first layer looks for edgers and corners by analyzing contrast; intermediate layers looks for basic features like a door or leaf; the final layer assembles the whole data into a complete interpretation.  Another approach Google uses to understand what goes on exactly at each layer is to work upside down and ask the AI to enhance an input image in a way that elicits a particular interpretation. For instance, the engineers asked the neural network what it thinks an image representing a banana should look like, all starting from random noise. In the blog post, they write ” that doesn’t work very well, but it does if we impose a prior constraint that the image should have similar statistics to natural images, such as neighboring pixels needing to be correlated. ”


Image: Google Research

Here are some other examples across different classes.

objects ai network

Image: Google Research

As mentioned earlier, the each layer of the Google AI amplifies a certain feature. Here are some images which shows how the AI amplifies features at a certain layer for both photos and drawings or paintings.

google ai

Image: Google Research

google ai

Image: Google Reserach

It gets far more interesting when you move farther up the neural layers where the AI starts to interpret things at a more abstract level. Here’s what happened when the researchers asked the AI “whatever you see there, I want more of it!”

Image: Google AI

Image: Google AI

This one I found really fascinating. You can see a lot of patterns and things popping up when you look at clouds in the sky: a country’s map, ducks, a woman’s breasts. The possibilities can be endless. That’s because the human brain is great at findings and fitting patterns. This is how we’re able to make sense of the world and to a greater extent that’s what makes us such a successful species. Of course, sometimes we confuse the patterns we see with reality. That’s why some people see things that look like pyramids on Mars and think intelligent Martian aliens are real. In the image above, the Google neural network interpreted some features as birds, amplified this and then it started “seeing” birds everywhere. Here are some more interesting high abstractions.

gooogle AI

Image: Google Research

Like humans, the Google neural network is also biased towards certain features because of the kind of images its been fed. Most images are those of animals, buildings, landmarks and so on. Naturally, it interprets these features even when these aren’t present.

image dream

Image: Google Research

The Google researchers also got a bit creative themselves and instructed the network to take the final image interpretation it produced and use it as the new picture to process. This renders an endless stream of new impressions.


Image: Google Research


Image: Google Research

Ultimately, these sort of interpretations, while incredibly fun to witness, are very important. It helps Google “understand and visualize how neural networks are able to carry out difficult classification tasks, improve network architecture, and check what the network has learned during training.” But the same process could also help humans underline their own creative process. Nobody knows exactly what human creativity is. Is it inherent to humans or can a machine also be creative? Google’s neural network, but also other examples like this extremely witty chatbot, hints towards the latter. I don’t know about you, but that sounds both amazing and scary at the same time.


Why the brain gets slower as we get older

From a certain age onward, humans seem to process information at a slower pace – learning new things becomes more difficult, remembering where you put the car keys seems to give headaches, and it gets ever worse as we age even more. Neuroscientists at the University of Bristol studying dysfunctional neural communication in Alzheimer patients demonstrated that the number one likely culprit to blame are the sodium channels, which are integral membrane proteins that have a direct influence on the degree of neural excitation. Although, the research was targeted on Alzheimer patients, the scientists found that the same degradation of Na+ channels in the brains of older, otherwise healthy individuals causes a loss of cognitive performance .

To encode and transmit information, the brain uses electrical signals. The researchers, lead by Professor Andy Randall and Dr Jon Brown from the University’s School of Physiology and Pharmacology, studied the electrical activity of the brain by recording the electrical signal in the hippocampus‘ cells, which plays a crucial role in the consolidation of short-term memory to long-term memory and spatial navigation. What the researchers were basically looking for was to determine the degree of neural excitation, whose main characteristic is the action potential.

“Much of our work is about understanding dysfunctional electrical signalling in the diseased brain, in particular Alzheimer’s disease. We began to question, however, why even the healthy brain can slow down once you reach my age. Previous investigations elsewhere have described age-related changes in processes that are triggered by action potentials, but our findings are significant because they show that generating the action potential in the first place is harder work in aged brain cells”, said Professor Randall.

neuronAn action potential is a brief, large electrical signal which instantly branches out in the rest of the cell, until it reaches the edge and activates the synapses made with the myriad of neighboring neurons. As we age, these action potentials are harder to trigger, and this relative reluctance arises from changes to the activation properties of membrane proteins called sodium channels, which mediate the rapid upstroke of the action potential by allowing a flow of sodium ions into neurons.

With this in mind, scientists might be able to develop treatments or drugs which could open more sodium channels, and thus improve cognitive abilities.

“Also by identifying sodium channels as the likely culprit for this reluctance to produce action potentials, our work even points to ways in which we might be able modify age-related changes to neuronal excitability, and by inference cognitive ability.”

[SciGuru] image credit

CalTech soup displays brainlike behaviour

Researchers from the CalTech University have managed to create the first artificial neural network from DNA, a circuit built out of interactinig molecules that can recall memories based on an incomplete pattern, in pretty much the same way a brain works.

“Although brainlike behaviors within artificial biochemical systems have been hypothesized for decades,” says Lulu Qian, a Caltech senior postdoctoral scholar in bioengineering, “they appeared to be very difficult to realize.”

Made out of 112 distinct DNA strands and four artificial neuron, this artificial neural network basically plays a mind-reading game in which it tries to identify a mystery scientist. The scientists have trained this neural network to recognize four scientists, each identified by a distinct set of four yes or no questions. After thinking of a scientist, the human player provides an incomplete subset of answers to partially identify the scientist. He would then add clues by dropping DNA strands in the test tube, which correspond to the answers.

The network would then communicate by fluorescent signals, and identify which scientist the player was thinking of; alternately, it could just state that there just isn’t enough information to make an accurate guess. The researchers played this game with the network using 27 different ways of answering the questions (out of 81 total combinations) and they got a correct result every single time.

However, the neural network, at this moment, is quite limited and slow taking about eight hours and using every molecule available for each guess.