Skip to main content
× has been updated to serve our three go-forward companies.

Please visit these standalone sites for more information

GE Aerospace | GE Vernova | GE HealthCare 

Press Release

GE, Siena College Scientists to Demonstrate AI Agent that Enables Machines to Acquire Language in a Classroom Style

November 08, 2019
  • Awarded funding through DARPA’s Grounded Artificial Intelligence Language Acquisition (GAILA) program to develop artificial intelligence (AI) that can learn language in a meaningful manner via visual and contextual cues
  • Will take AI from acting on statistical inferences to being able to learn and act based on a form of visual grounding.

NISKAYUNA, NY – November 8, 2019 – Could industrial machines become MacGyver-like in learning and acting on the fly to solve complex problems? One of the keys will be demonstrating AI that can meaningfully learn from visual and contextual cues. This is the focus of a new research project by scientists from GE and Siena College scientists through DARPA’s Grounded Artificial Intelligence Language Acquisition (GAILA) program.

DARPA’s GAILA program is focused on the development of AI that can achieve childlike language acquisition and understanding from visual concepts. GE scientists will build upon a well-established body of work of its computer vision research team, where it has developed and deployed AI algorithms that can interpret visual and contextual cues that range from medical and industrial inspection image data to human behavioral expressions related to public security.

“Today, 99.9% of AI is based on millions of known statistical datapoints with minimal interpretation beyond what the data says,” said Peter Tu, Chief Scientist for Artificial Intelligence at GE Research who is leading the DARPA project. “Through this project, we’re aiming to create an AI agent that can learn the meaning of things, not just the statistics of things.  This would unleash a whole new realm of capabilities for machines across multiple industry sectors.”

Tu added, “Today, AI integrated into wind farm operations can improve annualized energy output (AEP) by a few percentage points, which is all based on known datasets. But imagine if the turbines on these wind farms could observe and modify their operations based on entirely new situations they haven’t yet observed. If the AI was able to observe and derive meaningful actions in real-time, the energy output would be much greater.”

Tu explained that the AI agent under development is being designed to learn like a child learns when growing up. “Children pick up things from what they see and hear and spend time playing and experimenting,” Tu says.  “They can acquire a familiarity and context that machines can’t replicate today. We’re hoping to change that.”

GE Research has partnered with researchers from Siena College Institute of Artificial Intelligence (SCIAI), which has a strong natural language processing program and develops research in a broad range of Artificial Intelligence areas in partnership with private industry and others.

Dr. Sharon Gower Small, director of SCIAI who is leading the effort with Dr. Ting Liu from Siena, said, “Traditionally, many aspects of Natural Language Processing have relied heavily on tools that were built on large amounts of manually annotated text.  The challenge for the team at the Siena College Institute of Artificial Intelligence (SCIAI) is to develop novel techniques to acquire knowledge by combining the image analysis results from our GE partner and linguistic features generated from unsupervised machine learning techniques and vastly smaller amounts of data. These techniques will include a dialogue model that will interact with human experts to confirm/correct the learned knowledge, which mimics how children learn from their parents, teachers, and peers”,

GE Research and Siena have been awarded $500,000 for Phase I of the project, which will be completed over nine months.  The goal of Phase I is to demonstrate various forms of language acquisition based on visual grounding methods. Upon the successful completion of Phase I, DARPA will perform a down selection for Phase II of the program.

About GE Research

GE Research is GE’s innovation powerhouse where research meets reality. We are a world-class team of scientific, engineering and marketing minds working at the intersection of physics and markets, physical and digital technologies, and across a broad set of industries to deliver world-changing innovations and capabilities for our customers. To learn more, visit our website at

About Siena College Institute for Artificial Intelligence (SCIAI)

Research in the field of Artificial Intelligence consistently calls for a team effort among a wide variety of disciplines. SCIAI’s focus is to foster interdisciplinary relationships among faculty and community businesses in order to develop research in a broad range of Artificial Intelligence areas. SCIAI also focuses on offering research opportunities to our undergraduate computer science’s majors in the field of Artificial Intelligence.

To learn more, visit our website at

For media inquiries, please contact:

Todd Alhart
Director, Innovation Communications
GE Aerospace
+1 518 338 5880
[email protected]

business unit