This innovation boasts good performance improvements, with programs running up to ten times faster than their cloud-based counterparts and computer vision tasks achieving speedups of over 20 times.
Cornell University researchers have introduced Cascade, an open-source platform designed to revolutionize the way artificial intelligence (AI) models are executed, significantly reducing costs and energy consumption while enhancing performance. It is specifically tailored for real-time applications such as smart traffic management, medical diagnostics, augmented reality-based equipment servicing, digital agriculture, smart power grids, and automated product inspection in manufacturing, where rapid AI responses are critical. Already, it’s being utilized by College of Veterinary Medicine researchers to monitor cows for mastitis risk.
As the demand for AI continues to grow, many organizations have concerns about the mounting computational expenses, data privacy, and reliance on remote cloud servers. Current AI models often suffer from sluggish response times, which limits their effectiveness in scenarios requiring rapid data exchanges or real-time control of automated systems. To address these challenges, Professor Ken Birman and Senior Research Associate Weijia Song from the Cornell Ann S. Bowers College of Computing and Information Science collaborated to create Cascade, an edge computing system that keeps computation and data storage close to data sources, ensuring data security and minimizing data movement.
Cascade’s “zero-copy” edge computing design allows AI models to access data instantly, eliminating delays caused by data retrieval and resulting in faster response times. Birman emphasised, “Cascade enables users to put machine learning and data fusion close to the edge of the internet, so artificially intelligent actions can occur instantly,” distinguishing it from conventional cloud computing approaches. The performance improvements of Cascade are impressive, with most programs running two to ten times faster than cloud-based counterparts and some computer vision tasks experiencing speedups of over 20 times. Even larger AI models benefit significantly from this approach, and the transition to Cascade is often seamless for AI software. Doctoral student Alicia Yang contributed to the project by developing Navigator, a memory manager and task scheduler that further optimises AI workflows, making hardware usage more efficient and reducing processing time, particularly when multiple applications need to share resources.
In veterinary medicine, Cascade is being used to monitor dairy cows for early signs of mastitis, enhancing milk production by identifying potential issues in real-time. Thiago Garrett from the University of Oslo used Cascade to develop a “smart traffic intersection” prototype, capable of anticipating collisions within milliseconds and issuing warnings, far outperforming cloud-based systems. With the release of Cascade as an open-source platform, Birman’s team aims to encourage other researchers to explore its potential applications, democratizing AI technology and making it accessible to a broader audience. “Our goal is to see it used,” Birman emphasized, “This open-source release will allow the public to benefit from what we created.”
