Qualcomm Technologies has announced that it is working with Meta to optimise the execution of Meta’s Llama 2 large language models directly on-device, without relying on the sole use of cloud services.
The ability to run generative artificial intelligence (AI) models like Llama 2 on devices like smartphones, PCs, virtual reality (VR)/ augmented reality (AR) headsets and vehicles allows developers to save on cloud costs, and to provide users with private, more reliable and personalised experiences, Qualcomm said in a statement.
As a result, the chip-maker plans to “make available on-device Llama 2-based AI implementations to enable the creation of new and exciting AI applications.”
This will allow customers, partners and developers to build use cases, like intelligent virtual assistants, productivity applications, content creation tools, entertainment and more.
These new on-device AI experiences, powered by Snapdragon, can work in areas with no connectivity or even in aeroplane mode.
“To effectively scale generative AI into the mainstream, AI will need to run on both the cloud and devices at the edge, such as smartphones, laptops, vehicles, and IoT devices,” said Durga Malladi, senior vice president and general manager of technology, planning and edge solutions businesses, Qualcomm Technologies.
The chip-maker’s leadership in on-device AI uniquely positions it to support the Llama ecosystem.
“The Company has an unmatched footprint at the edge — with billions of smartphones, vehicles, XR headsets and glasses, PCs, IoT, and more, being powered by its industry-leading AI hardware and software solutions — enabling the opportunity for generative AI to scale,” Qualcomm said.
The chip-maker is scheduled to make available Llama 2-based AI implementation on-devices powered by Snapdragon starting from 2024 onwards.
However, developers can now start optimising applications for on-device AI using the Qualcomm AI Stack which is a dedicated set of tools that allow to process AI more efficiently on Snapdragon, making on-device AI possible even in small, thin and light devices, the company added.
