Tech Desk 
Jul 16, 2024
Researchers at Apple have developed a new method, MUSCLE (an acronym for Model Update Strategy for Compatible large language model (LLM) Evolution) used to enhance users' experience when an artificial intelligence (AI) model they were used to working with receives updates.
Apple’s researchers said that users create their system to interact with an LLM, including prompt styles and techniques. Switching to a newer model can be an exhausting task that rinses their experience utilising the AI model.
An update could result in forcing users to change the method they write prompts and while early adopters of models from ChatGPT might accept this, a mainstream audience utilising iOS is expected to find this unacceptable.
To resolve this issue, the team created metrics to compare reversion and inconsistencies between different model versions. Moreover, developed a training strategy to minimise those inconsistencies from happening in the first place.
Read more: iOS 18's new 'Recovered' album to restore pictures, videos
However, it isn't clear whether this will be part of a future iOS Apple Intelligence, it's clear that the Cupertino-based tech giant is preparing itself for what happens when it does upgrade its underlying models, ensuring Siri responds in the same way, to the same queries in future.
Making AI backwards-compatible
The researchers using the new method said that they handled to reduce negative flips, which is when an old model provides a correct answer while a newer model provides an incorrect one, by up to 40%.
Additionally, the paper’s authors also argued in favour of assuring that mistakes a new model makes are consistent with those you might see an older model make.
Read more: Apple releases public beta for iOS 18, macOS, and More — Here's what's new
“We argue that there is value in being consistent when both models are incorrect, a user may have developed coping strategies on how to interact with a model when it is incorrect.", the researchers stated.
Flexing their MUSCLE
The method used to overcome these obstacles is MUSCLE which does not need the base model’s training to be changed and depends on training adapters, primarily plugins for LLMs. They referred to these as compatibility adapters.
To test if their system worked, the research team upgraded LLMs like Llama and Phi and sometimes found negative flips of up to 60% in various tasks.
Tests they ran included asking the updated model's math questions to see if they still got the answer to a particular problem correct. Using their proposed MUSCLE system, the researchers say they managed to mitigate quite a number of those negative flips. Sometimes by up to 40%.
