Meta's THIS AI model generates text, images with single transformer

Although artificial intelligence (AI) has been evolving at a perpetual pace since it came into being, it still suffers from bottlenecks when it comes to training AI models.

That is because of the intricacies woven around language models, since they deal with discrete values, whereas image generation models deal with pixel values.

Multi-modal being developed these days employ techniques that result in a tarnished quality of data representation. To keep the quality of representing data intact and relatively higher, research scientists from Meta in collaboration with the University of Southern California introduced Transfusion, bringing an utmost method of handling both discrete and continuous modalities with a single model. 

Read more: GPT-40 Voice, Strawberry, Voice Engine OpenAI announced this year

Meta's new AI model transfusion

“Diffusion models and next-token-prediction autoregressive models represent the best worlds for generating continuous and discrete data respectively. This inspired us to develop a new multi-modal method that combines the best of both worlds in a natural and simple way,” Meta AI Senior Research Scientist Chunting Zhou said. 

Transfusion is a workaround to train a single model to handle multimodalities without quantization or separate modules. The primary objective behind this approach is to train a single model for both language modelling for text and diffusion for images. 

For the integration of the ability to process and generate both text and images, Transfusion merges these two objectives to train a single transformer model. 

While training this model interacts with both text and image data, prompting the implementation of the loss functions for language modeling and diffusion. 

“We show it is possible to fully integrate both modalities, with no information loss, by training a single model to both predict discrete text tokens and diffuse continuous images,” Meta researchers write in the research paper. 

Infinity Meta app 

Infinity Meta is a social media and metaverse platform that allows users to connect, create, and explore virtual worlds. To log into the Infinity Meta app, users need to download the app, create an account using their email or social media credentials, and verify their identity through a confirmation link or code sent to their email.