TL;DR:
- Portuguese team TWIZ, part of Universidade Nova de Lisboa and CMU Portugal Farfetch Chat R&D project, secures 1st Place in Alexa TaskBot Challenge 2, winning $500,000.
- The team led by João Magalhães, a researcher at NOVA LINCS, achieves victory with CMU Portugal Affiliated Ph.D. students Diogo Tavares and Diogo Silva.
- TWIZ’s work focuses on multimodal conversational agents for the online fashion marketplace, bridging natural language and visual data interactions.
- Alexa TaskBot Challenge 2 pushes boundaries in conversational AI, emphasizing complex task handling and incorporating visual aids.
- João Magalhães highlights TWIZ LLM’s impact on enriching Alexa’s conversational experience, including pioneering video dialog features.
- Challenges included improving visual interfaces and leveraging past experiences.
- CMU Portugal’s role in enabling this achievement through research and funding is acknowledged.
- TWIZ team has received substantial support, totaling around $1.3 million from Amazon.
Main AI News:
In a remarkable victory, the Portuguese team TWIZ, hailing from the Universidade Nova de Lisboa and CMU Portugal Farfetch Chat R&D project, has emerged as the victor of Alexa TaskBot Challenge 2, clinching a coveted $500,000 prize. The triumphant squad, led by João Magalhães, a distinguished researcher at NOVA LINCS, a Professor in the Department of Informatics at FCT NOVA, and the Principal Investigator at CMU Portugal Farfetch Chat R&D project, featured the notable presence of CMU Portugal Affiliated Ph.D. students Diogo Tavares and Diogo Silva.
Under the banner of Farfetch Chat R&D, the FCT NOVA team lent their unwavering support to the development of a cutting-edge multimodal conversational agent tailored for the online fashion marketplace in collaboration with Farfetch. This pioneering endeavor represents a burgeoning field of research where users engage in natural language and visual data interactions with conversational agents. Leading the Farfetch Chat R&D project was Ricardo G. Sousa, supported by other Principal Investigators, including João Paulo Costeira from Instituto Superior Técnico and Alexander Hauptmann from the Languages Technology Institute at CMU.
In this year’s edition of Alexa TaskBot Challenge 2, participants embarked on the formidable task of tackling one of the most intricate challenges in conversational AI: crafting next-generation conversational AI experiences that not only cater to the evolving needs of customers but also delight them as they navigate complex tasks. The competition expanded its horizons by incorporating a broader spectrum of hobbies and at-home activities. Teams were challenged to ingeniously incorporate visual aids into every conversation turn when a screen was available, with innovative ideas aimed at enhancing the presentation of visual aids and the seamless coordination of visual and verbal modalities being key criteria for evaluation.
João Magalhães, reflecting on this year’s competition and its associated challenges, expressed his elation, stating, “I’m extremely pleased with the team’s creative prowess in designing the groundbreaking TWIZ LLM for Alexa. TWIZ LLMs elevate the Alexa TaskBot conversational experience, infusing it with a sense of fun, richness, and multimodality. This innovation plays a pivotal role in features like ‘What’s in your fridge?’ and ‘fun facts,’ making task execution more seamless. The cherry on top of TWIZ’s accomplishments is the introduction of video dialog features. Conversations centered around video content take customer experience to an entirely new echelon, and I take immense pride in pioneering video dialog within the Alexa Prize. I firmly believe there’s a plethora of opportunities to explore in this domain.”
Diogo Silva, a Ph.D. student affiliated with CMU Portugal, contributed his perspective, noting, “Participating in the Alexa Prize Taskbot Challenge was a whirlwind experience, but in the best way possible. The team’s unwavering motivation gave rise to a plethora of exhilarating ideas, which, however, necessitated some tough decisions regarding which ones to pursue. Overall, I would assert that our most significant challenge lay in enhancing our visual interface, an integral component of the interaction, as our team lacked dedicated expertise in human-computer interaction.”
This marks the second occasion that the team has participated in the Alexa TaskBot Challenge, with Diogo Tavares, another CMU Portugal Affiliated Ph.D. student, being a constant presence in both editions. Diogo Tavares emphasized the value of their prior experience, stating, “The team and I take immense pride in our accomplishments. We believe we’ve crafted something that not only embodies cutting-edge technology but also serves as a practical and enjoyable tool. A significant portion of our success can be attributed to the insights we gained from last year’s experience, which offered us a glimpse into how users interact with chatbots in the real world. Moreover, it marked the first encounter with an Alexa device for most of us. This year, we were able to leverage our newfound knowledge to push the boundaries of what’s possible.”
The opportunity to partake in the inaugural Alexa TaskBot Challenge was made possible thanks to the ongoing research efforts in conversational assistants within the ambit of the CMU Portugal large-scale collaborative project, Farfetch Chat R&D. At that time, João Magalhães underscored the pivotal role played by CMU Portugal in the Alexa TaskBot Challenge, stating, “The Program played a pivotal role as an enabler, both through the Farfetch Chat R&D project and the funding of Ph.D. students by CMU Portugal. The Farfetch Chat R&D team at NOVA University had already established a robust research program in multimodal conversational agents, making this a natural progression. Our exceptional achievements owe much to the groundbreaking AI innovations pioneered by our students, including the robust dialog understanding Transformer algorithm and language generation algorithms capable of discussing interesting facts and answering questions about recipes and videos.”
In light of the cumulative prizes and accolades garnered by the TWIZ team between this year and the last, their support from Amazon has reached a substantial total of approximately $1.3 million.
Conclusion:
TWIZ’s impressive victory in the Alexa TaskBot Challenge 2 underscores the growing potential of multimodal conversational AI experiences. Their innovative approach not only delights customers but also highlights the significance of visual aids and multimodal interactions. This success signals a promising future for conversational AI in diverse industries, including e-commerce and customer service, as it strives to meet evolving user needs and preferences.
