- Labelbox and Google Cloud have partnered to offer LLM human assessment services.
- The collaboration aims to streamline the evaluation process for generative AI applications.
- Vertex AI platform users can access an integrated solution for LLM evaluation as a managed service.
- Labelbox’s LLM assessment solution provides easy access to human raters and customizable evaluation criteria.
- Google Cloud customers can now purchase Labelbox products on the Google Cloud Marketplace, enabling a hybrid approach of AI assistance and human evaluation.
Main AI News:
As developers advance from creating prototypes to deploying generative AI applications, assessing the efficacy of large language models (LLMs) becomes paramount. Cutting-edge methods for evaluating LLMs and complex AI systems, such as RAG, typically blend automated and human evaluations. Despite the optimization of LLMs for human judgment, evaluating these models remains a time-consuming and resource-intensive endeavor.
In a bid to empower teams to confidently assess and deploy LLM applications, Labelbox has joined forces with Google Cloud. This collaboration brings Vertex AI platform users an integrated solution for LLM evaluation as a fully managed service.
Vertex AI LLM Assessment Solution
Through this LLM assessment solution, Vertex AI platform users gain access to a streamlined process. They can initiate an LLM evaluation job directly within the Vertex AI platform interface, specifying their preferred evaluation type (e.g., single model or side-by-side comparison) and criteria (e.g., question-answer, multi-turn chat, summarization). Within days, users receive quality-reviewed results from skilled evaluation professionals.
Labelbox’s LLM assessment solution facilitates easy access to human raters, aiding in the assessment of organizational LLMs across various customizable criteria – from adherence to instructions and verbosity to the relevance of responses.
Integrated APIs streamline task configuration within the Vertex AI platform, with Labelbox handling the rest before the quality assurance process begins. Furthermore, seamless visualization of the labeling team’s responses within the Vertex AI platform enables users to review and approve outputs, ensuring full control over annotation quality.
Labelbox Products Now Available on Google Cloud Marketplace
For organizations seeking a hybrid approach that combines AI assistance with human evaluation, Google Cloud customers can now procure a comprehensive suite of Labelbox products via the Google Cloud Marketplace. Offering native no-code integrations with Google Cloud’s BigQuery, CloudSQL, and Google Sheets, customers can seamlessly integrate data pipelines with Labelbox in a matter of minutes.
This offering empowers users with a data-centric AI platform encompassing data curation, AI-assisted labeling, premium data labeling services, and model diagnostics to align task-specific models and develop intelligent applications. Recent enhancements to Labelbox’s products include model distillation, reinforcement learning with human feedback (RLHF), and LLM evaluation.
Conclusion:
The collaboration between Labelbox and Google Cloud signifies a significant advancement in the market for LLM evaluation services. By offering a streamlined and integrated solution, the partnership aims to address the critical need for efficient and effective evaluation of generative AI applications. This collaboration not only simplifies the evaluation process but also emphasizes the importance of combining automated and human evaluations for optimal results. As the demand for reliable LLM evaluation services continues to grow, this partnership is poised to make a substantial impact on the market, providing organizations with the tools they need to confidently deploy AI applications.
