TL;DR:
- Westpac conducted an experiment in its Growth Labs to assess the impact of generative AI on software development.
- Engineers using generative AI tools demonstrated a 46% increase in productivity compared to a control group that employed manual coding methods.
- Code quality remained uncompromised, with no noticeable drop-off in key metrics.
- The experiment revealed that generative AI tools significantly reduced completion times, making tasks three-and-a-half times faster.
- Junior engineers were impressed by the AI assistance, while experienced engineers appreciated the reduction in laborious tasks.
- Westpac plans to integrate generative AI capabilities into its internal engineering platform, the mesh, benefiting approximately 40% of its engineers.
- The bank is also collaborating with Kasisto to develop a finance-specific large language model for enhanced conversational experiences.
- By leveraging AI technologies, Westpac aims to empower its engineers, streamline workflows, and provide exceptional service to customers.
Main AI News:
In a groundbreaking experiment conducted within Westpac’s Growth Labs, the bank witnessed an astonishing 46% increase in productivity among software engineers without compromising the quality of their code. The experiment aimed to evaluate the impact of generative AI on software development and determine whether it served as a helpful tool or a hindrance to coders and their output.
Westpac’s Growth Lab, an extension of its innovation unit called ‘Co.Labs,’ has been actively exploring the potential of generative AI for over two years. The experiment involved 60 engineers who were randomly divided into four groups. One group served as the control group, performing their tasks using traditional manual coding methods. The other three groups were equipped with generative AI tools provided by Microsoft, Amazon, and OpenAI. These groups were given a brief period to familiarize themselves with the basics of the AI tools.
The engineers were assigned seven coding tasks spanning different programming languages, including data extraction, data transformation, and creating unit tests. The experiment lasted for several days, during which the generative AI-assisted teams showcased exceptional performance.
“The headline result was quite stellar,” exclaimed David Walker, Westpac’s Chief Technology Officer. The teams leveraging generative AI tools exhibited a remarkable 46% increase in overall productivity compared to the control group. This outcome is indeed remarkable, highlighting the potential of generative AI in supporting and enhancing the coding process.
Importantly, the significant productivity gain achieved through AI assistance did not come at the expense of code quality. The bank meticulously scanned all the generated code for vulnerabilities and thoroughly assessed its maintainability and reliability. Astonishingly, there was no discernible drop-off in any of the key metrics associated with code quality.
Walker emphasized the stark contrast in completion times between the hand-coding control team and the AI-assisted teams. On average, the control team took three-and-a-half times longer to complete their tasks compared to the generative AI teams. This substantial time-saving potential further underlines the immense benefits of utilizing AI tools in software development.
Furthermore, Westpac also investigated the impact of generative AI assistance on developers with varying experience levels. Junior engineers, comprising 83% of the participants, were particularly impressed by the support provided by the AI tools. They appreciated how the tools aided them in their early careers, making their tasks more manageable. On the other hand, experienced engineers found that the AI tools relieved them of laborious tasks, enabling them to focus on more complex aspects of software development.
While the experiment was relatively brief, the outcomes were nothing short of fantastic. Participants expressed extreme excitement about incorporating these AI tools into their daily workflow. Several engineers shared their positive experiences, noting how they were able to achieve precisely what they desired in Python code simply by asking the AI the right questions. This level of AI-powered assistance holds great promise, especially for developers with limited experience in specific programming languages.
Westpac envisions implementing the outcomes of this experiment through its internal engineering platform, known as ‘the mesh.’ The mesh has served as a highly productive environment for Westpac’s engineers over the past five years, facilitating the development of user interfaces, APIs, microservices, and various technical components for applications. By integrating generative AI capabilities into the mesh, approximately 40% of Westpac’s engineers will gain immediate access to these powerful tools. The bank has already started deploying the first teams on the mesh platform, signaling the beginning of a transformative journey.
David Walker believes that engineers are the backbone of the bank, and their efficiency and effectiveness are of utmost importance. Empowering them with generative AI tooling represents a significant leap forward, enabling them to not only streamline their work but also find greater fulfillment and professional growth in their careers.
It’s worth noting that Westpac’s AI coding experiment is distinct from its collaboration with Kasisto, a conversational AI company. The bank is leveraging Kasisto’s expertise to develop a finance-specific large language model (LLM) that surpasses the capabilities of ChatGPT in terms of safety and accuracy. By training LLMs on conversations specific to banking, Westpac aims to ensure the integrity and reliability of AI-generated responses to meet the unique needs of its customers and staff. With Kasisto’s specialized model, Westpac can confidently deliver exceptional conversational experiences without the risks associated with public LLMs.
Conclusion:
Westpac’s successful AI coding experiment, resulting in a substantial productivity gain without compromising code quality, showcases the transformative potential of generative AI in the market. By implementing these technologies, businesses can empower their engineers, optimize workflows, and achieve significant efficiency gains. This experiment highlights the importance of investing in AI capabilities to stay competitive and deliver exceptional results in the fast-paced world of software development.
