TL;DR:
- Code Intelligence unveils CI Spark, an AI tool for test code generation, claiming 15x productivity boost.
- Fuzzing, a technique involving randomized input testing, gains prominence in uncovering security issues.
- Google’s OSS-Fuzz project sees success with LLM-based AI in identifying under-tested code areas.
- CI, a provider of enterprise testing tools, introduces CI Spark for commercial software development.
- CI Spark features automatic test entry point identification, fuzz test generation, and unit test integration.
- Supported languages include JavaScript/TypeScript, Java, and C/C++.
- CI Spark reduces fuzz test creation time from 3 days to a few hours.
- Future plans include supporting different LLMs, enhancing test quality evaluation, and expanding language support.
Main AI News:
In the ever-evolving world of software development, Code Intelligence (CI) is making significant strides with its latest offering, CI Spark. This Large Language Model (LLM)-based AI assistant is set to revolutionize the way tests are created, boasting a claimed productivity increase of a staggering 15 times compared to manual test creation.
Traditionally, developers wrote tests to validate that their code’s input produced the expected output. While such tests remain crucial, a more potent approach to uncovering security vulnerabilities involves testing with unexpected and randomized input, deviating from the developer’s intended usage. This innovative technique, known as fuzzing, has proven invaluable in uncovering security issues that might have otherwise remained undetected.
Back in August, Google made headlines by incorporating LLM-based AI into its OSS-Fuzz project, aimed at continuous fuzz testing of open-source software. This fuzz introspection tool identifies under-tested code segments and prompts the LLM to create new fuzz test targets. If the code generated by the LLM encounters compilation issues, the tool initiates prompts for fixes. Remarkably, the results spoke for themselves, with projects experiencing code coverage gains ranging from 1.5% to an impressive 31%.
Code Intelligence stands as both a provider of enterprise testing tools and the maintainer of an open-source fuzzing tool called Jazzer, designed for Java Virtual Machine (JVM) languages. This tool has been integral to OSS-Fuzz since 2021. Given this established collaboration, it comes as no surprise that Code Intelligence is now introducing CI Spark. This new addition follows a similar approach but is tailored to the realm of enterprise software development. The company states that while OSS-Fuzz primarily focuses on securing open-source projects, CI Spark will soon extend its reach to commercial projects.
CI Spark comes equipped with a host of powerful features, including automatic identification of suitable entry points for fuzz tests, automatic fuzz test generation, assistance in enhancing existing fuzz tests, and the ability to use unit tests as hints to guide CI Spark in generating fuzz tests. Currently, the supported programming languages include JavaScript/TypeScript, Java, and C/C++.
During its initial testing within the OSS-Fuzz context, CI Spark demonstrated its prowess with JavaScript and Java projects. The company proudly claims that CI Spark has condensed the average workload required to generate a fuzz test from a laborious three days down to just a few hours.
However, it’s worth noting that CI Spark is still in its early stages, and the roadmap ahead includes plans for supporting different LLMs, introducing a new evaluation framework to assess test quality, implementing static analysis to enhance candidate identification for testing, and expanding support for additional programming languages.
In the fast-paced world of software development, the pressure to deliver features often leads to neglect in the testing department, be it unit testing or fuzz testing. This neglect can result in software that is less resilient, less secure, and harder to maintain. Applying AI to this problem is a logical step forward. Notably, GitHub’s Copilot coding assistant already supports unit test generation, and earlier this year, Tabnine introduced AI-powered unit testing. Fuzz testing with AI has been a topic of discussion for years, as evidenced by a Microsoft Research post from 2017, and it appears poised to enter the mainstream with the advent of tools like CI Spark.
Conclusion:
CI Spark’s introduction marks a significant leap in AI-powered test code generation. By simplifying and accelerating the process, it addresses a critical need in the software development market, ensuring more robust, secure, and maintainable software products. As the industry increasingly embraces AI-driven testing, CI Spark is poised to lead the way in reshaping software testing practices.
