TL;DR:
- Edmunds has overcome data consolidation challenges and is now focusing on AI integration.
- Greg Rokita leads the technology team at Edmunds, responsible for data engineering, analytics development, and vehicle inventory.
- Edmunds started as print buying guides, transitioned to CD-ROMs, and eventually moved online.
- The company’s website offers comprehensive data on vehicle prices, dealer listings, incentives, and reviews.
- Edmunds is developing a ChatGPT plugin to leverage generative AI and expose its up-to-date data to users.
- AI integration is crucial for staying competitive in the evolving landscape.
- Rokita emphasizes the need to view data warehousing and AI/ML as part of the same system.
- Edmunds has consolidated its data infrastructure, utilizing Databricks as a unified solution.
- The company is exploring Databricks Marketplace for accessing third-party data and models.
- Sharing data sets between Edmunds and Carmax, its parent company, becomes seamless through the marketplace.
Main AI News:
For over a decade, Edmunds, a leading online resource for automotive inventory and information, has been grappling with the challenge of consolidating its data infrastructure. However, having successfully tackled this hurdle, the California-based company is now envisioning a future fueled by the power of artificial intelligence (AI) and machine learning (ML).
“We have effectively addressed most of the consolidation challenges,” reveals Greg Rokita, Assistant VP of Technology at Edmunds. “Now, our focus lies on staying ahead in this AI landscape. We aim to develop foundational frameworks that enhance the productivity of our product teams, propelling us forward in competition.”
Rokita, a veteran at Edmunds for over 18 years, has played a pivotal role in the company’s technology journey since joining as Executive Director of Technology in 2005. His current responsibilities span data engineering, analytics development, and overseeing the vehicle inventory and statistics & pricing teams.
Founded in 1966 as a series of print buying guides, Edmunds transitioned to making its data available through CD-ROMs in the 1990s. The advent of the internet prompted a shift towards an online presence. Rokita came on board during the launch of Edmunds’ first free online magazine, and his team later introduced the company’s inaugural mobile phone apps.
Today, Edmunds’ website serves as a comprehensive hub, offering users a wealth of data on new and used vehicle prices, dealer and inventory listings, national and regional incentives and rebates, as well as vehicle reviews and expert advice on buying and owning cars. In 2021, the company was acquired by Carmax for an impressive $404 million.
To maintain an edge in the AI landscape, Rokita is spearheading the creation of a revolutionary ChatGPT plugin at Edmunds. This plugin will unlock access to the company’s vast trove of unstructured data, including vehicle reviews, ratings, and editorials, empowering the generative AI capabilities of ChatGPT.
ChatGPT, developed by OpenAI, is powered by a corpus of billions of publicly available web pages known as Common Crawl. However, in a rapidly evolving world, this data quickly becomes outdated. Edmunds’ innovative plugin aims to bridge this gap by enabling ChatGPT to tap into the company’s specialized and constantly updated data.
Rokita explains the significance of this breakthrough, stating, “If you ask it, ‘How does the Toyota Camry 2022 drive?’ you’re going to get nothing. By developing a plugin, we’re exposing our most recent data.”
The ultimate goal for Edmunds is to leverage generative AI capabilities to drive traffic to its site. Users seeking more detailed information or visuals of a vehicle can simply click on a link, thereby increasing Edmunds’ website traffic.
Similar to the transformative internet revolution of the 2000s, which revolutionized numerous industries, Rokita firmly believes that we are currently standing at a new inflection point.
“Twenty to 30 years ago, the internet became ingrained in every company’s operations,” reflects Rokita. “We believe the same is happening right now with AI. Whether you’re in agriculture, industry, or construction, AI will become an integral part of your company, optimizing material procurement and determining irrigation requirements for crops, among other applications.”
For Edmunds, failure to integrate AI into their core operations would result in falling behind the competition. Rokita emphasizes the challenge faced by his team in establishing frameworks that initiate the company’s journey toward AI integration.
Rokita’s vision for a successful transition lies in dispelling the notion that data warehousing and AI/ML are distinct entities with separate systems. He asserts, “It is crucial for people to understand that these are essentially different manifestations of the same system. The data warehouse pertains to historical data, while models pertain to future data. Envision a table where both past behavior and predicted future behavior coexist on a unified timeline.“
This concept drove Rokita’s determination to consolidate Edmunds’ data infrastructure. Like many early adopters of new data technologies, Edmunds initially relied on a series of best-of-breed point solutions, beginning with dedicated data warehouses built on Oracle racks, later transitioning to specialized systems like Netezza and Teradata. The data processing involved Hadoop, followed by data loading into Netezza for query purposes.
Approximately 10 years ago, Rokita set out on a mission to consolidate this infrastructure. The first step was transitioning to the cloud. Netezza was replaced with Amazon Redshift, and later the Databricks cloud platform was introduced for data science and AI purposes. However, further consolidation was required, as the team still had to contend with different systems for data science, data warehousing, and data processing, risking data synchronization issues.
“When analysts encounter data mismatches across multiple sources, their trust diminishes,” Rokita points out. “Providing users within the organization with a consistent view of the data is critical.”
Incorporating new data warehousing capabilities into the Databricks platform, Rokita decided to move away from Redshift and Hadoop. Instead, he opted to utilize Databricks as a comprehensive layer on top of AWS, effectively handling both data processing and serving. This change not only reduced costs but also simplified operational management.
“We now have a unified system that seamlessly handles data processing and serving, with the added benefit of enabling model creation without data duplication,” Rokita highlights.
Rokita and his team are currently exploring the possibilities presented by Databricks Marketplace, a platform offering data, AI models, and applications. Databricks curates and publishes open-source models across common use cases like instruction following and text summarization. Furthermore, the marketplace attracts third-party data providers, including S&P Global, Experian, Accuweather, and LexisNexis, among others.
Rokita envisions a world where joining third-party data with Edmunds’ own data is as simple as a single click, eliminating the need for development time. This newfound ease of integrating diverse datasets into Databricks empowers the company’s analytics and ML capabilities.
“For example, you can search for demographics data on potential car shoppers and utilize it in your advertising campaigns,” Rokita explains. “With just a click, the data set appears in Databricks, streamlining the process.”
He further emphasizes that while Edmunds’ parent company, Carmax, utilizes its instance of Databricks on Microsoft Azure, Edmunds operates its own instance on AWS. Thanks to Marketplace, infrastructure unification is unnecessary.
“We often need to share data between our companies,” Rokita notes. “Now, with no development costs, we can effortlessly share datasets, opening up new avenues for collaboration. We’re thrilled not only about data sharing but also about future prospects, including model sharing and dashboard sharing.”
Conclusion:
Edmunds’ focus on consolidating its data infrastructure and embracing AI technology positions the company for success in the market. By exposing its up-to-date data through the ChatGPT plugin, Edmunds can drive traffic to its website. The integration of AI and ML is becoming essential for companies across industries, and Edmunds’ efforts in this realm will enable them to maintain their competitive edge. The consolidation of data infrastructure and the utilization of Databricks Marketplace opens up new possibilities for data and model sharing, fostering collaboration and innovation within the industry.
