Nova AI Prefers Open Source LLMs Over OpenAI for Code Testing: Here's Why

Nova AI, a code-testing startup, opts to utilize open-source large language models (LLMs) more frequently than those from OpenAI for several reasons. Firstly, open-source LLMs offer greater flexibility and customization options, allowing Nova AI to tailor the models to their specific needs and use cases. Additionally, open-source LLMs often provide more transparent access to the underlying algorithms and code, enabling Nova AI to better understand and modify the models as necessary. Finally, the use of open-source LLMs may offer cost advantages over proprietary alternatives, aligning with Nova AI's business objectives and budget considerations.

It's widely accepted in the software development community that those who write code should not be the ones to test it. For one, many developers simply dislike testing tasks. Additionally, adhering to good auditing practices dictates that those responsible for the work should not also be the ones to verify it.

As a result, various forms of code testing, including usability tests, language- or task-specific tests, and end-to-end testing, have become focal points for a growing number of generative AI startups. TechCrunch frequently reports on such startups, including Antithesis (which raised $47 million), CodiumAI (raised $11 million), and QA Wolf (raised $20 million). New ventures, like recent Y Combinator graduate Momentic, continue to emerge.

Another player in this space is Nova AI, a year-old startup and graduate of the Unusual Academy accelerator, which recently secured a $1 million pre-seed round. The company aims to distinguish itself from competitors by offering end-to-end testing tools tailored for mid-size to large enterprises with intricate code bases and urgent needs.

Founder and CEO Zach Smith explains that while the standard Y Combinator approach is to start small, Nova AI is targeting companies with a pressing demand for its services. The company's technology automates test creation using GenAI, specifically catering to continuous integration and continuous delivery/deployment (CI/CD) environments where engineers frequently update production code.

Nova AI's inception stemmed from Smith and co-founder Jeffrey Shih's experiences as engineers at major tech firms. Smith, a former Googler, specialized in cloud-related automation, while Shih, who previously worked at Meta, Unity, and Microsoft, brings expertise in AI and synthetic data.

Unlike many AI startups building on OpenAI's GPT, Nova AI opts to minimize its reliance on OpenAI's Chat GPT-4 due to enterprise apprehensions about data privacy. Smith notes that while OpenAI assures that paid business plan data is not used for training models, large enterprises remain skeptical.

Instead, Nova AI heavily leans on open source models like Meta's Llama and StarCoder from the BigCoder community, as well as developing its own models. By avoiding sending customer data to OpenAI, Nova AI addresses enterprise concerns while also benefiting from cost-effective and proficient open source AI models.

Smith emphasizes the rapid progress of open source AI models, exemplified by Meta's latest Llama version, which rivals GPT-4. This advancement underscores the viability of alternatives to OpenAI and contributes to the growing adoption of open source AI solutions within the industry.

Source: Techcrunch‍