Gentrace, a developer platform for testing and monitoring artificial intelligence applications, said today it has raised $8 million in an early-stage funding round led by Matrix Partners to expand large language model testing product capabilities beyond engineering teams.

Today’s Series A funding round attracted participation from Headline and K9 Ventures and brings the company’s total raised to date to more than $14 million.

Founded in 2023, Gentrace offers a testing and monitoring product that allows non-technical users to participate in the evaluation, testing and monitoring of AI applications. According to the company, many industries are rushing to add generative AI to their offerings, development teams are facing the challenge of ensuring it remains reliable and safe.

Similarly, the ability to evaluate and test large language models remains largely the domain of development and engineering teams, making it difficult to collaborate with product managers, subject matter experts, designers and quality assurance teams.

“Generative AI represents a paradigm shift in software development, but the reality is there’s way too much noise and not enough signal on how to test and build them easily or correctly,” said Doug Safreno, co-founder and chief executive of Gentrace. “We’re not just creating another dev tool — we’re reimagining how entire organizations can collaborate and build better LLM products.”

To help tackle this challenge, Gentrace announced Experiments, a tool that allows cross-functional teams to collaborate in purpose-built testing environments to assess AI model performance. They can test AI outputs directly, preview test outcomes, anticipate errors and explore scenarios while exchanging data and information freely between technical and non-technical members.

The company’s platform and Experiments interfaces with many existing tools and model providers including OpenAI, vector database Pinecone Systems Inc. and visual LLM programming environment Rivet.

Early adopters, including companies such as Webflow and Quizlet using the platform helped predict AI-related issues before they affected users. According to Quizlet, by implementing Gentrace’s platform, the company increased testing frequency from two times per month to more than 20 times per week, significantly improving the speed of iteration with testing.

“Gentrace was the right product for us because it allowed us to implement our custom evaluations, which was crucial for our unique use cases,” said Madeline Gilbert, a staff machine learning engineer at Quizlet.

As an education technology company providing study tools for students and teachers, Quizlet uses generative AI on unstructured notes and materials to create study tools. According to Gilbert, even minor changes, such as a comma in a prompt, could significantly change the predictability of models. Gentrace’s solution allowed quality assurance teams and subject matter experts to evaluate and test quickly after any modification.

“It’s dramatically improved our ability to predict the impact of even small changes in our LLM implementations,” said Gilbert.

The company said it would use the new funding to expand its engineering, product and marketing efforts to meet the growing demand for AI development tools. The team also plans to develop the Experiments tool to provide greater access to generative AI testing to a greater variety of team makeups, with future plans including guided optimization.

Images: Pixabay, Gentrace