HumanLoop: The AI startup helping you make your own LLMs

In partnership with

(5 minutes)

Hi 👋, this is the Today in AI Newsletter: The weekly newsletter bringing you one step closer to building your own startup.

We analyze a cool, industry-shaping AI startup every week, with a full breakdown of what they do, how they make money, how much they’ve raised, and the opportunity ahead. 

Let’s get to the good stuff in this email: 

• This startup is building the LLM development and evals platform for enterprises

• Some of their customers include Gusto, Vanta and Duolingo

• They’ve raised over $7 Million from VCs like Index Ventures and Y Combinator

So what’s the startup and who are the founders behind it? Here’s the story of HumanLoop 📈

HumanLoop was founded by Raza Habib, Jordan Burgess and Peter Hayes in March 2020 with the mission to help businesses build reliable AI applications.

HumanLoop essentially helps companies that want to build products with Large Language Models (LLMs) to develop and evaluate these products. 🧑‍💻

Unlike software engineering, when you’re dealing with deterministic code and you can predict how it will behave; using AI is non-deterministic. It’ll produce different outputs depending on very minor changes. 🤏

People using HumanLoop are product teams building AI-focused applications, collaborating with engineers to get it into production.

There are 2 pillars to their current product 🏛️:

1. Everything related to prompt-engineering, versioning and management

2. Evaluation ✍️

Product teams use HumanLoop as their development environment where in that UI, they can collaborate with engineers to develop prompts and iterate.

Here’s a quick example of DuoLingo 🦜 uses HumanLoop to build their LLM applications:

A lot of prompt engineering is required when building AI apps like AI assistants or chatbots that users can practice languages with.

The best people to do the prompt engineering for applications like this are the domain experts: Linguists themselves. 🗣️

HumanLoop allows linguists to collaborate with engineers to build prompts and measure their performance. 📈

Backstory 👀

As an undergrad, Raza studied Physics at Cambridge, and was always super curious and a little bit nerdy. 🤓

His first ever job was right out of university when he joined a VC-backed startup called JustPark - a P2P car parking rental startup. 🧑‍🤝‍🧑Raza joined when they were just 6 people.

By the time he left, they’d grown to over 1 million users and 45 people on the team. It had been a really exciting experience scaling a company, but it wasn’t deeply technical.

Raza came from a technical background and really wanted to work on the frontier of what was possible. 🌌

He’d also been increasingly convinced that AI was gonna be one of the biggest technological achievements ever made. 🐣The year was 2015…

The youngster ended up going back to university, this time at UCL, to do both a Masters and PhD in Machine Learning. 

In 2015, Deep Learning began to take off. 🤖 It was a time where you’d get excited if you produced an LLM that could open and close a bracket. It felt like science fiction…

According to Raza, AI has been the most interesting intellectual problem of our time. 🧠

When he finished his PhD, he worked at Google AI for a bit, but didn’t really like it. He enjoyed tighter environments where connection to product was a lot closer.

Being part of a smaller team with tighter deadlines, pushing towards something gives you a sense that you can achieve something truly amazing. 💫

Raza concluded that he wanted to start a company and made a list of ideas he was most excited about, and a list of the smartest people he knew.

One of these people high up the list was a guy named Jordan, who had an incredible eye for detail and taste in product. 👁️

As a 15 year-old he’d built and sold his first website. By 19, he studied Machine Learning at Cambridge, doing an exchange semester at MIT. There, he was enthused by the startup ecosystem.

The 2 met up at UCL where there was a room where all 4 walls are whiteboards (Floor to ceiling!). 

The founders started writing and covered every single part of every wall with different iterations of their idea. 💡

The Hustle 🤑

Their idea was to help people annotate data to train LLMs - no one was doing this at the time and AI was growing exponentially.

They got into Y Combinator in the Summer 2020 batch - a force to help them push the product out and build faster. 💨

This worked on a small scale and Raza says they never truly found “product-market fit” with this first iteration of the idea.

By 2022, the young hustlers realised that as LLMs improved, data wouldn’t need to be annotated anymore. They realised there was going to be a paradigm shift from hand labelling data towards prompt engineering… 🌟

They started building and created an internal hackathon when they realized just how capable these models had become, especially with the release of OpenAI’s model - ChatGPT.

They talked to current customers who all had the same pain point of “How do I know how this system is behaving when it’s non-deterministic? How do I go and improve it?”. ⁉️

They gave themselves a sales experiment with a tight deadline. 😵

“If we can get 10 paying customers in 2 weeks, we know there’s a really strong pull in this direction and the timing’s right”

After only 2 days, HumanLoop officially had 10 paying customers. 🤯

They’d never felt traction like this before and made the decision to pivot to going all in on this.

Stats 📊

Raza appeared on a podcast in 2023 with Ali Rowghani (an MD at Y Combinator), to talk about the potential of Generative AI and what he was building. This blew up, and the video is now approaching 500,000 views. 🚀

In 2023, HumanLoop was also named one of the 21 generative AI startups to watch, according to Sifted. 📰

Today, HumanLoop works with customers like DuoLingo, Gusto, Vanta, Apollo.io and Fundrise to build LLM applications for them. 🛃

Since 2022, they’ve grown over 60x in usage and have around 14 employees.

In the last year they have:

• Had over 300 production deployments culminating in 50 product releases

• Added support for over 50 new LLM models 🤖

• Now processing millions of LLM logs daily across thousands of AI products deployed in production 💾

They’ve now raised about $7M in total from investors like Index Ventures, Y Combinator and the UCL Technology Fund. 💸

This includes their last public raise, a $2.6M Seed led by Index Ventures in July 2022. 💰

Before you go 👋

Save 13 Hours Weekly of Podcast Pitching with PodPitch.com

Stop wasting time – 2025 is going by fast. If you finally want to be a regular podcast guest in your industry, PodPitch.com will make it happen. Even the beehiiv team uses it!

Imagine snapping your fingers & getting booked on the exact podcasts your customers are already listening to…

With PodPitch.com, it takes 60 secs to start emailing tons of podcast hosts to pitch YOU as the perfect next guest.

• Sync your email address

• Load in your brand info

• Click "go"

Now, you've just automated thousands of personalized emails pitching YOU as the PERFECT next podcast guest. Sit back and relax as you watch the emails send out from your email address.

Big brands like Feastables, Jack Links, and hundreds more are already using PodPitch.com instead of expensive PR agencies.

PodPitch.com is so confident in their tech that they'll give you a FREE Starbucks gift card if PodPitch.com isn't the most impressive 20 minute demo you've ever seen.

Ready to make 2025 your year?

Get your FREE personalized demo by clicking here now!

Thank you for reading.

Know someone who'd love this newsletter? Share the love.🌟