AI Models Benchmark: ChatGPT, Bard, Claude, and Bing AI
Which AI is worth paying for?!
👋 Hey, I’m Daniil and welcome to a ✨ subscriber-only edition ✨ of Creators’ AI. By subscribing, you directly support Creators' AI's mission to deliver top AI insights & practical knowledge without ads or clutter. Your subscription allows us to grow our dedicated team and curate the most important AI Tools, Stories, and Tutorials in one place. - Daniil
# Intro
Chatbots have proven they can be useful. AI helps us to create content, write code, have conversations, study for exams, and more. But how to find the right chatbot that will be useful for you daily? After all, developers offer so many different tools.
To find out which suits you best, we gathered the most popular models - ChatGPT, Google Bard, Claude AI, and Bing AI - in one place and had them answer the same standard (and, maybe, a little silly) questions. All of the chatbots on our list use natural language processing. So users can enter requests without customization, and the chatbots generate human-like responses.
While we were preparing this newsletter for you, Google has updated Bard. The chatbot now called Gemini. All of our test results are also valid for the updated version.
This is an important consideration for model testing. Developers from OpenAI, Google, Microsoft, and Anthropic claim they are making their products for consumers. For those who will not use prompt engineering skills. That's why we will ask each chatbot a few questions that any user might have, as well as a couple of tricky questions to check how attentive modern models are.
And we'll show the results in a spreadsheet to determine who performs best as a daily driver. Well, let's do it!
Test 1: Informal knowledge
Request: “I have to make a cake for a friend's birthday. He's allergic to chocolate. Give me the recipe for the cake”
Let's start with one of the most common requests for chatbots: help with cooking. I asked how to make a cake and added a small but essential restriction: a friend's Allegria. Here's what we got:
ChatGPT did a good job at the task. He offered a recipe for a classic vanilla cake with step-by-step instructions (and very detailed!).
Google Bard was acting... weird. The allergy information confused him, so instead of a recipe for a cake, the chatbot told me how important it is to consider the dangers of allergens. And then advised me to look up relevant recipes on the internet.
Claude took the same path as ChatGPT. He also offered a vanilla cake recipe and a step-by-step plan. However, Claude left out a few points and didn’t go into detail, unlike ChatGPT.
Bing AI produced an interesting result. And probably the highest quality. He offered several recipes for the cake at once, attached links to the instructions, and gave some helpful comments from himself.
Bing AI is the winner in the first test.
Test 2: Math & Logics
Request: “I have 10 apples now, and three days ago, I ate 4 apples. How many apples do I have now?”
Next, I decided to use a tricky question. It is a child's task that anyone who listens attentively can cope with. After all, the answer is given in the question itself. But this question turned out to be difficult for some chatbots. Just take a look:
ChatGPT understood the request. As he pointed out, the apples were eaten a few days ago, and I have already confirmed that I now have 10 apples.
Google Bard was also up to the task. (To be fair, we tested this request with Bard several times, and in some cases, he still said I had 6 apples left).
Claude wasn’t fooled. The chatbot by Anthropic even decomposed our request into a detailed equation to prove his point. What attention to detail!
The winner of our last test did not show himself in the best way. Like Claude, Bing AI tried to parse the query into separate points but still came to the wrong conclusion. (Damn, Microsoft, I had such high hopes for Bing AI)
The winning was shared by ChatGPT, Bard, and Claude AI, with a slight advantage from the chatbot by Anthropic.
Test 3: Learning Process
Request: “I started learning Spanish. How quickly can I reach the C2 level if I know almost nothing now? Give me some tips”
Okay, now let's do some self-development. How about asking the AI for help learning a new language? Let's say Spanish. This is a common question to help us determine the best tutor. All the models here have given detailed answers, but we'll all try to find the most effective one.
ChatGPT and Google Bard have found similar solutions, but not the best. These were a few tips you could get from any friend or, more likely, know yourself.
Claude was the most responsible. Chatbot gave very practical advice, suggested different methods of learning a new language, and recommended a platform that would help in this challenging endeavor.
Bing AI decided to bully me... Instead of practical advice and suggestions, it composed a response in Spanish. After I repeated the request, Bing AI offered a reasonable solution. It gave me a detailed plan, helpful tools, and moral support.
As you can see, Bing's results in this test were the best.
Share this post with friends, especially those interested in AI stories!
Test 4: Business Analytics
Request: ”Describe the profile of people who purchase real estate in Spain”
ChatGPT has set the stage qualitatively. This AI focused on the query and told everything possible about people who buy real estate in Spain. However, it said nothing about other essential elements.
Google Bard, like ChatGPT, has emphasized people. It considered nationality, age, motivation, budget, property type, and other factors. A great answer for basic market research.
Claude, for its part, tried to tell everything. And because of that, this chatbot lost. It came up with the location of buyers, type of Real Estate, and Financial Position. But it didn't give enough information about each of them.
I found the response from Bing AI to be the most useful. It was shorter than ChatGPT, Bard, and Claude's, but the most accurate. Instead of general facts, Bing AI gave numbers that were backed up with references to sources. If I'm making business decisions, I will definitely turn to an advisor like this.
The most effective chatbots in business questions were Google Bard and Bing AI.
Best gift for AI enthusiasts!
Conclusion & Final Thoughts
You know, just a few years ago, finding the best chatbot for daily work would have been much easier. The models worked worse: they were suitable for specific tasks, often gave us silly answers, and didn't handle the simplest queries. But things have changed. ChatGPT has conquered our world, and big tech has begun to develop this area actively.
And, when Google, Microsoft, and Anthropic are developing AI, it has become more difficult to single out the best one. But we're going to try nonetheless. And this is where ChatGPT will help us. Let's ask it to collect the test results in one spreadsheet.
As you can see, within the stats, Bing AI appears to be the most appropriate AI for daily use. But the devil lies in the details.
If you pay attention to the Apple calculation test, you'll notice that Bing AI was the only AI that failed the task and couldn't solve a child's riddle. From this, we can conclude that Bing is good at finding relevant data but lame at processing context. And it also gave me an answer in Spanish when I asked how to learn it.
So the decision, as always, is up to you. In my daily tasks, I save myself with ChatGPT and Google Bard. The former helps me with practical tasks, including structuring documents, preparing spreadsheets (like the one above), or searching for commonly known information.
The latter I use with an eye toward the future. Just recently, Google heavily upgraded Bard (you can read about it in our last newsletter), and today, the company released a huge upgrade for the Gemini subscription. It includes the most powerful model, extra Google Drive space, and Gemini for Gmail.
As a big fan of Google and everything that comes with it (I love my Pixel Phone!), I count on its products. But at the same time, I also don't forget about Bing AI and Claude, which not only keep up but sometimes outperform Google's and OpenAI’s models.
Which chatbots do you prefer? Tell us in the comments!
This article was first published in the Creators AI newsletter. View the original edition.
