Join our Waitlist for Expert Advice!

Elon Musk's Newest AI Chatbot Outperformed ChatGPT in One Key Area Musk's AI startup announced an upgrade to its Grok chatbot on Thursday.

By Sherin Shibu

Key Takeaways

  • Elon Musk's xAI company is upgrading its Grok AI chatbot.
  • The new model outperformed OpenAI's AI model on one key HumanEval test.
  • Musk stated in a Friday social media post that Grok 1.5 should be available on X, formerly Twitter, by next week.

Nearly two weeks after Elon Musk's xAI startup opened up the AI model behind Grok to the public, its AI chatbot is set to get an upgrade.

The company announced Grok-1.5 on Thursday and claimed that its latest model can understand longer documents, handle more complex prompts, and perform more advanced reasoning.

While Grok-1.5 appears to be a step up from the original 1.0 with improvements in coding and math skills, its announcement post shows that it still lags behind Google's Gemini Pro 1.5 AI, OpenAI's GPT-4, and Anthropic's Claude 3 Opus in some benchmark tests, while outperforming OpenAI on one key HumanEval test.

Related: Meet Grok: Elon Musk Unveils 'Spicy' AI Chatbot Riddled With 'Sarcasm' and 'Humor'

Grok-1.5 scored higher than GPT-4 on the HumanEval benchmark, which consists of 164 challenging programming problems not included in the AI model's training data. GPT-4 had a score of 67% and Gemini Pro 1.5 scored 71.9%, while Grok-1.5 received 74.1%.

Elon Musk's xAI company is set to release a new version of the Grok AI chatbot, a ChatGPT competitor. Photo by Jaap Arriens/NurPhoto via Getty Images.

With a score of 81.3% on the MMLU test, which covers knowledge of 57 subjects from an elementary to an advanced level, Grok-1.5 performed close to Google Gemini's score (83.7%).

It also scored close to GPT-4's score of 52.9% with a score of 50.6% on the MATH test, a benchmark that covers grade school to high school math competition problems.

Related: Elon Musk Sues ChatGPT-Maker OpenAI, Accuses the Company of Working to 'Maximize Profits For Microsoft, Rather Than For the Benefit of Humanity'

Musk stated in a Friday social media post that Grok 1.5 should be available on X, formerly Twitter, by next week.

The X owner has high expectations for the next generation of Grok, writing that the next step after Grok-1.5 will outperform the AI currently available "on all metrics." Grok 2 is "in training now," he wrote in the post.

Grok AI is currently only available to those with a $16 a month or higher Premium+ subscription on X.

Musk sued OpenAI, a competitor of xAI, earlier this month and asked for a court ruling that would force OpenAI to make the research and technology behind its AI public.

Sherin Shibu

Entrepreneur Staff

News Reporter

Sherin Shibu is a business news reporter at Entrepreneur.com. She previously worked for PCMag, Business Insider, The Messenger, and ZDNET as a reporter and copyeditor. Her areas of coverage encompass tech, business, strategy, finance, and even space. She is a Columbia University graduate.

Want to be an Entrepreneur Leadership Network contributor? Apply now to join.

Leadership

5 Ways to Get Real Freedom as a Business Owner (Without Surrendering Control)

Use these five strategies to attain true freedom and shift your focus from daily operations to long-term business growth and personal time.

Business News

A Carpenter of 56 Years Is Retiring After He Found $20 on Ground and Turned It Into a $1 Million Lottery Win

The winner, Jerry Hicks, told the North Carolina lottery that the gas station was out of the tickets he usually buys.

Business News

You Have One Month Left to Buy a House, According to Barbara Corcoran. Here's Why.

"If you are planning on waiting a year and seeing where interest rates go, you are out of your mind," Corcoran said.

Business News

A Wells Fargo Worker Was Discovered 4 Days After Dying at Her Desk. Her Cause of Death Was Just Revealed.

Medical examiners have released the cause of death of Denise Prudhomme, who was found dead at her desk in Wells Fargo's Tempe, Ariz. office.

Culture

How to Ensure Ethical and Professional Business Practices with Your PR Firm

Choosing a PR firm is a personal decision. But it's also one that should be driven by serious considerations and fair assessments of professionalism. Here's some advice from a PR pro on how to make the best choice for your business.

Business News

Meta Fires Employee Making $400,000 Per Year Over a $25 Meal Voucher Issue

Other staff members were fired for the same reason, per a new report.