> ## Content Index > Fetch the complete content index at: https://serpapi.com/blog/llms.txt > Use this file to discover other available public pages before exploring further. # OpenAI GPT-4 vs Groq Mixtral-8x7b - URL: https://serpapi.com/blog/openai-gpt-4-vs-groq-mistral/ - Published: 2024-03-22T08:43:02.000Z - Updated: 2024-03-22T16:03:40.000Z - Author: Terry Tan - Tags: LLM, gpt-4, mistral, Benchmarks A few months ago, we ran a benchmark on a [traditional parser and Mistral 7B (Open Source LLM)](https://serpapi.com/blog/traditional-parsing-vs-mistral-7b/). The quality of the parsed result from Mistral 7B is quite impressive given it is only 7B parameters. One thing we weren't satisfied with is the processing time. Recently, I stumbled upon [Groq](https://wow.groq.com/why-groq/) who set the mission to revolutionize inference speed. They developed a chip for inference and they called it the Language Processing Unit (LPU). I have tested it and it is really impressive. I don't understand the technology of a chip but like CPU and GPU, I believe it will get faster and hopefully, we can get the inference speed down to 1 second consistently. Given the speed of inference has been brought down, I am interested to see if the quality is on par with OpenAI GPT-4 in parsing HTML. Like the previous article, we will do a similar benchmark. ## The Comparison ### Query: Dentist ![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/03/Screenshot-2024-03-22-at-2.40.02-PM-copy.png) Query: Dentist (1st Result) | GPT-4 | Mixtral 8x7B | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | { "title": "Dental Resource Management", "type": "Dentist", "phone": "(512) 462-2959", "address": "Austin, TX", "years\_in\_business": null, "hours": "Closed ⋅ Opens 6 AM", "rating": null, "reviews": "No reviews", "price": null, "description": null, "service\_options": {}, "is\_operating": false } Processed in 6.28s | { "title": "Dental Resource Management", "type": "Dentist", "phone": "(512) 462-2959", "address": "Austin, TX", "years\_in\_business": null, "hours": "Opens 6 AM", "rating": null, "reviews": "No reviews", "price": null, "description": null, "service\_options": { "in store shopping": null, "delivery": null }, "is\_operating": "Closed" } Processed in 0.91s | Mixtral almost nailed it, the only thing missing is the `Closed` in the `hours`. However, the inference time is only `0.91` seconds, impressive. --- ![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/03/Screenshot-2024-03-22-at-2.40.02-PM.png) Query: Dentist (2nd Result) | GPT-4 | Mixtral 8x7B | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | { "title": "Dr. Brendan Loehr", "type": "Dentist", "phone": "(512) 713-1099", "address": "201 W 5th St", "years\_in\_business": null, "hours": null, "rating": null, "reviews": "0", "price": null, "description": null, "service\_options": {}, "is\_operating": true } Processed in 5.87s | { "title": "Dr. Brendan Loehr", "type": "Dentist", "phone": "(512) 713-1099", "address": "201 W 5th St", "years\_in\_business": null, "hours": null, "rating": null, "reviews": "No reviews", "price": null, "description": null, "service\_options": { "in store shopping": false, "delivery": false }, "is\_operating": true } Processed in 0.86s | Mixtral scores perfectly and completes the task in only `0.86` seconds. --- ![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/03/Screenshot-2024-03-22-at-2.40.02-PM-copy-2.png) Query: Dentist (3rd Result) | GPT-4 | Mixtral 8x7B | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | { "title": "Tejas Patel, DDS", "type": "Cosmetic dentist", "phone": "(512) 476-2336", "address": "221 W 6th St #940, Austin, TX 78701", "years\_in\_business": null, "hours": "Closed ⋅ Opens 8 AM Mon", "rating": 4.9, "reviews": 275, "price": null, "description": "Best cosmetic Dentist in Austin.", "service\_options": null, "is\_operating": false } Processed in 9.82s | { "title": "Tejas Patel, DDS", "type": "Cosmetic dentist", "phone": "(512) 476-2336", "address": "221 W 6th St #940, Austin, TX 78701", "years\_in\_business": null, "hours": "Closed", "rating": 4.9, "reviews": 275, "price": null, "description": "Best cosmetic Dentist in Austin.", "service\_options": { "in store shopping": null, "delivery": null }, "is\_operating": false } Processed in 1.06s | Mixtral made a mistake in the `hours`, the opening time is missing this time. --- ### Query: Mexican Restaurant ![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/03/Screenshot-2024-03-22-at-2.39.53-PM.png) Query: Mexican Restaurant (1st Result) | GPT-4 | Mixtral 8x7B | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | { "title": "Tacos provincia", "type": "Mexican", "phone": "Not Available", "address": "658 Red River St", "years\_in\_business": "Not Available", "hours": "Not Available", "rating": "5.0", "reviews": "12", "price": "$1–10", "description": "Not Available", "service\_options": { "Dine-in": true, "Curbside pickup": true }, "is\_operating": "Not Available" } Processed in 8.75s | { "title": "Tacos provincia", "type": "Mexican", "phone": "N/A", "address": "658 Red River St", "years\_in\_business": "N/A", "hours": "N/A", "rating": 5.0, "reviews": 12, "price": "$1–10", "description": "N/A", "service\_options": { "Dine-in": true, "Curbside pickup": true }, "is\_operating": "N/A" } Processed in 1.55s | 👍 --- ![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/03/Screenshot-2024-03-22-at-2.39.53-PM-copy.png) Query: Mexican Restaurant (2nd Result) | GPT-4 | Mixtral 8x7B | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | { "title": "Velvet Taco", "type": "Mexican", "phone": "N/A", "address": "522 Congress Ave. Suite 100", "years\_in\_business": "N/A", "hours": "Closed ⋅ Opens 11 AM", "rating": 4.6, "reviews": 1500, "price": "$10–20", "description": "N/A", "service\_options": { "Dine-in": true, "Takeout": true, "No-contact delivery": true }, "is\_operating": false } Processed in 10.70s | { "title": "Velvet Taco", "type": "Mexican", "phone": null, "address": "522 Congress Ave. Suite 100", "years\_in\_business": null, "hours": "Opens 11 AM", "rating": 4.6, "reviews": 1500, "price": "$10–20", "description": null, "service\_options": { "Dine-in": true, "Takeout": true, "No-contact delivery": true }, "is\_operating": false } Processed in 1.02s | The only difference is in the `hours` where `Closed` is left out. However, it does interpret the `is_operating` correctly. --- ![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/03/Screenshot-2024-03-22-at-2.39.53-PM-copy-2.png) Query: Mexican Restaurant (3rd Result) | GPT-4 | Mixtral 8x7B | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | { "title": "Iron Cactus Mexican Restaurant and Margarita Bar", "type": "Mexican", "phone": "Not Available", "address": "606 Trinity St", "years\_in\_business": "Not Available", "hours": "Closed ⋅ Opens 11 AM", "rating": "4.1", "reviews": "3.7K", "price": "$20–30", "description": "Buzzy spot with happy hour & tequilas", "service\_options": { "Not Available": true }, "is\_operating": false } Processed in 8.36s | { "title": "Iron Cactus Mexican Restaurant and Margarita Bar", "type": "Mexican Restaurant", "phone": "Not Found", "address": "606 Trinity St", "years\_in\_business": "Not Found", "hours": "Closed - Opens 11 AM", "rating": 4.1, "reviews": 3.7, "price": "$20–30", "description": "Buzzy spot with happy hour & tequilas", "service\_options": { "in\_store\_shopping": false, "delivery": false, "curbside\_pickup": false }, "is\_operating": false } Processed in 1.22s | `3.7` should be `3.7k` or `3700` in Mixtral, a critical mistake. --- ### Query: Yoga Studio ![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/03/Screenshot-2024-03-22-at-2.39.32-PM.png) Query: Yoga Studio (1st Result) | GPT-4 | Mixtral 8x7B | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | { "title": "YTX Yoga, Strength, & HIIT Training", "type": "Yoga studio", "phone": "(512) 502-5183", "address": "206 E 4th St, Austin, TX 78701", "years\_in\_business": "10+ years", "hours": "Closed ⋅ Opens 6:30 AM", "rating": "4.7", "reviews": "297", "price": null, "description": null, "service\_options": { "Onsite services": true, "Online classes": true }, "is\_operating": false } Processed in 13.72s | { "title": "YTX Yoga, Strength, & HIIT Training", "type": "Yoga studio", "phone": "(512) 502-5183", "address": "206 E 4th St", "years\_in\_business": "10+", "hours": "Closed ⋅ Opens 6:30 AM", "rating": "4.7", "reviews": "297", "description": null, "service\_options": { "Onsite services": true, "Online classes": true }, "is\_operating": false } Processed in 1.71s | 👍 --- ![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/03/Screenshot-2024-03-22-at-2.39.32-PM-copy.png) Query: Yoga Studio (2nd Result) | GPT-4 | Mixtral 8x7B | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | { "title": "Practice Yoga Austin", "type": "Yoga studio", "phone": "(512) 730-1638", "address": "1103 E 6th St, Austin, TX", "years\_in\_business": "10+ years", "hours": "Closed ⋅ Opens 7 AM", "rating": "5.0", "reviews": "97", "price": null, "description": "\\"Practice yoga studio is very unique.\\"", "service\_options": {}, "is\_operating": false } Processed in 10.52s | { "title": "Practice Yoga Austin", "type": "Yoga studio", "phone": "(512) 730-1638", "address": "1103 E 6th St", "years\_in\_business": "10+", "hours": "Closed - Opens 7 AM", "rating": "5.0", "reviews": "97", "description": "Practice yoga studio is very unique.", "service\_options": { "in\_store\_shopping": false, "delivery": false, "curbside\_pickup": false }, "is\_operating": false } Processed in 1.34s | 👍 --- ![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/03/Screenshot-2024-03-22-at-2.39.32-PM-copy-2.png) Query: Yoga Studio (3rd Result) | GPT-4 | Mixtral 8x7B | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | { "title": "Sanctuary Studio & Loft Rentals", "type": "Yoga studio", "phone": "(512) 337-2213", "address": "1006 S 8th St, Austin, TX 78704", "years\_in\_business": "10+ years", "hours": "Closed ⋅ Opens 9 AM", "rating": "4.8", "reviews": "55", "description": "Easygoing non-profit yoga center", "service\_options": {}, "is\_operating": false } Processed in 9.40s | { "title": "Sanctuary Studio & Loft Rentals", "type": "Yoga studio", "phone": "(512) 337-2213", "address": "1006 S 8th St", "years\_in\_business": "10+ years", "hours": "Closed ⋅ Opens 9 AM", "rating": "4.8", "reviews": "55", "description": "Easygoing non-profit yoga center", "service\_options": { "in\_store\_shopping": false, "delivery": false, "curbside\_pickup": false }, "is\_operating": false } Processed in 1.74s | 👍 ## Thoughts GPT-4 scored a perfect score in parsing the HTML, however, the inference time isn't ideal. On the other hand, Mixtral 8x7b runs on Groq does perform much faster; for some of the results it even goes under 1 second. I think it could be considered for use in production, though it has made some mistakes in the results, this at least can be resolved by enhancing the prompt. I am looking forward to experimenting more with Open Source LLMs. Follow us to get our up-to-date sharing. --- Join us on [X](https://twitter.com/serp%5Fapi) | [YouTube](https://www.youtube.com/channel/UCUgIHlYBOD3yA3yDIRhg%5Fmg) Add a [Feature Request](https://github.com/serpapi/public-roadmap/issues)💫 or a [Bug](https://github.com/serpapi/public-roadmap/issues)🐞