Artificial Intelligence (AI) is the undeniable belle of the ball in today’s world. One company after another is running toward that bright light, hoping it’s the answer to bigger profits, faster service, and more customer satisfaction.
AI thinks highly of itself, too. When ConsumerAffairs asked ChatGPT what consumer-type things it could help with, it said it could assist with several consumer categories such as product recommendations, health information, and basic financial advice.
But, is a machine all it’s cracked up to be when it comes to informing the consumer? ConsumerAffairs decided to find out how far and how deep it can go in and still get things right versus what a basic web search can pull off.
We asked experts from those categories to put ChatGPT to the test and here’s what we found.
Product recommendations
Ask ChatGPT how much boning up it’s done on consumer products and it’ll tell you it’s "read" nearly every product recommendation on the internet. But, Ben Steele, an early adopter of generative AI tools and lead content creator at The Big Phone Store, says when ChatGPT is asked for a list of recommendations, it’s mostly copying what it’s seen online.
“The dark side of this is that many product summaries available online are paid advertising. Articles like ‘The 5 Best Phones For Outdoorsy Folk’ will often be paid for by affiliate links -- and these will usually be the articles that ChatGPT is copying information from,” Steele told ConsumerAffairs.
“AI’s main benefit is that it’s interactive, and that’s great for products you don’t know anything about. If I’m looking for personalized gift ideas, for instance, I usually have a lot of follow-up questions. The catch is that once I’ve got a list of recommended products from the AI, it’s only a ‘starting point’ – a term that ChatGPT actually used in responding to a product query. I still have to research each recommendation myself because even chatbots that can search the internet, like Bing, will often get confused.”
ChatGPT, despite its limitations, would be Ben’s choice over a basic web search. “Its answers may be biased and unreliable, but so are a lot of search results. What really cinches the deal for me is that ChatGPT often reminds you of its own shortcomings."
Healthcare information
For years, consumers have called on “Dr. Google” to find out what ails them. Can AI do better?
“AI can either be a great way for consumers to quickly and accurately obtain answers to health-related questions or a hallucinating nightmare, opening organizations up to potential legal problems as well as high customer churn rates,” Ashu Dubey, co-founder & CEO at Gleen, told us.
So why shouldn’t we just simply “Google” our health questions, then, like we’ve been doing for years?
“Theoretically it would produce the same information as an AI prompt to a chatbot, but that isn’t really the case. You can liken search results to shopping at Target or Wal-Mart. When you go to the store looking for something, you walk down the aisle looking for a specific product. However, the shelves have various products, with different prices, packaging and features,” Dubey said.
Who wins? Neither web searches nor AI. Instead, Dubey says if you’re seeking answers to questions in healthcare, using a specific tool from your healthcare provider would be a better bet.
“These are custom tools built with knowledge layers that will filter answers properly, keeping false or inaccurate information out of the response. As long as the provider is controlling the inputs of information that train the AI, then the outputs to questions will be more accurate and relevant to your specific question,” he said.
AI for healthcare isn’t not quite there, yet, but it’s headed in the right direction. “With even more advancements, healthcare providers will be able to personalize their AI tools and have patients receive 100% personalized information and responses to their questions,” he said.
Financial planning
Since the information that AI platforms absorb and use to answer consumer questions comes from extracting data from the internet, its responses should be close to what data is already available in a basic web search, right?
Like others, Ryan Doser, vice president of Inbound Marketing at Empathy First Media, suggests that AI platforms like ChatGPT are best for general advice and get bonus points for allowing the consumer to bypass paid search ads and website biases.
But trying to get anything as good as a certified financial planner is like getting blood from a turnip.
“For example, when asking ChatGPT 'Is it a good time to buy a house?' you will likely receive an output with general home buying advice,” Doser told ConsumerAffairs.
What AI really gets wrong is what Doser calls “recency.”
“Without real-time web browsing, AI tools are not sophisticated enough to provide legitimate answers for financial advice,” he noted.
“Markets and interest rates always fluctuate so a recency element is needed to accurately answer the question of ‘Is it a good time to buy a house?’”
The final analysis
If you think of ChatGPT as the sexier version of Wikipedia, you’re probably right. It knows who Taylor Swift has dated and when the Jets last won the Super Bowl, but as far as getting consumer questions, it – like some experts vented – can give a nice overview of the situation. But, getting specific questions answered comes with a lot of “howevers.”
At the end of the day, even ChatGPT was honest about its strengths and weaknesses, when it responded to ConsumerAffairs' question about whether AI or web searches were best.
“In the final analysis, a combination of both AI platforms and web searches can often be the most effective approach. You can use AI for in-depth, custom analysis of specific data sets or to automate repetitive tasks, while web searches can provide a broader view of consumer sentiment and trends,” the platform confessed.
“Ultimately, the choice depends on your resources, goals, and the specific nature of the consumer information you seek. It's common for businesses and researchers to leverage both methods to complement each other and gain a more comprehensive understanding of consumer behavior and preferences.”