So far in this series I’ve taken a look at image generation with Adobe’s Firefly, audio generation with tools like MusicLM and Stable audio, and voice generation via ElevenLabs. In this post, I’m going back to where this wave of AI hype began with ChatGPT and text generation.
However, plenty has changed since my early posts about ChatGPT at the end of 2022, and even since more recent posts. In competition with other chatbots like Google’s Bard, we’re now seeing successive releases of new features including real-time internet connection.
ChatGPT has actually trialled internet connection before, releasing the capability via a Microsoft Bing plugin. That first time around users quickly realised that the plugin was full of vulnerabilities, including one allowing users to bypass paywalled news articles.
This time around they’ve supposedly addressed those issues and the overall quality of the “browse with Bing” feature. I’ll be comparing it to four other internet-connected chatbots: Microsoft’s Bing Chat and Perplexity (both using GPT-3.5 and 4), Google Bard, and HuggingChat which is an open source chatbot built on Meta’s Llama model.
What is chat + search, and how is it different to standard ChatGPT?
When ChatGPT was released in November 2022, it was “disconnected” from the internet. Users could query the chatbot and get responses, but were often greeted with a stock phrase along the lines of “as a large language model trained by OpenAI, I have a knowledge cut-off of September 2021 and am unable to browse the internet.”
The September 2021 cut-off date was due to the training data used to build the model, and though subsequent releases have updated the model with newer data, the model still uses that date as its upper limit. That changes when you add browsing capabilities, however, and the model gains access to real-time internet data. The browsing feature uses Microsoft’s Bing search engine and provides clickable links to source data.
Both the original release of the Bing connection and the current update are only available to ChatGPT Plus subscribers using GPT-4, which might make it unattainable for some people with its price tag of around $30 AUD/month. There are also free chatbots available which have an internet connection, so in this post I’ll compare them and discuss whether it’s worth forking out the cash for the upgraded product from OpenAI.
Using chat + search for creative writing
The first test involves using the following prompt with the various models to gather some real information to use as inspiration for a fictional character:
I’m writing a short story and need to develop some character backstory. Use Bing to find at least three recent articles about people who love the australian rainforest. Try to find some interesting and unique people for inspiration
Prompt used in ChatGPT, Bing Chat, Bard, Perplexity, and HuggingChat
Here’s a short video demonstrating the different models and discussing some of the outcomes:
As you can see, there’s quite a variation in how the different chatbots handle the request. Arguably, Google’s Bard gives the best response to the prompt since it goes into more detail on how the news articles could be used in the context of the creative task. Honestly, I was quite surprised by that as Google’s chatbot has been pretty uninspiring up until now. Of all of the responses, I feel that ChatGPT was the worst. It was both the most simplistic response, and required re-prompting to get what I was actually looking for.
Similar articles were identified across the models. Bard and HuggingChat both use Google search, and ChatGPT and Bing use Microsoft’s search engine. ChatGPT, Bing, and Perplexity are all built on top of the same models, so it’s hardly surprising that their outputs are similar in style and language.
Using chat + search for research
Because of the tendency of GAI chatbots to create inaccurate content, they don’t make particularly good research assistants. Having to manually fact-check every comment that comes out of a chatbot makes it harder to use them than a simple Google search or browsing Wikipedia, and it can be difficult to detect when they’re inventing content.
Adding the search function mitigates this to a degree. First of all, the output of all these models includes links to the websites it has drawn information from, making it easier to fact-check. There’s also the added assurance that the search function is using “reliable” engines like Bing and Google which are typically viewed as trustworthy. However, the search function hasn’t totally removed the possibility of fabricated information, and information that’s available and searchable on the internet isn’t necessarily true, so it’s still not a given that you can trust AI output.
In this example, I’m using the following prompt to do a comparison of generative AI and stock images:
What are some of the limitations of using generative ai for image generation as opposed to paying for stock images?
Prompt used in ChatGPT, Bing, Bard, Perplexity, and HuggingChat
Click the images to enlarge and scroll through to compare the different chatbots.
In my opinion, Microsoft’s Bing Chat provides the best answers and the clearest references. Bard does a reasonable job, and the summary at the end is quite useful. The open source HuggingChat model also does a solid job of finding relevant information. Once again, ChatGPT is the least effective here.
What else can chat + search be used for?
By this point, you’re probably asking what the point of chat plus search is if a traditional search engine does a comparable job. Honestly, I feel the same. I think the current technology is still in its “proof of concept” phase rather than an actually useful product. Undoubtedly, Microsoft and Google will add increasing layers of chatbot-style output to their traditional search. In a way, this means that getting in now and using chatbots with search features will help prepare you for what’s coming next.
Chat + search provides a somewhat more reliable source but still can’t be fully trusted. In the long term, it will have the capacity to quickly summarise and synthesise large amounts of data online, including visuals, PDFs, and other formats.
Conclusion
Would I recommend shelling out the extra cash for ChatGPT Plus? No. Not based on the search functions, and certainly not compared to other free models. I use the Plus subscription as part of my studies and my work, and the “advanced data analysis” (formerly Code Interpreter) feature is interesting enough, but for the average user I’d say save your money and use Bing or Bard for most tasks.
Perplexity is an interesting app that is carving out its own space, and it’s using a social media style “threads” feature to promote community searching. HuggingChat is an interesting example of what the Open Source community can do and in the long term is definitely one to watch as people continue to recoil from the big developers. However, it’s important to recognise that open source models have their own issues and may not have the guardrails or saftey features to bring into a classroom. The version I used was also built on Meta’s Llama model, contentious for a number of reasons including the use of the book3 dataset.
If you enjoyed this article, join the list for more. From October this year I’ll be working on some new courses and online professional learning for generative AI. Join the list for updates:
Leave a Reply