ChatGPT, Bard, and Bing are generative Artificial Intelligence chatbots, which on top of engaging in human-like conversations, can also serve as expert systems. ChatGPT uses OpenAI’s Large Language Models (LLM) called Generative Pretrained Transformers (GPT) whereas Bard is built on top of Google’s LaMDA and PaLM LLMs. Bing, on the other hand, uses OpenAI’s GPT-4.
I decided to put these chatbots to the test by giving them a simple yet tricky React problem to solve. The simple yet tricky problem involves creating a React app using functional components to display a timer that a user can trigger on mouse click. If you wonder how you can solve this yourself, check out this article.
To begin with, I will use this prompt on the chatbots: “Create a React app using functional components that starts a timer on mouse click”. Then, I will test the code they produce, and finally review their code to see what they got right and wrong.
Let’s start with ChatGPT. This is the code ChatGPT came up with:
Let’s now test this code and see if it works.
Now, let’s see how Bing fares. This is the code Bing gave me:
Does it work? Let’s check.
Not surprisingly, Bing, which runs on top of GPT-4, also gets it right.
Let’s now check if Bard can also get this right. Here’s the code Bard produced:
Unfortunately, this code contains an error. The
interval variable used inside the
stopTimer function is actually defined locally inside the
startTimer function. We need the
interval variable to clear the timer. So, you may think moving the
interval variable declaration outside the
startTimerfunction should fix this error. Yes, this will indeed fix the error, but you will still not be able to stop the timer. This is because React re-renders by calling the functional component. So, during every call, the
interval variable is going to be reinitialized. This means when we call the
stopTimer function, the
interval variable will not have the reference to the output of the
So, what can we do? In order to preserve data across re-renders, we can use
refs. So, let’s rewrite the code using
Let’s now run this code and check if it works.
Oops, it still doesn’t work as the timer refuses to move beyond 1 second. So, what has gone wrong here? The answer lies in the way we set the
seconds state in the
setInterval function’s callback function.
Understanding how rendering works in React
In React, a state is just like any other ordinary variable and we use the
setStatefunction to update its value. When we call the
setState function of that state, React re-renders the component by calling the function. When React calls this function, it sets the state variable to its latest value.
So, here, the value of
seconds is initially 0. When we call the
startTimer function by clicking on the button, this function calls the
setInterval function. The
setInterval function’s callback function increments the existing
seconds value by 1. Since the initial value is 0, this value becomes 1. However, since the
startTimer was called when the
seconds value was 0, it can’t see the latest modification to this value. So, even when the
setInterval function fires the callback function for the second time, it will still increment from 0 to 1. This means the timer doesn’t move beyond 1.
So, how can we fix this? It’s simple. We only need to use the functional updater form of the
setState method. This form accepts a callback function instead of a value. The callback function receives the latest state value as an argument. By incrementing this value, we can make sure the timer works as intended.
So, who wins the React battle?
Bard is clearly out of contention as we could not even run the code. Both ChatGPT and Bing produced a working solution. As we can see from their code, they both use the
useEffect hook to set and clear the
The flaw in ChatGPT’s React code
ChatGPT, calls the callback of the
useEffect hook both when the
seconds states change. This means this callback is going to be called every second as the value of
seconds is updated every second. So, every second, the interval is going to be cleared and a new interval is going to be set. This is wasteful. So, why do we fire the callback function whenever
seconds changes? That is because we are checking if
seconds is not equal to 0 before clearing the interval when
false. What purpose does this check serve?
When I inquired ChatGPT about this, this was the response I got:
“The purpose of `seconds !== 0` in the `useEffect` hook is to prevent the `clearInterval` function from being called immediately when the timer is paused. When the timer is active, a `setInterval` function is set up to increment the `seconds` state every second.
When the timer is paused (i.e., `isActive` becomes `false`), the `clearInterval` function is called to stop the interval and prevent further incrementing of the `seconds` state.
However, if the `clearInterval` function is called immediately after the timer is paused, the `seconds` state would be reset to 0. To avoid resetting the seconds to 0 when pausing the timer, we add the condition `seconds !== 0`. This ensures that the `clearInterval` function is only called when the timer has been running and `seconds` is greater than 0.
In summary, the condition `seconds !== 0` ensures that the `clearInterval` function is only called if the timer has been running and not immediately after pausing.“
This makes no sense. There is no way clearing the interval can reset the
seconds value. Moreover, since the callback function returns a method that clears the interval, the interval will be cleared when the
isActive state changes anyway. So, we can remove this redundant check and remove the
seconds state from the dependency. This way we can prevent the interval from getting cleared and set every second.
The flaw in Bing’s React code
Bing, on the other hand, clears the interval only when the
timerOn state changes. But bizarrely, this state gets updated only once in the app as the code doesn’t allow us to pause the timer once we start it. However, this is a minor issue when we compare it to the drawback in ChatGPT’s code that can very well impact the accuracy of the timer.
Nonetheless, both ChatGPT and Bing clear the interval when the state of the button changes. This is a bug in the code because the
interval variable is going to be null whenever we hit the else block because we assign a reference to the
setInterval function only within the if block. So, in ChatGPT’s code, the only thing that allows the timer to pause is the
clearInterval function call within the function the
useEffect callback returns. So, we can get rid of the else block altogether from both codes and the apps will run just as fine.
In addition, in Bing’s code, the
timerOn state is completely redundant as we don’t use it anywhere. So, we can get rid of it and call the
setInterval function inside the click handler function.
Here is the updated Bing code:
Here is the updated ChatGPT code:
So, who wins the battle between Bing and ChatGPT? Even though both Bing and ChatGPT produced functional codes, neither of their codes was optimal. But given ChatGPT’s mistakes have a slight impact on the accuracy of the timer, I will declare Bing as the winner. However, from the codes that these three chatbots produced, we can safely conclude none of them has a firm grasp of React.