Google launched a limited beta test of its AI chatbot, Bard, last month. Since then, employees of some of the IT giant’s major contractors have been involved in testing the neural network, also checking the correctness of the answers given to them. It turns out that test-takers often don’t have enough time to check Bard’s answers, so they have to guess randomly.
Appen employees, who support Google in testing the Bard algorithm, told media representatives, on condition of anonymity, that they did not have enough time to meaningfully evaluate the AI bot’s responses. The instructions they received said that during the testing process it was necessary to prepare a request for the chatbot and after receiving two answers, choose the most coherent and correct answer. You also have the option to add a comment to explain why a particular option was chosen.
Testers point out that each request takes several minutes to process, but sometimes it’s reported as low as 60 seconds. It is often very difficult to properly assess the chatbot’s response at such a time, especially when the query relates to an unfamiliar topic. Payment for this work is calculated based on the time allotted for each task. In order not to lose money, testers must complete tasks within the given time, even in cases where they are not sure which of the answers suggested by the AI algorithm is more correct. Simply put, in order not to lose money, test-takers have to perform tasks at random, since they do not have enough time to qualitatively check Bard’s answers.
The source notes that Google contractor employees are increasingly demanding better working conditions. Back in February, participants in the Bard system beta test submitted a petition to Google management demanding higher wages. Appen employees testing Bard are reportedly making between $14 and $14.50 an hour.