This AI Software Is ‘Coaching’ Customer Service Workers. Soon It Could Be Bossing You Around, Too

I’m on the phone with a startup executive roleplaying as a frustrated customer, and a note along with a speedometer icon appears on my computer screen: Speaking slowly — You are speaking slower than usual. Try increasing your speaking pace. I speed up, and the message disappears, only to be replaced with another: Continuous Speaking — Finish your thought. Ask an open-ended question. Later, as the exec struggles to find the date of a made-up transaction, the software I’m using detects the strained note in his voice, and again decides I should intervene.