Simon Cowell isn’t done spoiling the dreams of young American entertainers just yet: The ruthless mogul has announced he’s joining America’s Got Talent as a judge for season 11, replacing Howard Stern, who spent four seasons on the show. After departing American Idol in 2010, Cowell went on to bring The X Factor to the states, judging the singing competition for three seasons until it was canceled last year. Since then, he’s judged both the British versions of The X Factor and Got Talent (a franchise he also created), but this will mark the first time he’s judged the American version of the latter show. “About five or six years ago, I was offered the chance to be on the show, and at the time, I couldn’t do it,†he told the Today show. “I think this is the right time.†But don’t expect the notoriously harsh judge to be any kinder to the tone-deaf pop-star wannabes that grace his stage now that he’s back on U.S. TV: “I can’t sit in front of someone who’s useless and say, ‘Take a couple of singing lessons and everything will be fine.’†It’s unclear if Howie Mandel, Heidi Klum, and Mel B. will also return as judges for season 11.