The Democrat seeking a third term in office made the awkward crack Sunday at Mount Neboh Baptist Church in Harlem.
Noting he was Catholic, Cuomo said Catholics and Baptists share many of the same beliefs but Catholics "do it without the rhythm."
He added: "But we try. We are not as without rhythm as our Jewish brothers and sisters."
Cuomo then singled out Hank Sheinkopf, a Jewish Democratic political consultant who was moving to the music in a front pew. Cuomo referred to Sheinkopf's movements as "ugly."
A Cuomo spokeswoman later told the New York Post the governor was poking fun at himself and a longtime friend.