The suit alleged the university failed to protect students at Yeshiva University High School for Boys and even promoted one of the rabbis to principal after receiving abuse reports.
A Yeshiva University spokesperson declined to comment, citing a school policy against speaking publicly about litigation.
The lawsuit is one of hundreds that have been filed over child sexual abuse allegations since last week, when New York state opened a one-year window for suits previously barred by the state's statute of limitations.
During a press conference Thursday, three of the alleged victims, flanked by their lawyers, spoke about disturbing behavior they say went on for decades.
"I didn't even understand at the time that this was sexual abuse; I just knew that this guy was putting his hands all over me," said Barry Singer, 61, speaking of one of the rabbis he said kept reaching into the boy's pants, even in school hallways.
The Associated Press doesn't typically identify people who say they are victims of sexual abuse unless they choose to be named.
One of the accused rabbis, Macy Gordon, died recently in Israel. The other, George Finkelstein, has denied the allegations.
Finkelstein was promoted from the school's assistant principal to principal even after some of the boys' parents reported the alleged abuse to school officials, the plaintiffs said. He also eventually moved to Israel, where he worked at Jerusalem's Great Synagogue. Calls to the synagogue rang unanswered Thursday.
Thirty-four of the plaintiffs attempted to sue Yeshiva University for sexual abuse and facilitating sexual abuse in 2013 but the case went nowhere because it was barred by the statute of limitations at the time
Plaintiff David Bressler, 51, said the abuse he suffered while a student in the early '80s led him to abandon his religion that now rekindles memories of the abuse. He now has no contact with his parents and other relatives who are observant Jews. When he married his Jewish wife a decade ago, he made her promise not to raise their children in the Jewish faith.
He said he still doesn't tuck in his shirt, a habit he started in high school to make it more difficult for his abuser to put his hand down his pants. He said there are days he can't bear being on a crowded subway because "I can't stand being touched by people."
"So you don't even realize what the long-term impact is," said Bressler, a father of two.