"The main voices were very skeptical about the direction of the papers prepared there," Czech Foreign Minister Karel Schwarzenberg said after EU foreign ministers discussed the Geneva meeting on Monday.
"Probably we will send now from the EU a suggestion of ours and if the conference papers are in line with that, we will stay, otherwise there is a strong call to withdraw," said Schwarzenberg, who chaired the talks.
"Some of the wording (in the conference's draft documents) is considered to be anti-Semitic," one EU diplomat said.
The bloc was also concerned about the mention of defamation against religion, he said. "The EU thinks this has nothing to do with human rights," the diplomat told Reuters.
German Foreign Minister Frank-Walter Steinmeier said he was concerned the conference "might be abused for one-sided statements on the Middle East conflict."
"I plead that we would cancel our participation unless there is in the next few days a change in the preparation," he told reporters in Brussels.
The United States has also said it will not attend the conference unless the wording of the final declaration is altered radically. Israel is calling for a boycott of the event.
But diplomats have said that memories of street marches targeting Jews in general at the first UN Racism conference in Durban in 2001 remain strong and fears of a repeat have grown after protests in Europe over Israel's war in Gaza.