High starting prices of between 19,000 and 45,000 euros ($21,000 and $50,000) and lingering suspicions about the authenticity of the artworks were thought to have scared off potential buyers.
The Weidler auction house did not comment on the reasons for the failure but said the paintings could yet be sold at a later date.
Nuremberg's Mayor Ulrich Maly had earlier condemned the sale as being "in bad taste."
Among the items that failed to sell were a mountain lake view and a painting of a wicker armchair with a swastika symbol presumed to have belonged to the late Nazi dictator.
The Weidler auction house held the "special sale" in Nuremberg, the city in which Nazi war criminals were tried in 1945.
Days before the sale a number of the artworks were withdrawn on suspicion they were fakes with prosecutors stepping in.
Hitler's work had the style of a 'slightly ambitious amateur', says art expert Stephan Klingen of the Central Institute for Art History in Munich
Sales of alleged artworks by Hitler -- who for a time tried to make a living as an artist in his native Austria -- regularly spark outrage that collectors are willing to pay high prices for art linked to the country's Nazi past.
"There's a long tradition of this trade in devotional objects linked to Nazism," Stephan Klingen of the Central Institute for Art History in Munich told AFP.
"Every time there's a media buzz about it... and the prices they're bringing in have been rising constantly. Personally, that's something that quite annoys me."
In Germany, public displays of Nazi symbols are illegal but exceptions can be made, in educational or historic contexts for instance.
To comply with the law, the auction house pixellated the swastikas on the wicker chair and a blue-and-white Meissen porcelain vase in catalogue photos, and covered them up on-site.
But none of the paintings included any of the totalitarian party's insignias.
According to Klingen, Hitler had the style of "a moderately ambitious amateur" but his creations did not stand out from "hundreds of thousands" of comparable works from the period -- making their authenticity especially hard to verify.
The watercolours, drawings and paintings bearing "Hitler" signatures featured views of Vienna or Nuremberg, female nudes and still life works, the auction house said. They were offered by 23 different owners.
Prosecutors on Wednesday collected 63 artworks from the Weidler premises bearing the signature "A.H." or "A. Hitler", including some not slated to go under the hammer.
Nuremberg-Fuerth prosecutor's office said it had opened an investigation against persons unknown "on suspicion of falsifying documents and attempted fraud", chief prosecutor Antje Gabriels-Gorsolke told AFP.
"If they turn out to be fakes, we will then try to determine who knew what in the chain of ownership," she said.
Weidler said in a statement that the paintings' withdrawal from sale did "not automatically mean they are fakes".