Attorneys for Oskar Groening, 96, filed the appeal with prosecutors in Lueneburg, where he was convicted in 2015 as an accessory to the murder of 300,000 Jews. He was sentenced to four years in prison, but hasn't yet spent any time behind bars because of the appeals process.
Lueneburg prosecutors' spokeswoman Wiebke Bethke said her office would likely make a decision on the clemency request this week, in consultation with the panel of judges who convicted Groening.
She said she wasn't permitted to give details of Groening's argument for why he shouldn't serve his sentence.
A doctor has previously declared Groening fit to go to prison so long as there is appropriate medical care.
Groening exhausted his last chance at legal appeals in December when the country's highest court rejected his attorneys' argument that imprisoning him would violate his constitutional right to life and physical safety.
The Federal Constitutional Court noted, however, that German law allows for prison sentences to be interrupted if a prisoner's health deteriorates significantly.
Hannover prosecutors, who have been handling Groening's case, told The Associated Press that Groening hasn't yet been summoned to report to prison following the final court ruling, but that consideration of the clemency appeal shouldn't delay that process.
Groening, who has been dubbed the "accountant of Auschwitz," testified at his trial that he oversaw the collection of prisoners' belongings and ensured that valuables and cash were separated to be sent to Berlin. He said he witnessed individual atrocities, but didn't acknowledge participating in any crimes.
The court that convicted him ruled, however, that he was part of the "machinery of death," helping the camp function and collecting money stolen from the victims to help the Nazi cause, and thus could be convicted as an accessory to the murders committed there.