Dressed in a dark suit, the bearded Breivik stared briefly at reporters while making the salute but didn't speak.
Judge Oystein Hermansen asked him not to repeat the salute, saying it insulted the dignity of the court.
"It also disturbs what we are dealing with here," Hermansen said, brushing aside attempts by Breivik to defend his action.
The 37-year-old right-wing extremist, who killed 77 people in a bomb and shooting rampage in 2011, sued the government last year. He argued that his solitary confinement, frequent strip searches and the fact that he was often handcuffed during the early part of his incarceration violated his human rights.
But lawyers representing the government said that he enjoys better prison conditions than some inmates in Norway. They also warned that he remains a threat and should continue to be held in solitary confinement.
The government is appealing a surprise decision in April by the Oslo District Court, which sided with Breivik's claims that his isolation in the maximum-security Skien prison breaches the European Convention on Human Rights.
The ruling said "the prohibition of inhuman and degrading treatment represents a fundamental value in a democratic society. This applies no matter what—also in the treatment of terrorists and killers." It also ordered the government to pay Breivik's legal costs of 331,000 kroner ($41,000).
However, it dismissed his claim that his right to respect for private and family life was violated by restrictions on contacts with other right-wing extremists.
Speaking for the state, Fredrik Sejersted said the government's view is that Breivik's prison conditions don't violate his human rights in any actual or legal sense.
Describing the killer as Norway's most expensive prisoner, Sejersted said that "in many ways they are better than (those) of other prisoners to compensate for the fact that he cannot make contact with other inmates."
"That is far from violating human rights," he said.
Breivik was convicted of mass murder and terrorism in 2012 and given a 21-year prison sentence that can be extended for as long as he's deemed dangerous to society. Legal experts say he will likely be locked up for life.
He is being held in isolation in a three-cell complex where he can play video games, watch TV and exercise. He has also complained about the quality of the prison food, having to eat with plastic utensils and not being able to communicate with sympathizers.
The government has rejected his complaints, saying he is treated humanely despite the severity of his crimes and that he must be separated from other inmates for safety reasons.
Breivik had carefully planned the attacks on July 22, 2011. He set off a car bomb outside the government headquarters in Oslo, killing eight people and wounding dozens. Dressed in a police uniform, Breivik then drove to the island of Utoya, about 40 kilometers away, where he opened fire on the annual summer camp of the left-wing Labor Party's youth wing. Sixty-nine people there were killed, most of them teenagers, before he surrendered to police.
At the time of the attacks, Breivik claimed to be the commander of a secret Christian military order plotting an anti-Muslim revolution in Europe, but now describes himself as a traditional neo-Nazi who prays to the Viking god Odin. He also made a Nazi salute to journalists at the start of his human rights case last year.
The massacre shocked the oil-rich, quiet Scandinavian country and many feel Breivik has had too much attention and visibility.
Sejersted described Breivik as Norway's most dangerous man at the time of the attacks and warned of the difficulty to assess how dangerous he will be "tomorrow, next year or in 10 years."
"But he has strengthened his right-wing beliefs," Sejersted told the court.
The hearing continues Wednesday in a makeshift courtroom in the gym of Skien prison in southern Norway, where Breivik is incarcerated. Government attorneys will continue to state their case before Breivik is heard on Thursday.
Six days have been reserved for the hearings by the Borgarting Court of Appeals. A ruling is expected in February.