The armed forces counter-intelligence agency is looking into the claims after 143 cases were reported last year and 53 so far in 2017.
Among the incidents were soldiers performing Nazi salutes or making racist remarks against colleagues with migrant backgrounds.
It is illegal in Germany to show Nazi symbols, such as swastikas, in public or make salutes associated with Adolf Hitler’s regime.
The country’s defence ministry detailed the investigation in a letter to parliament and said some of the most serious cases had not been dealt with strongly enough.
In one case, a soldier was heard saying “heil Hitler,” “heil our leader” and “sieg heil, comrades”.
The incident was passed to the military prosecutor and the public prosecutor’s office but the soldier did not get an early dismissal or a service ban, said the defence ministry.
In another case, a soldier made racist comments, such as demanding the death sentence for “typical foreigners” on a Facebook page linked with the far-right National Democratic Party (NPD).
The defence ministry said the soldier was “only disciplined”.
The country’s constitutional court said in January that the NPD resembled the Nazi party but decided not to ban it because it was too weak to threaten democracy.
In a third case, a soldier was allowed to keep his weapon after he was disciplined for performing the Nazi salute while on a trip to the Latvian capital, Riga.
There have been reports in Germany that Islamic extremists may be trying to join the country’s armed forces to get military training with the possible aim of carrying out attacks.
New security checks will begin in July in an attempt to weed out potentially dangerous recruits.