People threw punches, hurled water bottles and used pepper spray during the violence as riot police were deployed to try to disperse the crowds. At least eight people were hurt.
Among the three dead were two police officers who were the pilot and passenger on a helicopter that crashed near the scene.
The other death was a 32-year-old woman who had been crossing the street when she was struck by a car that hit a group of anti-Nazi campaigners also leaving 26 people injured.
The male driver has been arrested.
The rally by far-right activists was organised to protest against Charlottesville’s decision to remove a statue of Confederate General Robert E. Lee from the city.
Speaking about the disorder, US President Donald Trump said he condemned “in the strongest possible terms” what he called an “egregious display of hatred, bigotry and violence on many sides”.
He added: “What is vital now is a swift restoration of law and order and the protection of innocent lives.”
White supremacists and anti-fascist protesters have clashed in Charlottesville, Virginia
Anti-Nazi campaigners were protesting against a far-right rally in the city
During the scuffles, people threw punches, hurled water bottles and used pepper spray
The ‘Unite the Right’ gathering was a far-right protest against the decision to remove a statue of Confederate General Robert E. Lee from the city
White nationalist protesters receive first-aid after the clashes
Police fired tear gas and said that arrests were made
A car hit a crowd during the rally. One person died and 19 others were injured.
Rescue workers help a victim who was hurt when the car struck a group of counter protesters
A victim lies on a stretcher
The car that was allegedly involved in the incident. Meanwhile two other people died when a police helicopter, monitoring the violence from the air, crashed.
Charlottesville mayor Michael Signer had said he was disgusted the white nationalists had come to his town and blamed the US leader for inflaming racial prejudices during his presidential campaign last year.
Mr Signer said: “I’m not going to make any bones about it. I place the blame for a lot of what you’re seeing in America today right at the doorstep of the White House and the people around the president.”
— VA State Police (@VSPPIO) August 12, 2017
The fighting prompted Charlottesville city manager Maurice Jones and interim county executive Doug Walker to simultaneously declare local states of emergency for the two jurisdictions.
Virginia governor Terry McAuliffe declared an emergency as police struggled to manage the two groups.
Up to 6,000 white nationalists had been expected to attend the “Unite the Right” rally.
TV pictures showed one of the fights erupting when the anti-Nazi protesters approached the white supremacists with what looked like a wooden banner.
As they did so, they were met by a wall of men dressed in body armour and helmets.
Other footage showed rivals on either side using hand-made shields as they fought each other.
Police said some people in the crowd used pepper spray.
Our country encourages freedom of speech, but let’s communicate w/o hate in our hearts. No good comes from violence. #Charlottesville
— Melania Trump (@FLOTUS) August 12, 2017
Initially it appeared as though police were not getting involved but the the Virginia State Police have since posted pictures of heavily armed officers carrying out arrests.
The violence prompted First Lady Melania Trump to call for an end to the clashes, tweeting: “Our country encourages freedom of speech, but let’s communicate w/o hate in our hearts. No good comes from violence. #Charlottesville”
The clashes came after a federal judge ordered Charlottesville authorities to allow a weekend rally of white nationalists and other extremists to take place.
The rally was organised by right-wing blogger Jason Kessler, who said that he was saddened people had been hurt and that the driver of the car”did the wrong thing”.
The Southern Poverty Law Center had said it was expecting a range of far-right groups to attend, including the National Socialist Movement, the League of the South, as well as “various anti-government, Patriot, militia, Klan, III Percenters and anti-immigration groups”.