Shinzo Abe honoured those who lost their lives and promised that Japan would never again wage war on another country.
During an evocative speech Mr Abe said “We must never repeat the horrors of war again. This is the vow we the people of Japan have taken.”
Speaking alongside the memorial over the sunken battleship USS Arizona, he said: “To the souls of the servicemen who lie in eternal rest aboard the USS Arizona and to the American people and to all the people around the world, I pledge that unwavering vow here as the Prime Minister of Japan.”
Japanese forces attacked the US fleet in their Hawaii base of Pearl Harbor in December 1941, killing 2,400 service personnel.
The attack was one of the defining episodes of the Second World War and propelled America into the conflict.
The visit was the first time a Japanese and US premier had visited the site of the attack together.
It was designed to show a strong alliance between the two former foes amid concerns about China’s growing strength and worries about a more complex relationship with Japan after US President-elect Donald Trump takes office.
As expected, Mr Abe did not apologise for the attack but Mr Obama said his visit was a historic one and a reminder that “even the deepest wounds of war can give way to friendship and lasting peace”.
Earlier this year Mr Obama visited the Japanese city of Hiroshima where the United States dropped an atomic bomb in the dying days of the war with catastrophic consequences.
He was the first serving US president to make the highly symbolic trip and the visit to Pearl Harbor was seen as an act of reciprocity.
Underscoring the importance of the event, Mr Obama’s top Asia adviser in the White House Daniel Kritenbrink said: “This visit, and the president’s visit to Hiroshima earlier this year, would not have been possible eight years ago.
“That we are here today is the result of years of efforts at all levels of our government and societies, which has allowed us to jointly and directly deal with even the most sensitive aspects of our shared history.”