He was dismissed on the recommendation of US attorney general Jeff Sessions and the deputy attorney general, the statement added.
According to the New York Times, Mr Comey learned about his termination while addressing FBI agents in Los Angeles when he caught a glimpse of news reports on TV screens at the back of the room.
He initially made light of the story, reportedly believing it to be a prank, but was then led to a nearby office where his dismissal was confirmed.
Shortly afterwards, a letter from President Trump, dated Tuesday, was delivered to the FBI Headquarters in Washington. The letter was later released to reporters by the White House.
Sky News’ US Correspondent Greg Milam said the news had “taken all of Washington by surprise”.
In his letter President Trump wrote: “you are hereby terminated and removed from office, effective immediately.”
The letter went on: “While I greatly appreciate you informing me, on three separate occasions, that I am not under investigation, I nevertheless concur with the judgement of the Department of Justice that you are not able to effectively lead the Bureau.”
Mr Trump concluded: “It is essential that we find new leadership for the FBI that restores public trust and confidence in its vital law enforcement mission.”
The decision to sack Mr Comey will raise questions about Mr Trump’s motives, in light of the fact that Mr Comey had been leading the FBI investigation into the Trump campaign’s Russia links.
Milam added that some members of the committees investigating the Russian links “believe this is an abuse of power and that’s why the controversy about this isn’t going to go away, even though James Comey has left office”.
The search for Mr Comey’s successor will start immediately.
Mr Comey, 56, was nominated by Mr Trump’s predecessor Barack Obama in 2013 for a 10-year term, having served three decades in law enforcement.
The father of five was known for his tenacity, his towering height (he is 6’8 tall) and being a highly skilled political operator.
Many were surprised by Mr Trump’s decision to keep Mr Comey and some saw it as a reward for his role in damaging the presidential campaign of Democratic rival Hillary Clinton by reopening an investigation into her email practices as secretary of state.
But in March, Mr Comey took aim at Mr Trump during a hearing on Russia’s alleged meddling in the election.
He confirmed the FBI was investigating the allegations and Russia’s possible connections with Mr Trump’s campaign.
He also denied Mr Trump’s claim that he had been wiretapped by Mr Obama.
Democratic Senator Ron Wyden described the decision as “outrageous”, fellow Democrat Mark Warner said it was “shocking and deeply troubling”, while other Democrats called for an independent prosecutor to continue the investigation into Russian links with the presidential campaign.
Milam said the timing was “making a lot of people here, Democrats mainly, very uneasy”, adding that some were describing the situation as “a constitutional crisis”.
He added: “Here you have the head of an organisation that is investigating the White House and the White House taking the decision to remove the head of that organisation.”
It emerged that Mr Trump had called at least two members of the Senate Judiciary Committee before the White House announced the firing of Mr Comey – Lindsey Graham and Dianne Feinstein.
Mr Graham is heading the panel’s investigation into Russian interference in the 2016 election and Ms Feinstein is the committee’s top Democrat.
Mr Graham said that “given the recent controversies surrounding the director, I believe a fresh start will serve the FBI and the nation well”, while Ms Feinstein said Mr Trump told her the FBI needed a change, and that the next director “must be strong and independent.”
Reuters reported, however, that US intelligence agencies were not told of Mr Trump’s move in advance.