Ronny Jackson, physician and Trump’s pick for head for Veterans Affairs, withdrew his nomination Thursday amid a multitude of allegations against him, including being drunk on the job and improperly handing out prescription drugs such as Vanadium on international flights.

In response to the multiplying allegations from individuals who have experienced a flight with Jackson or claimed to have witnessed him drunk at work, Jackson claims the accounts “completely false and fabricated.”

Jackson, the White House physician who also reported Trump’s physical report to the press earlier this year, had faced growing allegations of misconduct, including being intoxicated while on the job, mishandling prescription drugs, and creating a toxic work environment.

Trump called into Fox News on Thursday shortly after the nomination was withdrawn to rip Jackson’s critics.

“He would have done a great job,” Trump said. “Tremendous heart. These are all false accusations. They’re trying to destroy a man. By the way, I did say, ‘Welcome to Washington. Welcome to the swamp.'”

The White House had stood by Jackson since as recently as Wednesday, even though Trump opened the door for his to withdraw his nomination prior to Thursday, according to NBC News.

Jackson, meanwhile, was back at work on Capitol Hill Thursday despite his withdrawal and allegations.

White House Press Secretary Sarah Huckabee Sanders commented on his stay at work, “Admiral Jackson is a doctor in the United States Navy assigned to the White House and is here at work today.”