Frost's shock resignation, announced late on Saturday, comes not only at a crucial time in talks with the European Union but also as Johnson's position looks increasingly under threat.
But Frost maintained: "I left the government because I couldn't support certain policies, most recently on Covid restrictions.
"If you're a minister, you have to support collective responsibility... and I couldn't and that's why I had to leave," he told Sky News outside his London home.
Frost was seen as Johnson's closest ally on Brexit, heading talks to secure a deal with the EU, and in recent talks to iron out problems on trading arrangements in Northern Ireland.
His departure, amid suggestions London could back down on its demands for revising the so-called Northern Ireland Protocol, could not have come at a worse time for Johnson.
He is under pressure on all fronts over his handling of the coronavirus pandemic, including claims of a series of lockdown-breaking parties held at Downing Street last year.
Last week, nearly 100 of his own MPs rebelled against new restrictions to prevent the spread of the Omicron variant, compounding a heavy by-election loss in a formerly safe seat.
Frost has been touted as a potential challenger to Johnson, and is popular with grassroots members of the Conservative party for his backing for a low-tax, lightly regulated economy.
But he insisted his resignation was "absolutely not about leadership".
"This is about policy. I'm absolutely confident this country has a great future under Boris Johnson's leadership if we can get the policies right."