David Cameron has been accused of secretly trying to pack public bodies with Conservative Party supporters.
The Labour peer Baroness Morgan of Huyton hit out after she was told she would not be appointed for a second three-year term as the chairman of the schools inspectorate, Ofsted.
Lady Morgan said her removal was part of a pattern which had seen a series of non-Conservative supporters on bodies like the Arts Council and the Charity Commission replaced with loyal Tories.
"I am the latest of a fairly long list of people now who are non Conservative supporters who are not being re-appointed. I think there is absolutely a pattern. It's extremely worrying," she told the BBC Radio 4 Today programme.
The Independent reported the Liberal Democrats were so concerned at what was happening that the Deputy Prime Minister Nick Clegg was believed to have raised the issue with the Cabinet Secretary, Sir Jeremy Heywood, in an attempt to stop Tory ministers making "party political" appointments.
Lady Morgan, who has agreed to stay on until the autumn while a successor is found, stressed she had not had any falling out with Education Secretary Michael Gove, and that the impetus behind the appointments appeared to come from No 10.
"One of the really important things about public appointments is that they are made on the basis of merit and they are seen to be transparently made. I think there is something going on in the centre that's mitigating against that," she said.
"I think there is an absolutely determined effort from No 10 that Conservative supporters will be appointed to public bodies. I think that is an issue for the Cabinet Secretary and the Cabinet Office to look at.
"It has been a quiet, quiet drip. I'm not talking about Labour people being replaced, I'm talking about non-Conservative supporters being replaced by Conservative supporters.
"There is a lot of concern about it. Often they are people who have been working really well with their organisations and, indeed, with their host departments, so I do think this is coming from No 10. I don't think it is coming from individual departments."
Although Lady Morgan is a former close adviser to Tony Blair, her departure from Ofsted was seen as a particular surprise as she was regarded as strong supporter of Mr Gove's free school and academies programmes.
The Education Secretary paid tribute to her "tremendous contribution" to the work of Ofsted.
"She has brought great knowledge and insight, leading the board strongly through a period of significant change, both managing the smooth transition when there were changes in chief inspector, and leading the reforms to the inspectorate and its work," he said.
"I hope that she continues to play an active role in helping to shape the education landscape to improve standards for all children and young people."