Before that he served for five years as the government's chief architect at the Property Services Agency, where he helped to oversee its privatisation.
With experience in private and public sector architecture, he appealed to both Conservative and Labour governments.
A DCMS spokesperson said: "Bryan was appointed many years ago when the government had a clear need to develop the architectural agenda. Now we have a ministerial group chaired by Lord Falconer, and the resources of the Commission for Architecture and the Built Environment, which has the expertise to take up his role."
Jefferson played a large part in setting up CABE in 1999 with a wider remit than the Royal Fine Art Commission that it replaced. His last high-profile task will be to chair the DCMS conference on good design in public building, which will take place in London next week.
