Daily entries from the 17th century London diary
Berkeley House was later demolished for the construction of Devonshire House (more at Wikipedia).
Berkeley House
Berkeley House, which John, Lord Berkeley, erected at a cost of over £30,000 on his return from his tenure of the viceroyalty of Ireland…was constructed from 1665 to 1673. The house was later occupied by Barbara Villiers, Duchess of Cleveland, a mistress of Charles II. The house, a classical mansion built by Hugh May, had been purchased by William Cavendish, 1st Duke of Devonshire in 1697 and subsequently renamed Devonshire House.
On 16 October 1733, the former Berkeley House, while undergoing refurbishment, was destroyed by fire. The cause was attributed to careless labourers. http://en.wikipedia.org/wiki/Devonshire_House#The_site