So named because it met at Brooke House.
From The Private Palaces of London Past and Present by E. Beresford Chancellor (1908):
the “Brooke House Committee,” … had been appointed, in 1668, to examine into the expenditure of certain moneys granted by Parliament to Charles II. for the ostensible purpose of prosecuting the war with Holland, but which seem, as was not then unusual, to have been employed by his Majesty in more peaceful projects. We find Pepys, on December 18th, wending his way thither, and carrying with him by order, the “Contract-books, from the beginning to the end of the late war.” “I found him” (Colonel Thomson), says the Diarist, “finding of errors in a ship’s book, where he showed me many, which must end in the ruin, I doubt, of the Comptroller.”