The castle was probably built shortly after 1200 as the seat of the Lords of Vaz. In the Late Middle Ages it passed to the House of Habsburg-Austria; for this reason it was attacked and destroyed in 1499. The Barons of Vaz were Graubünden's most powerful noble family in the 13th century, owning castles in Domleschg, Schams and in the Chur Rhine Valley.

Between 2002 and 2006 Belfort underwent a program of conservation. The accompanying architectural investigations yielded a wealth of information on the origins of the medieval castle. It was thus possible to identify defensive passages, an actual defensive platform, slits for observation, galleries, a projecting garderobe, the sentry post and the dungeon. A defect on the north façade dates with, great probability, to the attack by Graubünden's troops on 4th March 1499 in the forefront of the Battle of Calven, which was fought below Müstair on 22nd May of the same year.