Passing the tuff stone door with its 1908 gate, the castle's driveway, built in 1890, opens up. The castle of the Counts of Gruyère was built in the form of the “carré Savoyard” under Peter III during the second half of the 13th century before becoming the seat of Fribourg’s bailiffs (1555–1798) and later a prefecture (1798–1848). After its acquisition in 1849 by the industrialist Bovy family from Geneva, the castle was transformed into an artistic colony before it was bought by the State in 1938. The western esplanade, its surrounding wall and the Saint-Jean chapel were constructed after the castle. Transformations to the main building and construction of the stone galleries were carried out under the Counts Louis and François II (1476–1540).

The Bovy family began a complete refurbishment of the interiors in 1850.

Layed out in the East, the French garden is surrounded by a sentry walk and two towers at its outermost points.