The royal castle is mentioned in 1253 as the property of the de Vanch brothers, of whom Štefan was the archbishop of Esztergom. Together with the Revište castle, situated 15 kilometers south on the other bank of the Hron, it protected the passage and the road to the mining area, especially to Banská Štiavnica, and probably that is why it was acquired in 1320 by the Count of Štiavnica. The castle became the center of the Šášov estate. During the Gothic and Renaissance times, it often changed owners. King Sigismund donated it to Queen Barbora in 1424, in 1447 it was in the hands of Ján Jiskra. Queen Beatrix, the wife of King Matej Korvín, donated it to the Dócio family in 1490, who owned it until the extinction of the family in 1647. In 1650, Gašpar Lipai acquired the castle estate. During the Tököli Uprising in 1677, the insurgents seized the castle and looted it. Since then, he has been in the administration of the treasury, and since he has lost his defensive function, he has been abandoned since the 18th century.