Work on the defensive facilities continued in the 16th century, when they built a large trapezoidal tower and a cylindrical tower at the entrance to the castle. Increased robots, fees and taxes resulted in 1610 in a peasant uprising of subjects, which the dominion was cruelly suppressed. During the restoration of the castle, destroyed by the Turks after the fall of Nové Zámky in the autumn of 1663, they built a new fortification with cannon bastions on the southern, most endangered side. Gýmeš was one of the few castles where construction work took place in the 18th century. The transformation of the Gothic palace into the ancestral crypt of the Forgách family with a chapel dates back to this period. It was not until the middle of the 19th century that the manor left the castle definitively and began to disintegrate.
At the beginning of the 18th century, during the Rákoczi uprising, the owner of Gýmeš Castle, Sigismund II. Forgáč became the commander of Rákoczi's army, for which he got into exile and Gýmeš Castle became the property of the king. Emperor Joseph I sold Gýmeš Castle to the Czech chancellor gr. Ján Vratislav. The castle was bought from him by the Bishop of Rošoň, Pavol Forgáč, who repaired it. He had 4 chapels built on the castle, one of which has been preserved - St. Ignatius. He was the patron saint of the castle and there were festive feasts with a great parade on the anniversary. Priestly robes embroidered with women's hair were used. Below this chapel was the family tomb of the Forgáč family with a large cross made of alabaster. Bishop Pavol had a church and a baroque castle built in the village of Gýmeš in 1720-22. The transfer of the heirs to the manor house was the beginning of the end of Gýmeš Castle.