The castle, according to the Chronicle of Morea, was built in 1256 by the ruler of the Principality of Achaia, William II Villeardouin, to subdue the Tsakonians living in the area. The castle was called Castello Della Estella, that is, the Castle of Asterios, because of the asteroid peninsula on which it was built. This castle was built almost simultaneously with the Castle of Oria, which was built for the same purpose. In 1465, during the first Venetian-Turkish war, the castle was handed over to the Venetian governor of Nafplio, Christophoro de Priuli. In the following years, the castle declined and the medieval fortifications collapsed.

In the period of 1821, wealthy foreign merchants, brothers Zafiropoulos (Constantine, Ioannis, and Panagiotis), originally from Saint John, fortified the castle and built their houses there. On August 4, 1826, 1,200 soldiers locked inside the castle defeated the numerous army of Ibrahim who besieged it. In 1833, Panos Zafiropoulos (Akouros) dedicated the castle to the Kingdom of Greece and to King Otto. In the early 20th century, the castle was abandoned and deserted.

The shape of the castle is almost square while the cliffs to the west and the sea to the east make the narrow hill fort hard to dominate. The castle has two gates, one to the northwest end protected by a massive circular tower and the second to the east, accessed from the top of Astros Beach. Its fortified enclosure is up to 6 meters high and features mainly lanes on the east side, while openings on the south side have openings for firearms. On the top of the peak are ruined warehouses, water tanks, and the three houses of Zafiropoulos.

The first house, by Panagiotis Zafiropoulos, has a rectangular plan and is built on the foundations of an older building. The second, by Konstantinos Zafeiropoulos, has four apartments, utility rooms, a toilet, and cisterns. Ioannis Zafeiropoulos`s third home is not in good condition and looks very much like that of Constantine.