Ah, it was the four slotted hex-head bolts, one of which was pretty tight to get out. Maybe I should use some removable Locktite on them when they go back in.
Here's the assembly removed, which is that on page 2-25. The removed rod and holds the main tower to the metal base. Taking one circlip off is sufficient to allow wiggling the rod out of there and releasing the main tower.
The main turret has a grey base plate stuck to the bottom of the actual tower. That base plate is "poked" by a rod from beneath when the castle is "breaking". Hence the base plate has a load point where it meets the main tower... which is where it fractured in my case.
I will try to epoxy the base back onto the tower. Maybe that will hold. If it doesn't I could probably "beef up" the base by adding something under the plastic to strengthen it (carbon fibre comes to mind). If that doesn't work then I'll need the replacement part, which I suspect is both base plate and castle combined.