I think it could be fun, for sure.
I think the castle appears if you defeat 3 specific bosses first (their names are also found where the castle appears, on their busts or something like that). Also, I *think* there are some hints about this in the game (as in, somewhere they do mention stuff about the castle and these 3 people).