Run across the open field directly to the opposite side. There's a brick wall next to the opening of the castle; climb up the part that looks different from the rest and jump to the other side and ...