Clear the way for your wallbreakers with archers

Wallbreakers will attack the walls that are trapping the nearest enclosed building.

So for better targeting, you can use archers to clear out smaller wall boxes so that the wallbreakers will target the inner sections that you really want to get to.