Choose a point so that at full power your arrow will hit an enemy on the top of the first ladder. Remember that location. No enemy will rarely make it to the second level alive. If necessary, leave enemies free to shoot you from ground level in order to make sure that none of them get up the first ladder. You can kill them later.