The balance in Ironsworn between being too hard on your PC for failures and making the game challenging and tense is incredibly hard to find. With weak hits being the statistical average result, some level of set back is ingrained in the system. But the set back should be dispersed between narrative and resources. Not every Pay the Price needs to result in a -resource result. Often I don't even roll and make the obvious challenging outcome happen.