I can't pull the tap handle off the spindle, and I don't want to break it by using too much force.
Try using WD40 and leave to lubricate the joint. If the tap handle still can’t be removed, you could try purchasing a Tap Handle/Faucet Puller which should be available from your local hardware store. If this doesn’t work, you may have to contact a licensed plumber. It may be cheaper to call a plumber from the outset, as they can easily fix the issue rather than you risking damage to your pipes by trying to DIY.