When the whole sweater is done, you may also realize that you don't even notice the little things that were bothering you while knitting :) It may seem too loose up close, but when you see the whole fabric it's gonna be fine. Check your gauge and if it's fine, just trust the process :)
chekccard