I switched from sending everything priority after I looked at my bill last Nov-Dec of over $500. For anything in the same zone First Class is just as fast as long as the box isnt over 13ozs, most fivers run about 8-10oz. Priority is faster coast to coast so I use Priority for long distance stuff.
Shipping to Canada, always use FC, Priority does not get it there any faster. I am not sure about to US from Canada. But I would switch for FC to save yourself some money.
I do not count the packages I send to the troops. Shipped about 10 Flat Rate boxes this year so that's another $100+.