If natural gas is not available in your area, take a hard look at high efficiency heat pumps. They will be cheaper to run than propane or oil. Not sure against pellet stoves that is not a real comparison.
We are out in the country, so no NG nearby. I priced out hybrid heat pumps last year but it was around $16k to replace both units completely. A bit more than I was hoping for.