Emmie,
Hello this is Kitt. Buying a house together is a huge step. I guess if your feeling this way now my advice would be the best way to protect yourself: Don't buy the house with your boyfriend. Don't do ANYTHING jointly involving money with someone that you're not prepared to share your life with.
If your sure this is something you want to do, then have your own lawyer to make sure the ownership is set up so that you will not lose your investment in the house if things should not go well.
If you are not feeling anything for him, it feels to me like your not ready to buy the house and should cut your losses right now. Buying a house is always going to cause anxiety but your's is raising red flags and any time you have these kinds of doubts you should step back and talk it through with your therapist or someone that has your best interests at heart.
I wish you peace and please keep posting. We are here for you.
Hugs
Kitt