Im not sure weather i want to be with him?

If your not sure, if you have even any tiny doubts, then don't do it. Listen to your instincts. Something in there is telling you you're not really that into him, even though he's really sweet and everything, you can't be with a guy just for those reasons, you've gotta LIKE him! And it sounds like you don't, not enough anyway, so it's just never gonna work and you'd never be happy.