Does he sound like a loser? Am I being too judgmental?

Are you forgetting that you work at a coffee shop too? Also, you've only known him for a week, but have already decided the former girlfriend/wife is bitter and vengeful, and that she has no reason to be. You are making all kinds of umptions here. Why not take some time to get to know him, and see where it goes?