Definitely! I always hated sports, well, when I was at school I did classic ballet for a few years and loved it, but since then I never found an activity that I liked. Lately I realized I like to swim and that I could actually do it also during winter too...
You're very lucky that you can do that together with your husband, it's much easier to get motivated, you know... shared pain, half pain
... but one you get used to it going to the gym can be fun!