Mold odors can be pervasive, even though the mold growth may be limited in scope. According to The Money Pit's indoor air quality expert, Jeff May, despite the strong odor, there may not be any mold spores on your clothing. If, on the other hand, the mold growth in the house was disturbed (if you sat on a moldy couch or rug), your clothing may be carrying spores and even mold-eating mites.
This isn't that different from getting pollen or pet dander on your clothing, and you can wash or dry-clean your clothing to remove them. Figuring out the source of the moldy smell in your relative's house is another matter!