I was in the same situation with the rheumatologist my regular doctor referred me to for my fibromyalgia. I just said something like, "He seemed a little disappointed that he couldn't find something wrong with me that he could treat effectively, so I thought if it's okay with you, I'd like to just come back and see you for the follow up care." And he was fine with that and that's what I've been doing! (Of course he was fine with it, this way he gets my money instead!!)
Oh, and he treats my IBS too. He's really great. And he knew exactly what I was "saying between the lines" too.