My boyfriend said something hurtful to me

My boyfriend and I were talking on the phone today and he said something like, “most of the time you are definitely not broken, but sometimes your brain is.” He does know about my diagnosis. I got very quiet. Then he said that he was only kidding. I was very hurt by his words!! When we talked about this, he said that he never meant to hurt me and he apologized to me.