If it only occurs with the XBOX controller and not with the mobile phone, for example, then it is more likely to be due to the XBox COntroller. I also use the headset with the Xbox and have had no problems so far :-)

I only use it for my mobile phone.