Mine does the same thing IF there's no music already playing. Mine sounds like a 60hz tone which could be bad ground idk. But if I play even the slightest music it completely stops the tone.
I know it's no solution but maybe help you I some way lol
Edit: it does it if the device isn't sending a signal to it at the time
Example: if my laptop is plugged into it and the laptop goes to sleep it will start buzzing so I would imagine RCA related