Sorry for the confusion. Yes, you must disconnect headphones to activate the RCA outputs.
When headphones are plugged in, the RCA Outputs are floating (no connection inside of the amp besides ground). Thus, you are definitely describing a form of high frequency ground loop between the source and speakers. A ground loop isolator will likely help. Using different outputs rarely resolves this behavior.
係ground loop, 不過個ground loop inside the atom
吹漲