The speech noise remover isolates speech with the Heavy setting, but it does not select a particular speaker. Other people talking may remain because their voices are also speech. Compare a short section before processing a recording with overlapping voices.