Ignore those that make you fearful and sad, that degrade you back towards disease and death.
Some voices pull you toward life and some pull you toward decay, and you are allowed to choose which ones you listen to. The "disease and death" here is probably not only physical. It points to the numbness, dread and smallness that settle in when we spend time with people or ideas that feed our fear. The advice is not to hate those voices, only to stop giving them your attention.
The thought fits the Sufi tradition in which Rumi lived and wrote, where the spiritual path is about waking the soul and turning it toward love and joy. Companionship matters a great deal in that tradition, because the people near you shape what you become. A caution about attribution: this wording comes from a modern English rendering of Rumi's poetry, which is often quite free, so it may not follow any single Persian line closely.
In daily life, this might mean noticing which relationships or habits leave you feeling worse. Perhaps a group chat is full of gossip and doom, or a social media feed leaves you anxious every night. You could mute it, answer less often, and spend that time with someone who leaves you steadier. It is a small act, but it protects your energy.