'Low hanging fruit' or 'picking the lowest fruit' exists as a term in many cultures. Its likely that it passed to English from Latin roots. Its difficult to see the connection from an American singer's song to racism. Doesn't make sense. If Billie Holiday used some other term in his song, will that become a racist term.