I think the practical worry is that if we stop adding leap seconds, the time our computers use slowly drifts away from actual sun time, and anything that depends on timestamps matching up could get confused. I don't know, that just seems messy to me.