I guess I'm one of them. But for my own part I should probably clarify that I don't insist on anything. It's just an intuitive feeling, and I'm pretty sure that goes for all of "us."
My interpretation of a numerical scale is simply shaped by the ones that I've encountered in my past: The 0-6 scale at school, where 6 was the top grade - 3 or less was a disaster. At university, a 1-4 scale with 1.5 as the (realistic) top grade - anything below 2.5, unacceptable. Papers throwing the dice on movies and records - 3 or less definitely a signal that something doesn't ain't right in the critic's eye... the list goes on and on.