I think the different respect for referees is largely footballs own making. In football there is the philosophy that the referee is always right and does not need to account for decisions, which flies in the face of reality. For instance, the rules against reversing cards if the referee has seen the incident even when it is obvious he didn't see the important part. Or the reluctance to use technology as it will undermine the authority, as if he isn't undermined by videos showing the 50,000 people in the crowd and millions at home that the decision is wrong.
In contrast, the rugby the referee gets help and has tools to enforce the respect. The moving a penalty 10m or even reversing a decision stamps out dissent. Let a referee cancel/reverse a penalty when players surround him to ask for the player to be sent off. Can you imagine a manager like Fergie not doing something if his players kept getting penalised for dissent. Then you have the video technology where the referee can ask specific questions, from the general "any reason why I can't give the try" (any aspect including forward passes, crossing or failure to ground the ball) to a specific "can you check the ball was grounded". The referee is treated with respect because he is accountable and everyone behaves like grown ups.
The audio channel is also good as we get to understand what he is thinking. For instance, if we heard a player told to get up for diving or stop holding the short, the a subsequent booking would be clear. Players would soon stop swearing if fined and suspended when caught on mike. And the moment a rugby referee told George Gregan "Shut up, George. I'll do the refereeing" was priceless.