The tubes always have bias current in them, so they will wear. If the amp is class A, they will wear just as fast as if you are using them. For class AB (push-pull), they will wear much more slowly.
If you are using one channel at a moderate level, the tubes will not wear much faster than the unused channel. If you push the channel pretty hard during use, you will wear those tubes faster.
As with auto tires, you should consider rotating channels for even tube wear and best tube life.