I think you have it right. There are basically 3 different possible configuration:
1) single 8 ohm speaker plugged into the 8 ohm output
2) single 4 ohm speaker plugged into the 4 ohm output
3) dual 8 ohm speakers both plugged into the 4 ohm outputs
You should not have anything connected to...