Even adc $3, %rax can't usefully use the special rax-only encoding
REX.W + 15 id ADC RAX, imm32.
REX.W + 15 03 00 00 00 is 6 bytes. (adc rax, imm32)
REX.W + 83 mod/rm 03 is 4 bytes. (adc r/m32, imm8, where the mod/rm byte encodes rax as the destination, and /2 in the reg field is part of the opcode. The immediate-src operations share the first opcode byte.)
The (16bit version of) both encodings were introduced with 8086. See the link in the x86 wiki. Apparently the accumulator was expected to be used for everything all the time, and/or they weren't thinking of future instruction set extensions, so they thought it was worth spending that many opcodes on special al and ax versions of all the ALU immediate instructions.
If you look through two-operand integer ALU instructions (and, or, sub, test, etc.), each one has a special one-byte-shorter encoding for al and ax/eax/rax destinations, with full-sized immediate operands. (i.e. imm32, not imm8 sign-extended to 32 or 64b). So two extra opcodes for each instruction.
This only affects x86 code-size. Once the instructions have been decoded, there's no further difference in how they run. See http://agner.org/optimize/ to learn more about CPU internals.
AMD64 could have left these out of 64bit mode, freeing up a lot more coding space, but they probably weren't optimistic about killing off 32bit. If you want an instruction to work in 32 and 64bit mode, takes fewer decoder transistors if the encoding is the same in both modes. They could have used the coding space for setcc r32 or something, though. Not fancy new SIMD functionality, just un-gimp some of the basic instructions. You can almost never use setcc without an xor to zero the full register before the flag-setting operation. Anyway, AMD missed a golden opportunity to remove some cruft from x86.
Fun fact: on Broadwell / Skylake (and later?), the special-case AL/AX/EAX/RAX with immediate encodings of adc are actually slower. See Which Intel microarchitecture introduced the ADC reg,0 single-uop special case?
This may also apply to adc al,0 on earlier Sandybridge / Haswell. (adc eax, 0 wouldn't use that encoding.)