As others are saying the CPU registers and memory are both capable of storing strings of bits.
The primary functional difference between CPU registers and memory is that memory supports indexing.
With memory, addresses are first class. You may have heard the term first class being used with regard to functions: a programming language that has first-class functions can capture a reference to a function in a variable, such as a parameter, and hence supports passing functions as parameters. It is significant that the function referred to by a function variable (say, a parameter) can be invoked.
Similarly, memory addresses can be stored in variables and passed as parameters, etc.. These variables are called pointers. The memory referred to by a pointer can be accessed, and such access is called dereferencing, or a dereference. A dereference can be either a read from- or a write to-memory operation. Dereferencing is explicitly provided for by the hardware via its instruction set. The mechanism for dereference varies among different instruction sets, though all go to the concept of addressing mode, which is the approach for computing an effective address of something of interest, given some initial pointer value, and instructing the processor to read or write.
So, memory can be indexed: take the address of something, pass as a variable or parameter, dereference such pointer variable to access values stored at that address. Because of this, memory can be used to store arrays, objects, and even code.
By contrast, for the majority of instruction set architectures, there are no special processor features (analogous to addressing modes for memory) for indexing registers, for taking the address of a register, for a pointer-like variable that refers to a register, for dereferencing a register pointer.
Support for register references on today's processors would require either a switch statement with one "case" per register, or, self-modifying code, or writing all the registers to memory (so we can use the indexing features of memory). All of these would be rather inefficient (compared to using memory instead), so this generally isn't done. (Though operating systems do the last for context switching, and, debuggers read (or write) that memory to report (modify) register values of processes stopped during debugging.)
Registers can only be named in instructions, so cannot be indexed; registers have names but not addresses; there is no built-in mechanism to manipulate a register reference, and hence, registers cannot really store arrays, objects, or code — but they are fast and directly accessible to machine code instructions.
Of course, the hardware inside a processor is an interpreter of machine code instructions, and as such, internally, is fully capable of using register numbers as references to the CPU registers. But processors don't expose that feature to software except as what can be done by machine code instructions — which is to say machine code instructions identify the registers the program wants to use (each machine code instruction does this). No matter what the computer is doing, the processor hardware is interpreting machine code instructions from some programming (modulo modern low power modes where the processor may simply be doing nothing).
There are also non-functional differences, in that the registers are limited in count and total amount of storage, while memory is practically unlimited.
Since memory is addressed, the size of an address (in bits, sometimes called width, e.g. number of bits for a pointer variable, and number of bits used by dereference operations) determines how many different storage locations can be differentiated — hence this dictates the maximum size of memory. So, using 32 bits for addressing limits storage to 4GB (232), and since that is sometimes not enough, we have processors with 64-bit support.
Further, modern CPUs are byte addressable, meaning that every byte (8-bits) of memory has its own unique address. Variables of larger size than 8 bits (like 16-, 32- or 64-bits and larger) are often needed by programs — such a variable would necessarily occupy multiple bytes of memory — meaning also multiple addresses in memory.
Being byte addressable, in the context of needing more than one byte for certain data types, gives rise to two phenomena, one of which is scaling that is necessary in pointer arithmetic and the other is endian-ness for ordering of the bytes.
A reference to a multi-byte memory item is done using one address, namely the numerically lowest address of its range. Indexing into an array of 32-bit integers requires pointer arithmetic that scales the index by 4 since each such integer element takes 4 byte addresses (so index i is referred to as byte offset i*4).
Endian-ness is the property of exactly how the individual bytes of a multi-byte item are ordered in memory, since there are several possible orderings. An instruction set architecture will make design choices that favor one of those orderings over the others.