How does the carry and overflow flag behave when result is to big for register

Question

The following example should set both the carry and overflow flag:

mov eax, -120
add al, 140

add al, 140 will resemble the following operation in bit:

 10001000
+10001100
---------
= 100010100

So the result has 9 bits while the maximum amount of bits that can be put in al is 8. But the way I understand the overflow-flag it's only set when the result of an operation is "wrong". For E.g. when adding a positive number to a positive number but then the sign-flag is set so it yields a negative result.

And regarding the carry flag in the examples I've done so far, I only set it when I subtract a smaller value from a bigger value.

So my question boils down to how the carry and overflow flag behaves when the size of the operation can't be represented in the register.

Answer 1

Ok, the problem that you have here is a mismatch with how computer data types work and what you're doing.

In general the computer can work with (i.e. add) signed data types or unsigned data types — but it can be problematic to ask it to add a signed value to an unsigned value.

Here, -120 is negative, so must use a signed representation.

However, 140 itself does not fit in 8-bit signed, so in 8 bits, must use an unsigned representation.

This combination is not supported by the hardware — while the addition will work to give a truncated result, the hardware will not give you meaningful information in the flags if you perform 8-bit addition.

(The flags for an 8-bit addition are meaningful if both operands are 8-bit signed or else if both operands are 8-bit unsigned, but they are not always meaningful when mixing the two data types: in particular when the values involved are sufficiently large such that the upper bits are in use, as is the case with -120_{signed 8 bit} and 140_{unsigned 8 bit})

The approach to use to get proper results here is to convert both numbers to a larger data size that will accommodate both numbers (it will be a signed data type so it can accommodate -120, and of course, the method of conversion will differ for the two operands as they are different data types).

So, convert -120 from 8 bit to 16 bit (or larger) by sign extension.

And, convert 140 from 8 bit to 16 bit (or larger) by zero extension.

Then you can perform addition and get a signed result, with a meaningful overflow flag.  Of course that particular computation won't overflow in 16 bits — in fact, no addition in 16 bits that started with two 8-bit values can overflow.

However, if you want to take it back to 8 bits, you can check to see if it fits in that size & data type.  For all practical purposes, in this scenario the upper 8 bits of the 16-bit result contains the overflow information of interest regarding the lower 8-bit result, rather than that information being in the flags (overflow/carry).

First, decide which 8-bit data type you want to check: as to signed or unsigned.  There are several approaches that would work, but a simple one is to truncate the 16-bit value to 8 bits, the expand it back from 8 to 16, and see if that newly expanded 16 equals the original 16-bit result.

For signed 8 bit, you would truncate the 16-bit result to 8 bits and then sign extend it back to 16-bits, then to compare with the original 16-bit value.  If it differs the value didn't fit in 8-bits signed.

The same with unsigned 8 bit: truncate the 16-bit result to 8 bits and zero extend it back to 16 bits to compare with the original, if it doesn't compare equal then it doesn't fit in 8-bits unsigned.

Alternately, from the 16-bit original result, you can examine the sign bit of the low 8-bit value and the value of the upper 8 bits.  If all the upper bits are the same bit value as the sign bit of the lower 8 bits then the 16-bit result will fit in 8-bits signed without loss.

For unsigned, if the upper 8 bits are 0 the lower 8 will fit in 8-bit unsigned without loss.