Determining signed overflow (x86 Overflow / Auxilliary Flags)

Question

First of all: I really tried to find a matching answer for this, but I just wasn't successful.

I am currently working on a little 8086 emulator. What I haven't still figured out is how the Overflow and Auxilliary flags are calculated best for addition and subtraction.

As far as I know the Auxilliary Flag complies with the Overflow flag but only uses 4 bits while the Overflow Flag uses the whole size. So if I am adding two signed 1-byte integers the OF would check for 1-byte signed overflow while the Auxilliary Flag would only look at the lower 4 bytes of the two integers.

Are there any generic algorithms or "magic bitwise operations" for calculating the signed overflow for 4,8 and 16 bit addition and subtraction? (I don't mind what language there are written in)

Remark: I need to store the values in unsigned variables internally, so I do only have the possibility to work with unsigned values or bitwise calculations.

Might one solution that works for addition and subtraction be to check whether the "Sign Flag" (or bit 4 for the Auxilliary flag) has changed after the calculation is done?

Thanks in advance!

Answer 1

Overflow Flag indicates whether the result is too large/too small to fit in the destination operand, regardless of its size.

Auxilliary Flag indicates whether the result is too large/too small to fit in four bits.

Edit: How to determine AF: Explain how the AF flag works in an x86 instructions? .