Overflow flag cleared after SBC with overflow in 6502 emulation?

Question

I've "finished" emulating my 6502 processor, and I'm in the testing phase right now. Being the beginner that I am, I've been stuck on this one issue for a couple of hours already. I'm following an NES test which basically is just a homebrew ROM and a log someone made, and it says what value is supposed to be in every register after a step through the memory. I'm stuck on one part where the log is showing that the overflow flag is cleared after an immediate SBC instruction, even though the accumulator has the value 0x40, and first argument (memory pc + 1) is equal to 0x41. That means that 0x40 - 0x41 should be 0xFF, i.e. -1, so that means that there was an overflow, right? I read some article about the overflow flag, and it states that an overflow is produced when a value is too big or too small to be held in a signed byte, and therefore overflows to the other side (i.e. 2 8 bit positives become an 8 bit negative, and vice versa). So am I thinking the wrong way? This is the log line:

CBC6  E9 41     SBC #$41    A:40 X:AA Y:73 P:E5 

The line after that states that the P register has become 0xA4, which would mean the carry and overflow flags are both cleared. I get the carry part, but not the overflow part. This is the next line, if care for it:

CBC8  20 62 F9  JSR $F962   A:FF X:AA Y:73 P:A4

By the way; that second line proves my point: the A register has indeed become 0xFF, and 0xFF in a signed byte would equal 256 - 255 = -1........

Answer 1

As you correctly say, overflow is for signed operations if the result doesn't fit. Note that a signed byte can hold values between -128 and +127, and as such -1 fits nicely in that range. That it has crossed zero is not relevant for signed overflow because 0 is in the middle of the representable range. It's only interesting for unsigned carry. For overflow the boundary is at 0x80 (-128) which in turn is irrelevant for unsigned operations.