Doesn't this defeat the whole idea of keeping the layers seperate, because now when the network layer starts using addresses of a different size, the transport layer needs to change as well.
Yes. And indeed, this is why RFC 2460 redefines the pseudo-header for IPv6:
Any transport or other upper-layer protocol that includes the addresses from the IP header in its checksum computation must be modified for use over IPv6, to include the 128-bit IPv6 addresses instead of 32-bit IPv4 addresses. [...]