You are more or less right. Routers besides adding MF (more fragments) flag are doing some more processing:
To fragment/segment a long internet packet, a router (R1 in the figure below) creates a new IP packet and copies the contents of the IP header fields from the long packet into the new IP header. The data of the long packet is then divided into two portions on a 8 byte (64 bit) boundary, so that the first packet is less than the MTU of the out-going interface. The more-fragments flag (MF) in the first packet is set to one (to indicate that more fragments of this packet follow). The More Flag may already be set in this packet if it has already been fragmented by another system. This packet is forwarded.
The second created new packet is then processed. The packet header field is identical to that of the original packet (including the same value of the packet ID, the total length field, the more-fragments flag (MF) and the fragment offset field in the original packet). The packet header field is updated with a new offset field, by adding the number of payload bytes sent in the first fragment. If this new packet is larger than the allowed link MTU, the packet is again fragmented.
So in nutshell for already fragmented packets whole IP header is being copied to new packets (smaller fragments of fragment). It means that if IP packet has MF set it will be also copied to last fragmented packet. Thanks to changing offset and packets ID host receiving fragmented packets will be able to reassemble those.