That's controlled by your line join style, not your line cap style.
CGContextSetLineJoin(context, kCGLineJoinRound);
The default line join style is miter, and it looks like you're hitting the miter limit, which is why it becomes round at some angle. (See CGContextSetMiterLimit).