No, we don't violate.
When we tell the Factory class to create an object instance, all the responsibility is within the Factory class. The caller has no influence what the concrete class will be.
The Factory class itself also doesn't break that rule. It is doing what it has to do: based on the given rules (let it be hard-coded or externally set, or maybe something more complex) decides what kind of object to generate. But every bit of the logic is inside the factory method.