I think it is considering the possibility that the call to Install() fails and then the container, which is not assigned to any variable yet, is not disposed. You can add a try/catch there, making the expression much more verbose but compliant with the rule like (not tested, but you'd get the idea):
private readonly Lazy<IWindsorContainer> LazyContainer =
new Lazy<IWindsorContainer>(() => {
var container = new WindsorContainer();
try { container.Install(new WindsorInstaller())); }
catch { using(container) {} throw; }
return container; });