You have to redesign your code to use Dependency Injection. In your case define ILeaveFactoryCategory with GetCategory method. Make your ProRate class dependent on it (for example set factory by constructor parameter). Then mock the factory interface not calculator itself and set expectations for them. Use mocked object as parameter for class under test (LeaveCalculator). Verify expectations for you mocked object.
ILeaveCalculation expectedCalculator = new MyCalculator();
LeaveCalculationType expectedCalculationType = LeaveCalculationType.MyType;
ILeaveFactoryCategory factoryMock = MockRepository.GenerateMock<ILeaveFactoryCategory >();
factoryMock.Expect(f => f.GetCategory(Arg<LeaveCalculationType>.Is.Equal(expectedCalculationType)).Returns(expectedCalculator);
var proRataClass = new ProRata(factoryMock);
var availableLeaveCount = proRataClass.Calculate(employee, systemDateTime.Now.Date, leaveCount);
factoryMock.VerifyAllExpectations();
This code verifies that factory was used not the result of calculation. If you want to test results it's better to use Stub method instead of Expect and verify calculation result instead of expected behavior.