I have not created a CustomMembershipProvider personally (using MVC4 based SimpleMembershipProvider). However looking at the information around can tell you that when you implement your CustomMembershipProvider deriving from MembershipProvider, you will implement below method.
public override void Initialize(string name, NameValueCollection config)
Initialize() method will be called by your application root (unless you want to control lifetime based on your different need).
For example if you create a new MVC4 web application internet template in VS2012, You will see that AccountController has an attribute "InitializeSimpleMembershipAttribute" applied to it. This attribute ensure that the Membership provider is initialize and initialize only once per application lifetime.
You may see this initialization done in different way e.g. Register your attribute in RegisterGlobalFilters() method which is setup during application startup in Global.asax.cs, or register a start method using and initialize membership provider there.
assembly: WebActivator.PreApplicationStartMethod
Based on how you initialize your custom membership provider, you will see when it is disposed off. If it is initialized during the application startup as part of application initialization code then it will be disposed of during application tear down or in a method that you register with "ApplicationShutdownMethodAttribute".
Please check this link for more information on how to create a custom membership provider, might be helpful.