If you are doing anything expensive, you should do it in a method, not a property.
Properties should be fast and inexpensive as this is the normal expectation by most programmers (principle of least astonishment).
In your case, don't have a getter - use:
public MyExpensive GetExpensive()
{
LazyInitializer.EnsureInitialized(ref _expensive, () = > new MyExpensive ());
return _expensive;
}
And of course, you have have parameterized overloads.