You need to replicate the functionality in your code - if you change the name property, NHibernate will not by default send a new query to the database to evaluate the generated properties.
You need to do something like:
public virtual string subName { get { return name.Substring(0, 5); } set { } }
Note that you will need the empty setter otherwise NHibernate will complain
Edit: I see that you want to index the column in the database, so you will need to create a computed column for that table in the database. This itself has nothing to do with NHibernate, and you can either ignore the column entirely in your mapping (so long as you replicate it in your code), or drop the .Formula(..) part (since that's specified in the computed column already)