INDIRECT will work and is closest to the solution you described, but I prefer OFFSET, which uses proper references. (For example, if you insert a column in the sheet before L, INDIRECT will break while OFFSET will just update its reference as expected.
Two ways to go with OFFSET:
1 - Start at L$1 and go down $V$4-1 rows. (This will work with $V$4 as you've defined it now.)
=MIN(L12:OFFSET(L$1,$V$4-1,0))
2 - In $V$4, provide the height of the range you want.
=MIN(OFFSET(L12,0,0,$V$4,0))
It's hard to make suggestions without more context, but I'm sure you can tweak one of these patterns to meet your needs.