You were looping through the rows in your range and also advancing the variable i
within your loop.
You can reference each variable that you are looping through.
Try this
Set fillcolumnrange = Sheet1.Range("A5:A20")
For Each cell In fillcolumnrange.Cells
If Not cell = "" Then
MsgBox cell
End If
Next cell