You need to set you $ErrorActionPreference to stop:
$ErrorActionPreference = "Stop"
By default this is set to Continue, so when Powershell is called from SQL Agent the job will continue on error unless you change the setting.
I have a few more tips on calling Powershell from SQL Agent here: http://blogs.technet.com/b/heyscriptingguy/archive/2013/05/06/10-tips-for-the-sql-server-powershell-scripter.aspx