Not sure what the UNIX prompt defaults too but this should do what I think you want it to do.
powershell -noexit -command "function prompt {'{0}$ ' -f $pwd}"
If you use single quotes in the prompt function the $
doesn't get interpolated, and you don't have to worry about to many quotes.
Like any function, the Prompt function exists only in the current session. To save the Prompt function for future sessions, add it to your Windows PowerShell profiles. For more information about profiles, see about_Profiles.
Here's how to create a new profile:
if (!(test-path $profile))
{new-item -type file -path $profile -force}
notepad $profile