The error your getting typically indicates that the account you're using for the remote connection is not a member of the necessary RBAC Role Group in Exchange.
When the remote session is initialized, the server will check which Roles the account making the connection has, and create a set of proxy functions in the session customized for that set of role memeberships. If the account doesn't belong to a Role Group that manages public folders, you will not have proxy functions for the public folder management cmdlets available in the session.