It shouldn't be a problem. Heroku has a guide to upgrade with backups, so I'd recommend taking a backup of your current database, downloading it to your local computer, then spinning up a development database.
Once the development (free) DB is ready, restore from the pgbackup on your local. As long as you're under the row limit, you should be fine.
Obviously, you'd want to put the site in maintenance mode when you do all of this - but it shouldn't be down for more than 5-10 minutes.