One of the current maintainer of the RGeo gem. We're improving documentation, and I've edited the part mentioning geos installation on heroku.
To avoid any link rot, here's a solution that should work:
echo 'libgeos-dev=3.7.1-1~pgdg18.04+1' > Aptfile # Use the version you want here
git add Aptfile
git commit -m 'Add Aptfile with libgeos'
heroku buildpacks:add --index=1 heroku-community/apt
# If you already have installed the gem earlier, you will need to purge your repo cache:
# heroku plugins:install heroku-repo
# heroku repo:purge_cache
git push heroku main
heroku console <<< 'puts "RGeo is configured with Geos !" if RGeo::Geos.capi_supported?;exit'
And if you also want support from the standard geographic libraries (such as Proj4), you should consider using the heroku-geo-buildpack. This will be a bit slower and you'll have a bigger slug since it will install libraries you may not need.