Instead of using a Main method to start the server, I solved this by changing the Procfile of Heroku using below.
web: java $JAVA_OPTS -jar target/dependency/jetty-runner.jar --port $PORT target/*.war
Hope this helps anyone facing same issue.