As stated by @ChrisK, to correctly build a Console Application that will return an error message to SQL Server Agent, the code must use a try/catch and write the error message to the Console.Error
Writing the code that way solved my problem:
static int Main(string[] args)
{
try
{
var x = 0;
var y = 1 / x;
}
catch (Exception ex)
{
Console.Error.WriteLine(ex);
return 1;
}
return 0;
}