Ssh issues connecting to Google Compute Engine

Question

I've just updated a few servers to the newer version of the gce stack, and I've started having some ssh issues that I'm not quite sure how to fix. I've already looked into issues with the firewall, and the ssh docs aren't a huge help.

This is how I'm currently connecting:

gcutil --service_version="v1" --project="myproject" ssh  --zone="us-central1-a" "myproject-prod"

which was working up until very recently. I was doing some bash hacking, adding and removing a number of apt and pip packages, so I assume it has something to do with that, but I'm really not sure. When I try to connect with the above code I get the following error:

INFO: Running command line: ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -i /home/user/.ssh/google_compute_engine -A -p 22 user@108.59.84.53 --
ssh: connect to host 108.59.84.53 port 22: Connection refused

My firewalls seem to be in order:

user@computer:~$ gcutil --project="myproject-backend" listfirewalls
+------------------------+---------+
| name                   | network |
+------------------------+---------+
| default-allow-internal | default |
+------------------------+---------+
| default-ssh            | default |
+------------------------+---------+
| http2                  | default |
+------------------------+---------+

Any thoughts or resources on how to resolve this issue?

Answer 1

Is the VM connected to the default network?

If not, is the ssh-key that you are using (/home/user/.ssh/google_compute_engine) entered into the metadata section for the VM?

Answer 2

I suggest to look into serial console first and check for obvious messages like failed SSH service startup failure. You can also create a snapshot of your boot disk -> create a new Persistent Disk out of it -> mount it on a temp instance and review logs/startup scripts, etc.

Answer 3

We have created a startup script to self-manage and troubleshoot ssh connectivity issues https://github.com/GoogleCloudPlatform/compute-ssh-diagnostic-sh/