The approach you are describing is pretty much a DFS - choose one branch - and develop it as much as you can.
Another common graph discovery algorithm is BFS - "discover" all nodes at distance 0 from the source, then at distance 1, .... until all graph is discovered.
Note that a variation of DFS, that holds a dynamic 'visited' set (and allows to rediscover nodes that are already discovered, but not in the current branch) can be used for cycles detection (assuming you want all cycles), but may take a long time to run - because there could be exponential number of cycles in the graph