Given your clarifications, you need to choose your game here. The most straightforward solution is indeed running a Dijkstra algorithm and building the shortest paths from any waypoint to any other waypoint (this means a single algorithm run for each possible source point). Note however that this is only done once (at least until some waypoint is moved or removed). By the way A* isn't related here, it's intended to run on decision/game trees and find min/max optimal path (max benefit for you, min for the opponent), that's a different story. Another alternative is to run Floyd–Warshall which find all paths in a single run.
Anyway, once you ran that, you can keep for each waypoint a table, saying what is the next node in the shortest path for each destination waypoint. You don't need to entire path, just the next vertice. Once the robot gets there, it's told where to go next. This is basically the same thing done in most network routing algorithms.
Now, if you want to showcase how the robots are working, you could make them run this algorithm online and calculate the paths for themselves, but then the communication with the waypoints is rather pointless.
Either way, looks like you're having some fun coming your way, enjoy :-)