Openshift: Why can't I add a node to a district?

Question

I'm trying to add a node to a district:

[root@broker ~]# oo-admin-ctl-district -c add-node -n small_district -i node1.example.com
ERROR OUTPUT:
Node with server identity: node1.example.com is of node profile '' and needs to be 'small'     to add to district 'small_district'

But, when I go to the node, it seems to know that it should be a small:

[root@node1 ~]# grep -i profile /etc/mcollective/facts.yaml
node_profile: small

I ran oo-diagnostics on the broker and got:

[root@broker ~]# oo-diagnostics 
FAIL: test_node_profiles_districts_from_broker
      No node hosts found. Please install some,
      or ensure the existing ones respond to 'mco ping'.
      OpenShift cannot host gears without at least one node host responding.

FAIL: run_script
oo-accept-systems -w 2 had errors:
--BEGIN OUTPUT--
FAIL: No node hosts responded. Run 'mco ping' and troubleshoot if this is unexpected.

1 ERRORS

But mco ping shows no problems:

[root@broker ~]# mco ping
node1.example.com                        time=106.82 ms


---- ping statistics ----
1 replies max: 106.82 min: 106.82 avg: 106.82 

I also found https://lists.openshift.redhat.com/openshift-archives/users/2013-November/msg00006.html, which lists the same error message. However, I already have everything in /etc/mcollective/facts.yaml that the thread suggets:

[root@node1 ~]# grep 'node_profile' /etc/mcollective/facts.yaml 
node_profile: small

What could be preventing the node from being added to the district?

Answer 1

The problem was a misconfiguration of the node. https://bugzilla.redhat.com/show_bug.cgi?id=1064977 should resolve the documentation issue that led to this.

Resolution was to update /etc/mcollective/server.cfg:

[root@node1 ~]# git diff --color /etc/mcollective/server.cfg.old /etc/mcollective/server.cfg
diff --git a/etc/mcollective/server.cfg.old b/etc/mcollective/server.cfg
index c614ed9..fff36c5 100644
--- a/etc/mcollective/server.cfg.old
+++ b/etc/mcollective/server.cfg
@@ -22,4 +22,5 @@ plugin.activemq.pool.1.password = marionette

 # Facts
 factsource = yaml
-plugin.yaml = /opt/rh/ruby193/root/etc/mcollective/facts.yaml
+plugin.yaml = /etc/mcollective/facts.yaml