The only way I was able to solve this is to blur the contenteditable
div:
CKEDITOR.currentInstance.element.$; //Grab the DOM node which is the contenteditable
CKEDITOR.currentInstance.element.$.blur(); //Blur it
CKEDITOR.currentInstance.focusManager.blur(true); //CKEditor's blur hides the toolbar