If you call the command kextstat, it will display a list of all the loaded kernel extensions with info like this one:-
115 3 0xffffff7f814f4000 0x32000 0x32000 com.apple.iokit.IOAudioFamily (1.8.9fc11) <114 5 4 3 1>
The first number (115) is the id of the kernel extension.
At the end, the set of numbers <114 5 4 3 1> are the other kernel extension ids that are being referenced by this particular kext.
If you match the id of your kext to any of those listed in another, then that is a reference that kextunload is referring to.
Also note that when writing any code in the kernel, globals variables are global across the whole kernel, so if you've declared any that may not have a unique name and is used by another kext, I have found this to cause a similar issue. The solution here is to prepend any globals with a reverse company URI (e.g. com_apple_globalVarName).