I've solved the problem, but I'm no Red5 expert so take the following explanation with a grain of salt and correct me if I'm wrong... It seems that either Red5, or the "live" application included with Red5, attempts to instantiate whatever objects it receives via a SharedObject on the server-side. So, I believe you would actually need to replicate in Java whatever CustomClasses you're sharing from the client-side in AS3. Seems really odd to me, but that's all I can gather, and that would explain the Red5 error logs.
This was unacceptable for my needs, as I plan to have other programmers take my client-side code as an API and abstract away the server-side details... so recreating whatever classes they come up with on the server isn't really feasible.
An easy workaround is to "wrap" whatever you're sharing in a ByteArray and then unwrap it when you retrieve it. Red5 doesn't seem to complain about ByteArrays, and happily broadcasts them out to the client with no knowledge of what's inside. Here are a couple functions I whipped up to handle this...
private function setRed5Property(propertyName:String, data:Object):void
{
var dataBytes:ByteArray = new ByteArray();
dataBytes.writeObject(data);
_red5SharedObject.setProperty(propertyName, dataBytes);
_red5SharedObject.setDirty(propertyName);
}
private function getRed5Property(propertyName:String):Object
{
var dataBytes:ByteArray = new ByteArray();
dataBytes.writeBytes(_red5SharedObject.data[propertyName]);
dataBytes.position = 0;
return dataBytes.readObject() as Object;
}