Short answer - you can't.
Long answer:
Consider these two trees:
A
/ \
B C A
/ \ / \ / \
D E F G B C
/ \ / \ / \ / \
H I J K L M N O
They have the same balance factor, but they aren't the same height.
So, if you only have the balance factor of the child, you don't know how high that subtree is, thus you can't use only that to calculate the balance factor of the parent.