To convert 64.96 to octal, first decide how many places past the "octal point" you want show. Let's say you want 3. Then multiply your number by 8^3 or 512.
64.96 * 512 = 33259.52
Discard the fractional part. This, by the way is how far off of 64.96 your representation will be. Your answer will be 0.52/512 = 0.001015625 too small.
Now, for the conversion. Repeatedly divide your number (33259) by 8, keeping track of the remainders:
33259 / 8 = 4157 remainder 3
4157 / 8 = 519 remainder 5
519 / 8 = 64 remainder 7
64 / 8 = 8 remainder 0
8 / 8 = 1 remainder 0
1 / 8 = 0 remainder 1
Now, take your remainders in reverse order and insert the "octal point" 3 from the end:
100.753