You're asking about vector projection.
Given two vectors A
and B
, what is A
projected onto B
?
In your case, A
seems to be the difference between red and pink, where B
is what you're calling a custom axis.
Calculating this projection usually involves a dot product. Lucky for you, Unity provides Vector3.Dot
to make this easy.
We can calculate the projection as a scalar. A is "this much" in the direction of B:
float projScalar = Vector3.Dot(A, B.normalized);
This gives us the length you're asking about.
If needed, we can convert that result into a vector, by casting that length in the direction of B:
Vector3 projVector = B.normalized * projScalar;