On Windows child processes normally live on their own and once started they do not depend on their parent process. You are looking for job objects. With jobs you can control an entire process tree lifetime, all child processes can be deterministically terminated if the parent ends (by having the parent own the job, strictly speaking all child processes terminate if the job is killed). There is no managed .Net API for it, but p-Invoke works fine.
So if you experience unexpected "strange behavior" make sure your process is not launched in the context of a job, causing your child processes to also be part of the job. Process Explorer can show job properties of a process.