In traditional terminology, the "input layer" is a set of neurons that output the inputs, so they can be consumed by further layers that actually do computations. Input neurons don't do computations, since they don't have any input to do computations upon.
So your understanding is correct. In the "traditional" diagram, the input layer is not a "compute layer". You need to only implement the actual "compute layers".
From what I read in your post, it seems that the AForge library may simply be using different terminology, i.e. it may be refering to the first compute layer, the one that is reading the inputs, as the input layer.