You will need:
1 - Boards: I started with a beagleboard. A new beaglebone black is available now. There's a large support community for the beagles; many howto pages here and here, and ready to install images. You can also build the image yourself (step 3). These boards have most of the peripherals that you may need to play with, and can also be used as extra computer !
2 - Books: As Mali noted, Linux is a fast moving object, but in the early phases of learning you will need a solid reference. I'd suggest "Embedded Linux Primer: A Practical Real-World Approach" as it has many examples and takes you step-by-step. There's also "Building Embedded Linux Systems".
3 - The firmware: a) toolchain, b) root filesystem and c) kernel image. "Buildroot" is the easiest to start with. Openembedded and Yocto have a very steep learning curve. "Embedded Linux Primer: A Practical Real-World Approach" has some examples on how to use buildroot.