It is definitely possible to flash Arduino code into your own creations. I have done this several times. However you need an ISP to flash the target.
Depending on the hardware of your PCB some libraries may not work or not deliver the desired results.
Everything that relies on exact timing requires a crystal oscillator. (However the newer Arduinos only have a resonator, thus this might not be an issue for you).
Everything that needs to run at 16 Mhz requires a crystall as the internal PLL only supports up to 8 Mhz.
Everything that requires serial IO must be compiled with matching frequencies.
--> Unless you know what you do it is recommended to have a 16 Mhz crystall oscilator.
The serial to USB converter is purely optional but if you ommit it you will not be able to use the IDE's serial monitor or the boot loader option.
In all cases you have to ensure that you set the fuses properly. If you flash a bootloader with the IDE the IDE will automatically set the fuses properly. Since you need an ISP anyway you might want to consider to remove the bootloader afterwards. But this is mostly a matter of personal preferences.