Following a similar question last year I compiled the techniques and technologies used by some of the larger social networking sites.
The following architecture concepts are prevalent among such sites:
Scalability
- Caching (heavily, across multiple tiers and layers)
- Data Sharding (preferrably by data-locality criteria)
- In-Memory DBs for often referenced data
- Efficient wire-level protocols (as opposed to what an enterprise typically considers state of the art)
- asynchronous processing
Flexibility
- service oriented architecture as a baseline principle
- decoupled and layered components
- asynchronous processing
Reliability
- asynchronous processing
- replication
- cell architecture (independently operated subsets, e.g. by geographical criteria)
NB: If you start a new site, you are unlikely to have the kind of scaling or reliability requirements that these extremely large sites face. Hence the best advice is to start small but keep it flexible. One approach is to use an application framework that starts out simple but has flexibility to scale later, e.g. Django.