There are three different designs and all are used.
Exclusive: Data in the L1 cache is never in the L2 cache. Data in the L2 cache is never in the L1 cache.
Inclusive: Data in the L1 cache must also be in the L2 cache.
Neither: Data in the L1 cache may or may not be in the L2 cache.
Each of these has advantages and disadvantages. The inclusive scheme allows the cache coherency protocol to ignore the L1 cache -- if data isn't in the L2 cache, it isn't in the L1 cache. But the exclusive scheme makes the most effective use of precious cache memory.
Inclusive designs are becoming more popular because faster inter-core synchronization is becoming more important than having slightly larger effective cache sizes.