It is not possible; Silverlight does not support Entity Framework libraries.
Silverlight applications can not connect to SQL server directly; you only have one option: Creating web services (or WCF) to expose you SQL server operations to Silverlight client.