By nesting one openquery within another, you are telling server2 to run an openquery to server1, which you have stated you cannot do. There are also errors above, in that you are nesting @string into @sql, but you are executing @string, not @sql.
If you are expecting a relatively small return, I'd suggest using temp tables to hold the data from server1, then filter the return from server2 using that.
IF OBJECT_ID('TEMPDB..#TMP') IS NOT NULL
DROP TABLE #TMP
SELECT COL1
INTO #TMP
FROM OPENQUERY(SERVER1, 'SELECT COL1 FROM TABLE1 WHERE COL2 = ''A''')
SELECT *
FROM OPENQUERY(SERVER2, 'SELECT * FROM TABLE2')
WHERE COL1 IN (SELECT COL1 FROM #TMP)
As a followup edit, if the return from server2 is large, you should consider filtering it before it returns to your local server so you're not transferring massive amounts of data over the wire. In this case, I'd convert the values of #tmp into a comma separated string, and nest that into the 2nd openquery in a variable, and execute it with sp_executesql. Here is a great SO post on converting a column to a comma separated list.