All you need to use is appAPI.pageAction.setPages.setPages setting the icon and page filter in the options passed to the method. In your extension, I can see that you are using appAPI.pageAction.setResourceIcon which has the effect of setting the icon on every page and hence your issue.
To achieve your goal, simply use something like the following code in your extension.js file:
appAPI.ready(function($) {
appAPI.pageAction.setPages({
pages: ["*.google.com/*", /^http\:\/\/twitter\.com/],
iconResourcePath:'icons/icon.jpg',
html: '<html><head></head><body>Hello World!</body></html>',
height: 100,
width: 300
});
});
[Disclosure: I am a Crossrider employee]