There is not a direct way that I know of to see what company file is currently open (if any) without calling BeginSession and checking for errors. If you supply a company file name and a different company is open, you will get the "A QuickBooks company data file is already open and it is different from the one requested." error.
If you omit the company name when you call BeginSession, QuickBooks will use whatever company file is open and present the prompt(assuming the rights have not already been granted). However, if there is not a company file open, then you get an error "Could not start QuickBooks." (if QuickBooks isn't running at all), or "If the QuickBooks company data file is not open, a call to the "BeginSession" method must include the name of the data file." error if QuickBooks is open, but has no company file open.
Most programs will save the company file that they have been linked to, so they will pass the file name in their BeginSession call, and then check for the "A QuickBooks company data file is already open and it is different from the one requested." error and present the information in a clean way to the customer. For example, the QuickBooks POS software will prompt the customer if they want to continue using the old file that was setup previously, or if they want to link to the file that is currently open.