Can't open MS Office documents from SharePoint

Question

We have an on-premise SharePoint Server 2016 (Vers. 16.0.4690) installation with multiple web applications installed.

One of these web applications is FBA-enabled.

I can sign-in to the SharePoint site with Windows- or Forms-Based-Credentials through any browser (Chrome, IE, Edge).

Since a few days, out of nowhere, I get the following window, when I try to open a MS Word or Excel document from a library in this Web Application:

(Anmelden = Sign-In)

When I choose Windows-Authentication, the window just reloads. If I choose Forms-Based-Authentication, I can enter credentials, but I'm unable to login.

If I close the window, the document won't be opened.

In another web application, where FBA is not enabled, I can open documents without any problem.

The problems occurs on every client machine (equipped with Win 7 / Win 10 and Office 2016)

What I have done so far:

• Set "EnableADAL" registry-key on the client machine to 0
• Installed the latest updates on the SharePoint Server (Windows Server 2016) and the client machines
• Checked, that "Client Integration" is enabled in the web appliction
• Added the site to the Trusted Intranet Sites.
• Enabled IE Protected Mode for "Local intranet"
• Tried to open documents from Google Chrome, Edge and Internet Explorer 11
• Tried to open the documents from within the Office application -> same behavior

Has anybody an idea, how I could solve this?

Best regards

Sebastian

Answer 1

Chris Coulson has a great explanation: Important – Additional Steps for enabling FBA for Office Applications

Even with FBA configured in SharePoint 2016, it still won’t authenticate properly when opening a document from SharePoint with Office 2016 (Word, Excel, PowerPoint…). Office 2016 now uses “Modern Authentication” by default – which does not support Claims authentication. SharePoint 2016 releases from the June 2016 CU and onward allow Modern Authentication to be turned off. To turn it off run the following in the SharePoint 2016 Management Shell:

$sts = Get-SPSecurityTokenServiceConfig
$sts.SuppressModernAuthForOfficeClients = $True
$sts.update()
iisreset

Also, I had the same question.

Answer 2

I've seen this recently on this blog telling at Step 7:

Let’s choose Windows Authentication first, since we did not give explicit permissions to the FBA User yet, that is why system won’t allow the user to login to the SharePoint Site. And being an authenticated Windows user, we are good to go and able to see the Web application homepage.

I'm not sure but this seems to be a question of setup of user permissions. Hope it helps.