Is it the best practice to run user profile service in WFE servers and search with user profile service?

Question

Hi In new SharePoint 2016 Farm I have below topology

• 2 APP Servers((Application With Search ROLE) = User Profile Service + and Search Host Controller Service, Search Query and Site Settings Service & central admin
• 2 WFE Servers (Front end with DCache )= Distributed Cache, Managed Metadata Web Service Microsoft SharePoint Foundation Web Application, User Profile Service

Is it the best practice to run user profile service in WFE servers and search with user profile service?

I get below errors in App servers and search service not working

he Execute method of job definition Microsoft.Office.Server.UserProfiles.FeedCacheRepopulationJob (ID c1835671-c88c-4821-bf32-7d0d639c714b) threw an exception. More information is included below.

Unexpected exception in FeedCacheService.IsRepopulationNeeded: Failed to Decrypt data...

The Execute method of job definition Microsoft.Office.Server.UserProfiles.LMTRepopulationJob (ID 6318bf99-45ef-425c-9c9b-61d50fdce0d8) threw an exception. More information is included below.

Unexpected exception in FeedCacheService.IsRepopulationNeeded: Failed to Decrypt data..

Answer 1

You need "Application with Search" MinRole for your app servers or custom role otherwise your server will Non-Compliant and for the WFE you need "Front-end with Distributed Cache" otherwise your wfe will be non-complaint . I dont think you need 4 UPA instances (as you have 2 on app servers and 2 on WFE), 2 instances on WFE are enough. Other than that I am not seeing any problem and things as per the MSFT guideline.

You can read more about it here: Description of MinRole and associated services in SharePoint Servers 2016 and 2019

Answer 2

SharePoint's 2016 MinRole concept starts a lot of services on Application servers as well as on WebFrontEnd servers for performance-reasons. This includes UserProfileService and so you have 4 running instances.

Spencer Harbar describes the details in this excellent article:
SharePoint 2016 Nugget #1: Topology Service in MinRole Farms

In previous versions of SharePoint (i.e. 2010 and 2013) the Application Discovery and Load Balancing Service (aka the Topology Service) was deployed to every server in the farm. This service amongst other things is responsible for maintaining a list of addresses of service application endpoints. These are used for Web Application <-> Service Application and Service Application <-> Service Application communication. In the previous versions, it would be extremely likely to go to another machine even if the requested service was running locally. It was a basic round robin style approach.

In SharePoint 2016 this has been changed to always “prefer local”. In other words if the requested service is running locally, go there. Instead of hoping over to another machine. If that isn't possible then it will go to a remote server. This change was implemented after the Product Group was able to measure the positive impact of the “prefer local” model whilst running SharePoint Online. And the change has been baked into SharePoint 2016. This is one of a number of small but significant changes under the hood, which mean SharePoint 2016 can perform significantly better than SharePoint 2013. This is exactly the sort of engineering improvement needed across the product in broader terms.