I found and solved the problem. When checking the error message more carefully, I noticed it said it cannot load Newtonsoft.Json.dll version 4.5.0 However, in my project I used nuget package for NewtonSoft, which was version 6.0.2
This leaded me to check what actual version of NewtonSoft the api.dll was built against.
Luckily I had access to api dll source, and I noticed it was built against an older version of Newtonsoft, 4.5.0 for NET 3.5
Even if there was the app.config for dll that defined the version redirection, it seems it didn't worked when the COM-enabled DLL was accessed from COM client - MS Access.
But after I rebuilt the api.dll to use the actual current version of NewtonSoft dll, all started to work ok even from COM call