Why am I unable to view stats in the WSO2 API Manager Store?

Question

Everyone please note that this question has been updated as I've recently had logs to work with. Below the log is the original posting with my configs, and the behavior I was seeing. Today, WSO2 APIM continues to send to BAM receivers which update the Stats database. However when I click on any of the statistics link in the publisher I get:

TID: [0] [AM] [2014-03-04 13:43:18,815] ERROR {org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject} -  Error while invoking APIUsageStatisticsClient for ProviderAPIUsage {org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject}
org.wso2.carbon.apimgt.usage.client.exception.APIMgtUsageQueryServiceClientException: Error occurred while querying from JDBC database
        at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.queryFirstAccess(APIUsageStatisticsClient.java:1747)
        at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.getFirstAccessTime(APIUsageStatisticsClient.java:1675)
        at org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject.jsFunction_getFirstAccessTime(APIProviderHostObject.java:2911)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
        at org.mozilla.javascript.FunctionObject.call(FunctionObject.java:386)
        at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42)
        at org.jaggeryjs.rhino.publisher.modules.statistics.c1._c_getFirstAccessTime_13(/publisher/modules/statistics/usage.jag:351)
        at org.jaggeryjs.rhino.publisher.modules.statistics.c1.call(/publisher/modules/statistics/usage.jag)
        at org.mozilla.javascript.ScriptRuntime.applyOrCall(ScriptRuntime.java:2430)
        at org.mozilla.javascript.BaseFunction.execIdCall(BaseFunction.java:269)
        at org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:97)
        at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42)
        at org.jaggeryjs.rhino.publisher.modules.statistics.c0._c_anonymous_13(/publisher/modules/statistics/module.jag:29)
        at org.jaggeryjs.rhino.publisher.modules.statistics.c0.call(/publisher/modules/statistics/module.jag)
        at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)
        at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0._c_anonymous_1(/publisher/site/blocks/stats/ajax/stats.jag:220)
        at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.call(/publisher/site/blocks/stats/ajax/stats.jag)
        at org.mozilla.javascript.optimizer.OptRuntime.call0(OptRuntime.java:23)
        at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0._c_script_0(/publisher/site/blocks/stats/ajax/stats.jag:4)
        at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.call(/publisher/site/blocks/stats/ajax/stats.jag)
        at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
        at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
        at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.call(/publisher/site/blocks/stats/ajax/stats.jag)
        at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.exec(/publisher/site/blocks/stats/ajax/stats.jag)
        at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:570)
        at org.jaggeryjs.scriptengine.engine.RhinoEngine.exec(RhinoEngine.java:273)
        at org.jaggeryjs.jaggery.core.manager.WebAppManager.execute(WebAppManager.java:432)
        at org.jaggeryjs.jaggery.core.JaggeryServlet.doPost(JaggeryServlet.java:29)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749)
        at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487)
        at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:379)
        at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339)
        at org.jaggeryjs.jaggery.core.JaggeryFilter.doFilter(JaggeryFilter.java:21)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
        at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
        at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
        at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
        at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
        at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
        at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
        at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: java.sql.SQLException: Incorrect syntax near 'limit'.
        at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:372)
        at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2988)
        at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2421)
        at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:671)
        at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQLQuery(JtdsStatement.java:505)
        at net.sourceforge.jtds.jdbc.JtdsStatement.executeQuery(JtdsStatement.java:1427)
        at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.queryFirstAccess(APIUsageStatisticsClient.java:1729)
        ... 63 more
TID: [0] [AM] [2014-03-04 13:43:18,836] ERROR {JAGGERY.modules.statistics.usage:jag} -  java.lang.NullPointerException: null {JAGGERY.modules.statistics.usage:jag}

I have configured WSo2 API Manager 1.6.0 and BAM 2.4.0 to both use the same datasource configuration for WSO2AM_STATS_DB. For the store it is a single WSO2AM_STATS_DB entry that matches the entry on the BAM server (below) and API-Manager.xml is updated to:

  <APIUsageTracking>
        <Enabled>true</Enabled>
<PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass>
        <ThriftPort>7612</ThriftPort>
        <BAMServerURL>tcp://myBAMserver:7612/</BAMServerURL>
        <BAMUsername>user</BAMUsername>
        <BAMPassword>pwd</BAMPassword>
        <DataSourceName>jdbc/WSO2AM_STATS_DB</DataSourceName>
        <GoogleAnalyticsTracking>
             <Enabled>false</Enabled>
             <TrackingID>UA-XXXXXXXX-X</TrackingID>
       </GoogleAnalyticsTracking>
    </APIUsageTracking>

and now the datasource for APIM:

<datasource>
    <name>WSO2AM_STATS_DB</name>
    <description>The datasource used for getting statistics to API Manager</description>
    <jndiConfig>
        <name>jdbc/WSO2AM_STATS_DB</name>
    </jndiConfig>
    <definition type="RDBMS">
        <configuration>
            <defaultAutoCommit>false</defaultAutoCommit>
            <url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_apiStatsdb</url>
            <username>wso2storeuser</username>
            <password>storepwd</password>
            <driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
            <maxActive>50</maxActive>
            <maxWait>60000</maxWait>
            <testOnBorrow>true</testOnBorrow>
            <validationQuery>SELECT 1</validationQuery>
            <validationInterval>30000</validationInterval>
        </configuration>
    </definition>
 </datasource>

For the BAM server here are my datasources:

<datasource>
        <name>WSO2_CARBON_DB</name>
        <description>The datasource used for API Manager database</description>
        <jndiConfig>
            <name>jdbc/WSO2CarbonDB</name>
        </jndiConfig>
        <definition type="RDBMS">
            <configuration>
                <defaultAutoCommit>false</defaultAutoCommit>
                <url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_carbondb</url>
                <username>user</username>
                <password>pwd</password>
                <driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
                <maxActive>50</maxActive>
                <maxWait>60000</maxWait>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1</validationQuery>
                <validationInterval>30000</validationInterval>
            </configuration>
        </definition>
    </datasource>

    <datasource>
        <name>WSO2AM_STATS_DB</name>
        <description>The datasource used for getting statistics to API Manager</description>
        <jndiConfig>
            <name>jdbc/WSO2AM_STATS_DB</name>
        </jndiConfig>
        <definition type="RDBMS">
            <configuration>
                <defaultAutoCommit>false</defaultAutoCommit>
                <url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_apiStatsdb</url>
                <username>user</username>
                <password>pwd</password>
                <driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
                <maxActive>50</maxActive>
                <maxWait>60000</maxWait>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1</validationQuery>
                <validationInterval>30000</validationInterval>
            </configuration>
        </definition>
     </datasource>

    <datasource>
        <name>WSO2BAM_DATASOURCE</name>
        <description>The datasource used for analyzer data</description>
        <definition type="RDBMS">
            <configuration>
                <defaultAutoCommit>false</defaultAutoCommit>
                <url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_apiStatsdb</url>
                <username>user</username>
                <password>pwd</password>
                <driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
                <maxActive>50</maxActive>
                <maxWait>60000</maxWait>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1</validationQuery>
                <validationInterval>30000</validationInterval>
            </configuration>
        </definition>
    </datasource>

        <datasource>
        <name>WSO2BAM_CASSANDRA_DATASOURCE</name>
        <description>The datasource used for Cassandra data</description>
        <definition type="RDBMS">
            <configuration>
                <url>jdbc:cassandra://localhost:9161/EVENT_KS</url>
                <username>admin</username>
                <password>admin</password>
            </configuration>
        </definition>
    </datasource>

    <datasource>
        <name>WSO2BAM_UTIL_DATASOURCE</name>
        <description>The datasource used for BAM utilities, such as message store etc..</description>
        <definition type="RDBMS">
            <configuration>
                <url>jdbc:cassandra://localhost:9161/BAM_UTIL_KS</url>
                <username>admin</username>
                <password>admin</password>
            </configuration>
        </definition>
    </datasource>

    <!-- The URL configs are loaded from cassandra-component.xml -->
    <datasource>
        <name>WSO2BAM_HIVE_INCREMENTAL_DATASOURCE</name>
        <definition type="RDBMS">
            <configuration>
                <username>admin</username>
                <password>admin</password>
                <dataSourceProps>
                    <property name="replicationFactor">1</property>
                    <property name="strategyClass">org.apache.cassandra.locator.SimpleStrategy</property>
                    <property name="readConsistencyLevel">QUORUM</property>
                    <property name="writeConsistencyLevel">QUORUM</property>
                    <property name="keyspaceName">HIVE_INCREMENTAL_KS</property>
                </dataSourceProps>
            </configuration>
        </definition>
    </datasource>

When I look in mydbserver/wso2_apiStatsdb I see that the following tables were created and have been updated with data:

API_REQUEST_SUMMARY, API_Resource_USAGE_SUMMARY update: some additional tables have been added and updated... API_FAULT_SUMMARY, API_RESPONSE_SUMMARY, API_VERSION_USAGE_SUMMARY

However when I go to the publisher page I get:

As far as I can tell I've updated the publisher/store and gateway/key manager to send data to the BAM server. The BAM server appears to process and put that summarized data in the sql server database. My entries for SQL server database match for the publisher/store config and the BAM server config. Does anyone have any ideas what I'm missing that will enable me to see the stats when in the publisher?

Answer 1

Gammonster, this has been figured out. It appears that the devil is indeed in the details - the runtime was trying to execute a sql query using the 'limit' keyword in a SQL Server database and SQL Server does not support 'limit'. It has 'top' instead.

Would have appreciated if the product documentation expressly identified SQL Server to not be a preferred DB instance for BAM.

Answer 2

Can u please check if the EnableBillingAndUsage is set to false. This is available in conf/api-manager.xml

<EnableBillingAndUsage>false</EnableBillingAndUsage>

Answer 3

Are you running the BAM on port offset 1? As far as I can see you have BAM running on a different server. Try running BAM with port offset 1. The default config provided in the site is for a usecase where API Manager and BAM runs on the same server with port offsets (API M -0 and BAM -1). Hence all the relevant thrift ports are also incremented by 1 in the default config. Hence I suggest you to run BAM on offset 1.

In the meantime pls check if the MQSQL DB that has summarized stats contain any data inside them.

Answer 4

This can also happen if you haven't configured the summarized database correctly in API Manager. From what you have mentioned, API Manager seems to publish data correctly, but it's not fetching summarized data. Please see if you have defined WSO2AM_STATS_DB in master-datasources.xml inside APIM directory.