APIM Gateway tracks the analytics data if analytics is enabled and configured. But in order to use this feature APIM analytics need to be configured. If analytics is correctly configured these data processed and generate summary data that can be visualized in publisher and store statistics apps. But there can be requirements to get these analytics data without configuring apim analytics. Because adding another analytics server or deployment to the existing setup may need additional changes and required additional resources. As a solution, you can get these data to a file similar to the logging.
For this, we can add a custom event publisher using existing extension points and publish these data to the logger instead of the analyzer. So we can use the APIM component to a maven project and apply the configuration in the gateway node.
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.rukspot.samples</groupId> <artifactId>com.rukspot.samples.log_analytics</artifactId> <version>1.0-SNAPSHOT</version> <dependencies> <dependency> <groupId>org.wso2.carbon.apimgt</groupId> <artifactId>org.wso2.carbon.apimgt.usage.publisher</artifactId> <version>6.4.50</version> </dependency> </dependencies> <repositories> <repository> <id>wso2-nexus</id> <name>WSO2 internal Repository</name> <url>http://maven.wso2.org/nexus/content/groups/wso2-public/</url> <releases> <enabled>true</enabled> <updatePolicy>daily</updatePolicy> <checksumPolicy>ignore</checksumPolicy> </releases> </repository> </repositories> </project>
package com.rukspot.samples.analytics.publisher; import com.google.gson.Gson; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.wso2.carbon.apimgt.api.APIManagementException; import org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher; import org.wso2.carbon.apimgt.usage.publisher.dto.AlertTypeDTO; import org.wso2.carbon.apimgt.usage.publisher.dto.FaultPublisherDTO; import org.wso2.carbon.apimgt.usage.publisher.dto.RequestResponseStreamDTO; import org.wso2.carbon.apimgt.usage.publisher.dto.ThrottlePublisherDTO; public class FilePublisher extends APIMgtUsageDataBridgeDataPublisher { private static final Log log = LogFactory.getLog(FilePublisher.class); private Gson gson = new Gson(); public FilePublisher() { // Do nothing } @Override public void init() { // Do nothing } @Override public void publishEvent(FaultPublisherDTO faultPublisherDTO) { log.info(gson.toJson(faultPublisherDTO)); } @Override public void publishEvent(ThrottlePublisherDTO throttPublisherDTO) { log.info(gson.toJson(throttPublisherDTO)); } @Override public void publishEvent(AlertTypeDTO alertTypeDTO) throws APIManagementException { // Required only to log alert configurations // log.info(gson.toJson(alertTypeDTO)); } @Override public void publishEvent(RequestResponseStreamDTO requestStream) { log.info(gson.toJson(requestStream)); } }
<Analytics> <!-- Enable Analytics for API Manager --> <Enabled>false</Enabled> <!--Skip workflow data publisher initiation and even publishing--> <SkipWorkflowEventPublisher>true</SkipWorkflowEventPublisher> <!-- API Usage Data Publisher. --> <PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass> ...... </Analytics>
log4j.category.com.rukspot.samples.analytics.publisher=INFO
[2019-04-06 19:04:20,658] INFO - FilePublisher {"applicationConsumerKey":"waxwrjrA2vWIqnxDOJ0PST7IXLwa","applicationName":"DefaultApplication","applicationId":"1","applicationOwner":"admin","apiContext":"/pizzashack/1.0.0","apiName":"PizzaShackAPI","apiVersion":"1.0.0","apiResourcePath":"/menu","apiResourceTemplate":"/menu","apiMethod":"GET","apiCreator":"admin","apiCreatorTenantDomain":"carbon.super","apiTier":"Unlimited","apiHostname":"172.17.0.1","username":"admin@carbon.super","userTenantDomain":"carbon.super","userIp":"10.10.10.2","userAgent":"curl/7.47.0","serviceTime":3,"requestTimestamp":1554557657696,"throttledOut":false,"backendTime":20,"responseCacheHit":false,"responseSize":0,"protocol":"https-8243","responseCode":200,"destination":"https://localhost:9443/am/sample/pizzashack/v1/api/","metaClientType":"PRODUCTION","responseTime":23,"gatewayType":"SYNAPSE","correlationID":"5fe11502-8d98-439d-9056-8b5762236f4a","label":"Synapse","executionTime":{"securityLatency":0,"throttlingLatency":0,"requestMediationLatency":2,"responseMediationLatency":0,"backEndLatency":20,"otherLatency":0}} [2019-04-06 19:07:43,363] INFO - FilePublisher {"applicationConsumerKey":"waxwrjrA2vWIqnxDOJ0PST7IXLwa","applicationName":"DefaultApplication","applicationId":"1","applicationOwner":"admin","apiContext":"/pizzashack/1.0.0","apiName":"PizzaShackAPI","apiVersion":"1.0.0","apiResourcePath":"/menu","apiResourceTemplate":"/menu","apiMethod":"GET","apiCreator":"admin","apiCreatorTenantDomain":"carbon.super","apiTier":"Unlimited","apiHostname":"172.17.0.1","username":"admin@carbon.super","userTenantDomain":"carbon.super","userIp":"10.10.10.2","userAgent":"curl/7.47.0","serviceTime":4,"requestTimestamp":1554557863334,"throttledOut":false,"backendTime":17,"responseCacheHit":false,"responseSize":0,"protocol":"https-8243","responseCode":200,"destination":"https://localhost:9443/am/sample/pizzashack/v1/api/","metaClientType":"PRODUCTION","responseTime":21,"gatewayType":"SYNAPSE","correlationID":"cb93d815-d738-4879-8305-2124fb48ff39","label":"Synapse","executionTime":{"securityLatency":3,"throttlingLatency":0,"requestMediationLatency":0,"responseMediationLatency":0,"backEndLatency":17,"otherLatency":0}}
Add Comment
Comments (3)
I just set values below in the file "<AM_HOME>/repository/conf/api-manager.xml"
<Analytics>
<Enabled>false</Enabled>
<SkipWorkflowEventPublisher>true</SkipWorkflowEventPublisher>
</Analytics>
and also just added the line below at the end of the "log4j.properties" file.
But I couldn't get the logs in the log file.
What should I do? My wso2 AM version is 2.6, if I could run it successfully, I will build it for version 3 as well.
Can you set the config like this? It should work in 2.6.0.
<Analytics>
<!-- Enable Analytics for API Manager -->
<Enabled>true</Enabled>
<!--Skip workflow data publisher initiation and even publishing-->
<SkipWorkflowEventPublisher>true</SkipWorkflowEventPublisher>
<!-- API Usage Data Publisher. -->
<PublisherClass>com.rukspot.samples.analytics.publisher. FilePublisher</PublisherClass>
......
</Analytics>
In the Configuration section in the post, Shouldn't the api-manager.xml is looked like below?
<Analytics>
<!-- Enable Analytics for API Manager -->
<Enabled>true</Enabled>
<!--Skip workflow data publisher initiation and even publishing-->
<SkipWorkflowEventPublisher>true</SkipWorkflowEventPublisher>
<!-- API Usage Data Publisher. -->
<PublisherClass>com.rukspot.samples.analytics.publisher. FilePublisher</PublisherClass>
......
</Analytics>
yes. you need to set <PublisherClass>com.rukspot.samples.analytics.publisher. FilePublisher</PublisherClass>
Is it work in AM 3?
yes it work with AM 3.0.0. But make sure to use carbon.apimgt.version as 6.5.349
In the AM 3.0, every call raise the attached error.
I changed the dependency as you said:
<dependency>
<groupId>org.wso2.carbon.apimgt</groupId>
<artifactId>org.wso2.carbon.apimgt.usage.publisher</artifactId>
<version>6.5.349</version>
</dependency>
This is not an analytics related issue. This is issues happen when publishing throttle related information to Traffic manager. Seems your client IP is an IPv6 address and hence the issue. Is 'X-Forwarded-For' header is present in your request and is it ipv6 address? Can you send an ipv4 address with 'X-Forwarded-For' header in your request?
I just deployed the sample after setting the configurations in the deployment.toml, but the configuration in api-manager.xml won't change at all.
here is my deployment.toml config:
#[apim.analytics]
enable = true
event_publisher_impl = "com.rukspot.samples.analytics.publisher.FilePublisher"
and in the log4j2.properties:
log4j.category.com.rukspot.samples.analytics.publisher=INFO
I am testing the sample API using the swagger in the "Try Out" section and got that error you saw, the service works correctly but I don't receive any log in my console. If I use the curl and invoke the service via curl, it does not generate any error in the logs, but still I don't get any analytics log either.
github project is updated for the APIM 300. please find and follow the instruction given in [1]
[1] https://github.com/ruks/logs-analytics-events/tree/300