Tuesday, April 21, 2015

Wired network not detected in ubuntu 14.04

I was at a customer site, and it was required that I connect to their wired network. Everyone in the customer site could connect to the wired network, except me... :) Why? well probably two reasons, I had a ubuntu machine where as everyone else had windows, and I had a machine I used at WSO2,  with WSO2 network.

Hence, I googled a bit, and figured out the issue. The issue in my case was that the /etc/network/interfaces file had some unwanted lines in there.

If you do happen to come across the same issue, you could try this as well.

Comment or delete any lines in the /etc/network/interfaces file except for the lines 

auto lo
iface lo inet loopback

I personally would comment.. :)


[1] http://ubuntuforums.org/showthread.php?t=2247351

Waiting for network configuration at boot up in ubuntu (14.04)

Today I was meddling around with the network configurations as I could not connect to a wired network, and with something I did, when I restarted I was keeping on getting the message "Waiting for network configuration at boot up in ubuntu" and then wait upto 60 seconds, and start up without any network manager settings.

Because of this I could not connect to a network, and the network manager was literally stalled.

After looking at my previous steps, hitting the history command


it shows that I was meddling with the resolve.conf file. But when I looked at the file, it showed that no DNS nameservers were defined in the file. The file was EMPTY.... PANIC!!!!

So then I tried setting the nameserver in the /etc/resolve.conf file, however it was getting replaced.

The solution was,

1) go to /etc/resolveconf/resolv.conf.d/
      # cd  /etc/resolveconf/resolv.conf.d/
2) Open the base file n super user mode
     # sudo vim base
3) add the nameserver details
    e.g : nameserver
4) Save file, and restart.

This should fix the issue.


[1] http://unix.stackexchange.com/questions/128220/how-do-i-set-my-dns-on-ubuntu-14-04

Monday, April 20, 2015

[WSO2 APIM] - The Basic Architecture of WSO2 APIM -part 5- Statistics in APIM

In this blog post I will briefly go about how the APIM publisher and store are able to draw up graphs with statistical information on the API's. This blog post is targetted for new bee's who would like to get a bird's eye view on the functionality.

In the WSO2 APIM Gatway, as I explained in my previous post The Basic Architecture of WSO2 APIM - part 3- Gateway & Key Manager - Under the hoods, it has a set of handlers defined per API.  One of these handlers is the APIMgtUsageHandler , which would invoke the org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataPublisher .  The APIMgtUsageDataPublisher  is configured to publish the events to the BAM server that has been configured with the APIM. 

Illustrated by a diagram is the process of publishing of stats and viewing them on the publisher /store app's 

The WSO2 APIM Gateway would  publish the events  via the thrift protocol to the WSO2 BAM server. The BAM is more likely a BAM cluster in the real world production environment. The BAM server then writes the data to a No Sql database, cassandra.  Then the BAM Analyzer to which the APIM tool box is deployed to, will fetch the data batch by batch from the  cassandra database.  The BAM Analyzer  is capable of summarizing the data, by using hive. The hive scripts have to pre written and deployed in the server. The BAM analyzer then will push the summarized data to an RDBMS.

The WSO2 APIM Store and the WSO2 Publisher then will pull the data from the RDBMS  and will display the data in the APIM Store and publisher analytics pages.

This is a very brief explanation of what happens in the APIM when statistics are to be displayed.

Detailed information can be found  at the references listed below.


[1] https://docs.wso2.com/display/AM180/Publishing+API+Runtime+Statistics#PublishingAPIRuntimeStatistics-ConfigureWSO2BAM
[2] http://wso2.com/library/articles/2013/08/how-to-use-wso2-api-manager-and-bam-together-to-analyse-api-usage-stats/


[WSO2 APIM] - The Basic Architecture of WSO2 APIM - part 4- Gateway & Key Manager - Under the hoods

In this post I will briefly introduce how the Gatway and the Key manager interacts in order for an API to be successfully invoked.

Let me bring up a diagram first.

In a real world distributed deployment, only the WSO2 APIM Gateway would be  exposed to the outside world. So with the creation of keys for a certain application that would be subscribed to many API's, (If you are not familiar with this context, please refer to the blog post I had written before The Basic Architecture of WSO2 APIM - Part 2- The Store - under the hood  for a crash course) you would be able to invoke an API via the gateway.

Ideally an Application would have the consumer_key and the consumer_secret hardcoded in the application it self. When invoking the API, the username, password would have to be added and the application would pass in the username, password, consumer_key and consumer_secret  to the gateway. The Gateway has some token API's [1]. which are
   - /token
   - /revoke

When calls are made to these API's,  the gateway calls the Key manager Jax-RX  in order to verify the access the token.  The key-manager would call the AM_DB, retrieve the access token, and verify the access token.  it will return the API Info DTO which includes the meta data of the access token, which includes the validity period, refresh token and scopes.

When the API's are invoked, the Gateway does not call the Key manager at every invocation.  The APIM Gatway makes use of a cache  for this. However, this cache can be turned off. [2]

Invoking the API

When the API's are invoked there are several authorization grant types that we could use. The WSO2 APIM supports the same grant types the WSO2 IS supports.  The password grant type is used only when the application is a trusted application  Client Credentials grant type requires only the consumer_key & consumer_secret. [1].

API Handlers

When the API's is created it will store the synapse configuration of the API in the WSO2 APIM Gateway, being another ESB (WSO2 ESB).  When the api is invoked and it hits the Gateway, it will execute the API's in and out sequences  which would have a set of handlers. The API when created would have a set of default handlers defined in the API [3].

   1) Security handler/ APIAuthenticationHandler
     The security handler is to validate the Oauth token used to invoke the API.
  2) Throttle handler/ APIThrottleHandler
     Throttles the request based on the throttle policy. This is done based on two counts, 
     the global count and the local count.

   3) Stats hander/ APIMgtUsageHandler
     Helps to push data to the BAM for analytics.
   4) Google analytics handler/ APIMgtGoogleAnalyticsTrackingHandler
      Pushes events to Google analytics for analytics.

   5) Extentions handler/ APIManagerExtensionHandler
     Executes extention's sequences

 Each of these handler classes has 2 methods
  1) handle request
  2) handle response

 These methods have been overridden in each of these handlers to accomplish the task that the handler is written for. There is also a possibility that you could write your own handler, and plug it in. Details on this could be found at [4].

I hope you were able to gain a basic knowledge on what happens internally when an API is invoked on the WSO2 APIM, on a highlevel.  By going through the references you would be able to gain a much detailed knowledge on the APIM gateway and WSO2 APIM as a whole.


[1] https://docs.wso2.com/display/AM180/Token+API
[2] https://docs.wso2.com/display/AM180/Configuring+Caching
[3] https://docs.wso2.com/display/AM180/Writing+Custom+Handlers
[4] https://docs.wso2.com/display/AM180/Writing+Custom+Handlers#WritingCustomHandlers-Writingacustomhandler

Friday, April 17, 2015

[WSO2 APIM] - The Basic Architecture of WSO2 APIM - Part 3- The Store - under the hood

 In continuation to my previous post two posts on the The Basic Architecture of WSO2 APIM - Part 1  &   The Basic Architecture of WSO2 APIM - Part 2- The Publisher - under the hood  , this post is going to briefly discuss the architecture of the API Store component.  These posts are mainly targeted for dummies on WSO2 APIM. The aforementioned posts are a recommended read in order to understand this post clearly.

First off all, what is the API Store?

The API store is the play ground for the API consumer. An API consumer can self-register,  discover on API functionalities, subscribe to the API's, evaluate and interact with API's.

The APIM store is also a jaggery web app.

Databases associated with the API Store

 The database configuration can be found at <APIM_HOME>/repository /conf/datasources/master-datasources.xml

    - Registry Database
      The db handles all registry related storage.
    - AM_DB
       This db stores information required for the APIM.
           * The AM_ tables store information related to API's.
           * The IDN_ tables store information related to OAuth Identity.

View API

The APIM store is multi tenanted, and hence, at login, if multiple tenants are registered, it will prompt to select the tenant.

Upon selection of tenant, the API's which are published by that tenant can be viewed on the API store. Here the APIM-Store will extract API's to display from the registry_db through a cache.

When logged in, a user can view the API in a much detailed version and also edit it if permission is granted.

The store has a feature where it shows the recently added api's for convenience.

 An Application & Subscribe to an API

An Application in the APIM world is a concept of detaching API's from consumers.

An Application is a single entity to which api's can be subscribed to. This application is created on the AM_DB database and when the api is subscribed the subscription is also recorded on the APIM.

This application is then the unit of reference in the APIM store.

According to WSO2 docs an application is defined as ,

An application is primarily used to decouple the consumer from the APIs. It allows you to :
  • Generate and use a single key for multiple APIs
  • Subscribe multiple times to a single API with different SLA le
 Creating Keys

Creating keys in order to invoke an API can be done in the APIM store. Once the application is created, we can create tokens for that application.

When it comes to tokens, we can create application tokens (access tokens) and application user tokens. The types are APPLICATION & APPLICATION_USER. Access tokens are tokens which the application developer gets.

In the APIM Store when we create these tokens, we get the consumer_key and consumer_secret which is per application. The APIM store will talk to the APIM Key manager (in future releases there will be the capability of plugging in a custom Key manager, but for the time being it is either only the WSO2 APIM key manager or WSO2 IS as a keymanager) and the key manager will generate the keys. These keys will be stored in the AM_DB as well.

The tokens generated are associated to the application with a validity period.

 Note : Consumer key is analogous to a user name and the consumer secret it analogous to a user password.

Regenerating Tokens

The WSO2 APIM allows the regeneration of access tokens. In this process  there are 3 steps that will be executed.

 1) The existing token will be revoked.
 2) A new token will be generated
 3) The validity period of the token will be updated in the database.

When these three steps are performed the new generated token will be returned.


The store indexes the API's in order for better performance. the index info can be found at /solr location in the registry. Information on this can be found at [1].


In the APIM store internals, many workflows are used. One such example of this is the  subscription creation.

In the subscription creation a workflow executor is used. This workflow executors have two methods.

     1. execute.

     2. complete

 In order for it to be more clear on the workflow implementations, let me bring up a diagram explaining it.

The implementation of the workflow could take two paths.

1) The default implementation
     this is where the method "complete" is directly called within the method "execute" method.
2) A custom workflow definition.
   for example's sake we are using WSO2 BPS as the workflow execution unit here.  We need to write a custom workflow executor and use it in the APIM. Via a web service the external workflow will be executed.  (for the soap service, the call back URL would be the BPS url)

When a workflow is executed, the workflow detail is being put into the AM_WORKFLOWS in the AM_DB, and the status of the workflow is being moved to ON_HOLD. Once the complete is called, it updates the status to either APPROVED or REJECTED.

More detailed information on this can be found at [2] & [3].


[1] https://docs.wso2.com/display/AM180/Add+Apache+Solr-Based+Indexing
[2] https://docs.wso2.com/display/AM180/Customizing+a+Workflow+Extension
[3] https://docs.wso2.com/display/AM180/Adding+an+API+Subscription+Workflow
[4] https://docs.wso2.com/display/AM180/Quick+Start+Guide#QuickStartGuide-Introductiontobasicconcepts

Unable to get the hostName or IP address of the server when starting WSO2 Products

Today I faced an issue when a wso2 server that needed restarting did not start up, after shutting down due to the error

{org.apache.synapse.ServerConfigurationInformation} -  Unable to get the hostName or IP address of the server {org.apache.synapse.ServerConfigurationInformation}
java.net.UnknownHostException: <hostname> : <hostname> : Name or service not known
at java.net.InetAddress.getLocalHost(InetAddress.java:1473)
at org.apache.synapse.ServerConfigurationInformation.initServerHostAndIP(ServerConfigurationInformation.java:215)
at org.apache.synapse.ServerConfigurationInformation.<init>(ServerConfigurationInformation.java:66)
at org.apache.synapse.ServerConfigurationInformationFactory.createServerConfigurationInformation(ServerConfigurationInformationFactory.java:48)
at org.wso2.carbon.mediation.initializer.ServiceBusInitializer.initESB(ServiceBusInitializer.java:365)
at org.wso2.carbon.mediation.initializer.ServiceBusInitializer.activate(ServiceBusInitializer.java:182)

The issue in my case was that it was not a wso2 bug. It is an issue with the machine.

The OS I was using was ubuntu. The issue was that the hostname was not correctly set.

The easy way  - temporary fix

1) Run the following command. (assuming the hostname you want to set is abc001)

                    #    hostname abc001

2) To verify the hostname just run the following command

                           # hostname

it should output the hostname that we set as the output.

Proper fix

1) We need to fix this properly. In that case, we need to change the /sysconfig/network file.

open this file as sudo or super user, and change the hostname in that file to be the hostname you require.


2) Then you need to update the etc/hosts file, to reflect this change. you should map the entry


                      <local-ip> abc001

3) After that you need to restart the network service.

                   #service network restart
                   #/etc/init.d/network restart

4) Then verify the hostname has been set by running the command

                    # hostname


1) http://www.howtogeek.com/50631/how-to-change-your-linux-hostname-without-rebooting/