I was at a customer site, and it was required that I connect to their wired network. Everyone in the customer site could connect to the wired network, except me... :) Why? well probably two reasons, I had a ubuntu machine where as everyone else had windows, and I had a machine I used at WSO2, with WSO2 network.
Hence, I googled a bit, and figured out the issue. The issue in my case was that the /etc/network/interfaces file had some unwanted lines in there.
If you do happen to come across the same issue, you could try this as well.
Comment or delete any lines in the /etc/network/interfaces file except for the lines
auto lo
iface lo inet loopback
I personally would comment.. :)
References
[1] http://ubuntuforums.org/showthread.php?t=2247351
Tuesday, April 21, 2015
Waiting for network configuration at boot up in ubuntu (14.04)
Today I was meddling around with the network configurations as I could not connect to a wired network, and with something I did, when I restarted I was keeping on getting the message "Waiting for network configuration at boot up in ubuntu" and then wait upto 60 seconds, and start up without any network manager settings.
Because of this I could not connect to a network, and the network manager was literally stalled.
After looking at my previous steps, hitting the history command
#history
it shows that I was meddling with the resolve.conf file. But when I looked at the file, it showed that no DNS nameservers were defined in the file. The file was EMPTY.... PANIC!!!!
So then I tried setting the nameserver in the /etc/resolve.conf file, however it was getting replaced.
The solution was,
1) go to /etc/resolveconf/resolv.conf.d/
# cd /etc/resolveconf/resolv.conf.d/
2) Open the base file n super user mode
# sudo vim base
3) add the nameserver details
e.g : nameserver 127.0.1.1
4) Save file, and restart.
This should fix the issue.
References
[1] http://unix.stackexchange.com/questions/128220/how-do-i-set-my-dns-on-ubuntu-14-04
Because of this I could not connect to a network, and the network manager was literally stalled.
After looking at my previous steps, hitting the history command
#history
it shows that I was meddling with the resolve.conf file. But when I looked at the file, it showed that no DNS nameservers were defined in the file. The file was EMPTY.... PANIC!!!!
So then I tried setting the nameserver in the /etc/resolve.conf file, however it was getting replaced.
The solution was,
1) go to /etc/resolveconf/resolv.conf.d/
# cd /etc/resolveconf/resolv.conf.d/
2) Open the base file n super user mode
# sudo vim base
3) add the nameserver details
e.g : nameserver 127.0.1.1
4) Save file, and restart.
This should fix the issue.
References
[1] http://unix.stackexchange.com/questions/128220/how-do-i-set-my-dns-on-ubuntu-14-04
Monday, April 20, 2015
[WSO2 APIM] - The Basic Architecture of WSO2 APIM -part 5- Statistics in APIM
In this blog post I will briefly go about how the APIM publisher and store are able to draw up graphs with statistical information on the API's. This blog post is targetted for new bee's who would like to get a bird's eye view on the functionality.
In the WSO2 APIM Gatway, as I explained in my previous post The Basic Architecture of WSO2 APIM - part 3- Gateway & Key Manager - Under the hoods, it has a set of handlers defined per API. One of these handlers is the
In the WSO2 APIM Gatway, as I explained in my previous post The Basic Architecture of WSO2 APIM - part 3- Gateway & Key Manager - Under the hoods, it has a set of handlers defined per API. One of these handlers is the
APIMgtUsageHandler , which would invoke the org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataPublisher . The APIMgtUsageDataPublisher is configured to publish the events to the BAM server that has been configured with the APIM. Illustrated by a diagram is the process of publishing of stats and viewing them on the publisher /store app's .jpg)
The WSO2 APIM Gateway would publish the events via the thrift protocol to the WSO2 BAM server. The BAM is more likely a BAM cluster in the real world production environment. The BAM server then writes the data to a No Sql database, cassandra. Then the BAM Analyzer to which the APIM tool box is deployed to, will fetch the data batch by batch from the cassandra database. The BAM Analyzer is capable of summarizing the data, by using hive. The hive scripts have to pre written and deployed in the server. The BAM analyzer then will push the summarized data to an RDBMS.The WSO2 APIM Store and the WSO2 Publisher then will pull the data from the RDBMS and will display the data in the APIM Store and publisher analytics pages.This is a very brief explanation of what happens in the APIM when statistics are to be displayed. Detailed information can be found at the references listed below.
References
[1] https://docs.wso2.com/display/AM180/Publishing+API+Runtime+Statistics#PublishingAPIRuntimeStatistics-ConfigureWSO2BAM[2] http://wso2.com/library/articles/2013/08/how-to-use-wso2-api-manager-and-bam-together-to-analyse-api-usage-stats/ [WSO2 APIM] - The Basic Architecture of WSO2 APIM - part 4- Gateway & Key Manager - Under the hoods
In this post I will briefly introduce how the Gatway and the Key manager interacts in order for an API to be successfully invoked.
Let me bring up a diagram first.
.jpg)
In a real world distributed deployment, only the WSO2 APIM Gateway would be exposed to the outside world. So with the creation of keys for a certain application that would be subscribed to many API's, (If you are not familiar with this context, please refer to the blog post I had written before The Basic Architecture of WSO2 APIM - Part 2- The Store - under the hood for a crash course) you would be able to invoke an API via the gateway.
Ideally an Application would have the consumer_key and the consumer_secret hardcoded in the application it self. When invoking the API, the username, password would have to be added and the application would pass in the username, password, consumer_key and consumer_secret to the gateway. The Gateway has some token API's [1]. which are
- /token
- /revoke
When calls are made to these API's, the gateway calls the Key manager Jax-RX in order to verify the access the token. The key-manager would call the AM_DB, retrieve the access token, and verify the access token. it will return the API Info DTO which includes the meta data of the access token, which includes the validity period, refresh token and scopes.
When the API's are invoked, the Gateway does not call the Key manager at every invocation. The APIM Gatway makes use of a cache for this. However, this cache can be turned off. [2]
Invoking the API
When the API's are invoked there are several authorization grant types that we could use. The WSO2 APIM supports the same grant types the WSO2 IS supports. The password grant type is used only when the application is a trusted application Client Credentials grant type requires only the consumer_key & consumer_secret. [1].
API Handlers
When the API's is created it will store the synapse configuration of the API in the WSO2 APIM Gateway, being another ESB (WSO2 ESB). When the api is invoked and it hits the Gateway, it will execute the API's in and out sequences which would have a set of handlers. The API when created would have a set of default handlers defined in the API [3].
1) Security handler/
3) Stats hander/
4) Google analytics handler/
5) Extentions handler/
Each of these handler classes has 2 methods
1) handle request
2) handle response
These methods have been overridden in each of these handlers to accomplish the task that the handler is written for. There is also a possibility that you could write your own handler, and plug it in. Details on this could be found at [4].
I hope you were able to gain a basic knowledge on what happens internally when an API is invoked on the WSO2 APIM, on a highlevel. By going through the references you would be able to gain a much detailed knowledge on the APIM gateway and WSO2 APIM as a whole.
References
[1] https://docs.wso2.com/display/AM180/Token+API
[2] https://docs.wso2.com/display/AM180/Configuring+Caching
[3] https://docs.wso2.com/display/AM180/Writing+Custom+Handlers
[4] https://docs.wso2.com/display/AM180/Writing+Custom+Handlers#WritingCustomHandlers-Writingacustomhandler
Let me bring up a diagram first.
In a real world distributed deployment, only the WSO2 APIM Gateway would be exposed to the outside world. So with the creation of keys for a certain application that would be subscribed to many API's, (If you are not familiar with this context, please refer to the blog post I had written before The Basic Architecture of WSO2 APIM - Part 2- The Store - under the hood for a crash course) you would be able to invoke an API via the gateway.
Ideally an Application would have the consumer_key and the consumer_secret hardcoded in the application it self. When invoking the API, the username, password would have to be added and the application would pass in the username, password, consumer_key and consumer_secret to the gateway. The Gateway has some token API's [1]. which are
- /token
- /revoke
When calls are made to these API's, the gateway calls the Key manager Jax-RX in order to verify the access the token. The key-manager would call the AM_DB, retrieve the access token, and verify the access token. it will return the API Info DTO which includes the meta data of the access token, which includes the validity period, refresh token and scopes.
When the API's are invoked, the Gateway does not call the Key manager at every invocation. The APIM Gatway makes use of a cache for this. However, this cache can be turned off. [2]
Invoking the API
When the API's are invoked there are several authorization grant types that we could use. The WSO2 APIM supports the same grant types the WSO2 IS supports. The password grant type is used only when the application is a trusted application Client Credentials grant type requires only the consumer_key & consumer_secret. [1].
API Handlers
When the API's is created it will store the synapse configuration of the API in the WSO2 APIM Gateway, being another ESB (WSO2 ESB). When the api is invoked and it hits the Gateway, it will execute the API's in and out sequences which would have a set of handlers. The API when created would have a set of default handlers defined in the API [3].
1) Security handler/
APIAuthenticationHandler The security handler is to validate the Oauth token used to invoke the API. 2) Throttle handler/ APIThrottleHandler Throttles the request based on the throttle policy. This is done based on two counts, the global count and the local count.
3) Stats hander/
APIMgtUsageHandler Helps to push data to the BAM for analytics. 4) Google analytics handler/
APIMgtGoogleAnalyticsTrackingHandler Pushes events to Google analytics for analytics.
5) Extentions handler/
APIManagerExtensionHandler Executes extention's sequences
Each of these handler classes has 2 methods
1) handle request
2) handle response
These methods have been overridden in each of these handlers to accomplish the task that the handler is written for. There is also a possibility that you could write your own handler, and plug it in. Details on this could be found at [4].
I hope you were able to gain a basic knowledge on what happens internally when an API is invoked on the WSO2 APIM, on a highlevel. By going through the references you would be able to gain a much detailed knowledge on the APIM gateway and WSO2 APIM as a whole.
References
[1] https://docs.wso2.com/display/AM180/Token+API
[2] https://docs.wso2.com/display/AM180/Configuring+Caching
[3] https://docs.wso2.com/display/AM180/Writing+Custom+Handlers
[4] https://docs.wso2.com/display/AM180/Writing+Custom+Handlers#WritingCustomHandlers-Writingacustomhandler
Friday, April 17, 2015
[WSO2 APIM] - The Basic Architecture of WSO2 APIM - Part 3- The Store - under the hood
In continuation to my previous post two posts on the The Basic Architecture of WSO2 APIM - Part 1 & The Basic Architecture of WSO2 APIM - Part 2- The Publisher - under the hood , this post is going to briefly discuss the architecture of the API Store component. These posts are mainly targeted for dummies on WSO2 APIM. The aforementioned posts are a recommended read in order to understand this post clearly.
First off all, what is the API Store?
The API store is the play ground for the API consumer. An API consumer can self-register, discover on API functionalities, subscribe to the API's, evaluate and interact with API's.
The APIM store is also a jaggery web app.
.jpg)
Databases associated with the API Store
The database configuration can be found at <APIM_HOME>/repository /conf/datasources/master-datasources.xml
- Registry Database
The db handles all registry related storage.
- AM_DB
This db stores information required for the APIM.
* The AM_ tables store information related to API's.
* The IDN_ tables store information related to OAuth Identity.
View API
The APIM store is multi tenanted, and hence, at login, if multiple tenants are registered, it will prompt to select the tenant.
Upon selection of tenant, the API's which are published by that tenant can be viewed on the API store. Here the APIM-Store will extract API's to display from the registry_db through a cache.
When logged in, a user can view the API in a much detailed version and also edit it if permission is granted.
The store has a feature where it shows the recently added api's for convenience.
An Application & Subscribe to an API
An Application in the APIM world is a concept of detaching API's from consumers.
An Application is a single entity to which api's can be subscribed to. This application is created on the AM_DB database and when the api is subscribed the subscription is also recorded on the APIM.
This application is then the unit of reference in the APIM store.
According to WSO2 docs an application is defined as ,
An application is primarily used to decouple the consumer from the APIs. It allows you to :
Creating keys in order to invoke an API can be done in the APIM store. Once the application is created, we can create tokens for that application.
When it comes to tokens, we can create application tokens (access tokens) and application user tokens. The types are APPLICATION & APPLICATION_USER. Access tokens are tokens which the application developer gets.
In the APIM Store when we create these tokens, we get the consumer_key and consumer_secret which is per application. The APIM store will talk to the APIM Key manager (in future releases there will be the capability of plugging in a custom Key manager, but for the time being it is either only the WSO2 APIM key manager or WSO2 IS as a keymanager) and the key manager will generate the keys. These keys will be stored in the AM_DB as well.
The tokens generated are associated to the application with a validity period.
Note : Consumer key is analogous to a user name and the consumer secret it analogous to a user password.
Regenerating Tokens
The WSO2 APIM allows the regeneration of access tokens. In this process there are 3 steps that will be executed.
1) The existing token will be revoked.
2) A new token will be generated
3) The validity period of the token will be updated in the database.
When these three steps are performed the new generated token will be returned.
Indexing
The store indexes the API's in order for better performance. the index info can be found at /solr location in the registry. Information on this can be found at [1].
Workflows
In the APIM store internals, many workflows are used. One such example of this is the subscription creation.
In the subscription creation a workflow executor is used. This workflow executors have two methods.
1. execute.
2. complete
In order for it to be more clear on the workflow implementations, let me bring up a diagram explaining it.
The implementation of the workflow could take two paths.
1) The default implementation
this is where the method "complete" is directly called within the method "execute" method.
2) A custom workflow definition.
for example's sake we are using WSO2 BPS as the workflow execution unit here. We need to write a custom workflow executor and use it in the APIM. Via a web service the external workflow will be executed. (for the soap service, the call back URL would be the BPS url)
When a workflow is executed, the workflow detail is being put into the
More detailed information on this can be found at [2] & [3].
References
[1] https://docs.wso2.com/display/AM180/Add+Apache+Solr-Based+Indexing
[2] https://docs.wso2.com/display/AM180/Customizing+a+Workflow+Extension
[3] https://docs.wso2.com/display/AM180/Adding+an+API+Subscription+Workflow
[4] https://docs.wso2.com/display/AM180/Quick+Start+Guide#QuickStartGuide-Introductiontobasicconcepts
First off all, what is the API Store?
The API store is the play ground for the API consumer. An API consumer can self-register, discover on API functionalities, subscribe to the API's, evaluate and interact with API's.
The APIM store is also a jaggery web app.
Databases associated with the API Store
The database configuration can be found at <APIM_HOME>/repository /conf/datasources/master-datasources.xml
- Registry Database
The db handles all registry related storage.
- AM_DB
This db stores information required for the APIM.
* The AM_ tables store information related to API's.
* The IDN_ tables store information related to OAuth Identity.
View API
The APIM store is multi tenanted, and hence, at login, if multiple tenants are registered, it will prompt to select the tenant.
Upon selection of tenant, the API's which are published by that tenant can be viewed on the API store. Here the APIM-Store will extract API's to display from the registry_db through a cache.
When logged in, a user can view the API in a much detailed version and also edit it if permission is granted.
The store has a feature where it shows the recently added api's for convenience.
An Application & Subscribe to an API
An Application in the APIM world is a concept of detaching API's from consumers.
An Application is a single entity to which api's can be subscribed to. This application is created on the AM_DB database and when the api is subscribed the subscription is also recorded on the APIM.
This application is then the unit of reference in the APIM store.
According to WSO2 docs an application is defined as ,
An application is primarily used to decouple the consumer from the APIs. It allows you to :
- Generate and use a single key for multiple APIs
- Subscribe multiple times to a single API with different SLA le
Creating keys in order to invoke an API can be done in the APIM store. Once the application is created, we can create tokens for that application.
When it comes to tokens, we can create application tokens (access tokens) and application user tokens. The types are APPLICATION & APPLICATION_USER. Access tokens are tokens which the application developer gets.
In the APIM Store when we create these tokens, we get the consumer_key and consumer_secret which is per application. The APIM store will talk to the APIM Key manager (in future releases there will be the capability of plugging in a custom Key manager, but for the time being it is either only the WSO2 APIM key manager or WSO2 IS as a keymanager) and the key manager will generate the keys. These keys will be stored in the AM_DB as well.
The tokens generated are associated to the application with a validity period.
Note : Consumer key is analogous to a user name and the consumer secret it analogous to a user password.
Regenerating Tokens
The WSO2 APIM allows the regeneration of access tokens. In this process there are 3 steps that will be executed.
1) The existing token will be revoked.
2) A new token will be generated
3) The validity period of the token will be updated in the database.
When these three steps are performed the new generated token will be returned.
Indexing
The store indexes the API's in order for better performance. the index info can be found at /solr location in the registry. Information on this can be found at [1].
Workflows
In the APIM store internals, many workflows are used. One such example of this is the subscription creation.
In the subscription creation a workflow executor is used. This workflow executors have two methods.
1. execute.
2. complete
In order for it to be more clear on the workflow implementations, let me bring up a diagram explaining it.
The implementation of the workflow could take two paths.
1) The default implementation
this is where the method "complete" is directly called within the method "execute" method.
2) A custom workflow definition.
for example's sake we are using WSO2 BPS as the workflow execution unit here. We need to write a custom workflow executor and use it in the APIM. Via a web service the external workflow will be executed. (for the soap service, the call back URL would be the BPS url)
When a workflow is executed, the workflow detail is being put into the
AM_WORKFLOWS in the AM_DB, and the status of the workflow is being moved to ON_HOLD. Once the complete is called, it updates the status to either APPROVED or REJECTED.More detailed information on this can be found at [2] & [3].
References
[1] https://docs.wso2.com/display/AM180/Add+Apache+Solr-Based+Indexing
[2] https://docs.wso2.com/display/AM180/Customizing+a+Workflow+Extension
[3] https://docs.wso2.com/display/AM180/Adding+an+API+Subscription+Workflow
[4] https://docs.wso2.com/display/AM180/Quick+Start+Guide#QuickStartGuide-Introductiontobasicconcepts
Unable to get the hostName or IP address of the server when starting WSO2 Products
Today I faced an issue when a wso2 server that needed restarting did not start up, after shutting down due to the error
{org.apache.synapse.ServerConfigurationInformation} - Unable to get the hostName or IP address of the server {org.apache.synapse.ServerConfigurationInformation}
java.net.UnknownHostException: <hostname> : <hostname> : Name or service not known
at java.net.InetAddress.getLocalHost(InetAddress.java:1473)
at org.apache.synapse.ServerConfigurationInformation.initServerHostAndIP(ServerConfigurationInformation.java:215)
at org.apache.synapse.ServerConfigurationInformation.<init>(ServerConfigurationInformation.java:66)
at org.apache.synapse.ServerConfigurationInformationFactory.createServerConfigurationInformation(ServerConfigurationInformationFactory.java:48)
at org.wso2.carbon.mediation.initializer.ServiceBusInitializer.initESB(ServiceBusInitializer.java:365)
at org.wso2.carbon.mediation.initializer.ServiceBusInitializer.activate(ServiceBusInitializer.java:182)
The issue in my case was that it was not a wso2 bug. It is an issue with the machine.
The OS I was using was ubuntu. The issue was that the hostname was not correctly set.
The easy way - temporary fix
1) Run the following command. (assuming the hostname you want to set is abc001)
# hostname abc001
2) To verify the hostname just run the following command
# hostname
it should output the hostname that we set as the output.
Proper fix
1) We need to fix this properly. In that case, we need to change the /sysconfig/network file.
open this file as sudo or super user, and change the hostname in that file to be the hostname you require.
HOSTNAME=abc001
2) Then you need to update the etc/hosts file, to reflect this change. you should map the entry
127.0.0.1 abc001
or
<local-ip> abc001
3) After that you need to restart the network service.
#service network restart
or
#/etc/init.d/network restart
4) Then verify the hostname has been set by running the command
# hostname
References
1) http://www.howtogeek.com/50631/how-to-change-your-linux-hostname-without-rebooting/
{org.apache.synapse.ServerConfigurationInformation} - Unable to get the hostName or IP address of the server {org.apache.synapse.ServerConfigurationInformation}
java.net.UnknownHostException: <hostname> : <hostname> : Name or service not known
at java.net.InetAddress.getLocalHost(InetAddress.java:1473)
at org.apache.synapse.ServerConfigurationInformation.initServerHostAndIP(ServerConfigurationInformation.java:215)
at org.apache.synapse.ServerConfigurationInformation.<init>(ServerConfigurationInformation.java:66)
at org.apache.synapse.ServerConfigurationInformationFactory.createServerConfigurationInformation(ServerConfigurationInformationFactory.java:48)
at org.wso2.carbon.mediation.initializer.ServiceBusInitializer.initESB(ServiceBusInitializer.java:365)
at org.wso2.carbon.mediation.initializer.ServiceBusInitializer.activate(ServiceBusInitializer.java:182)
The issue in my case was that it was not a wso2 bug. It is an issue with the machine.
The OS I was using was ubuntu. The issue was that the hostname was not correctly set.
The easy way - temporary fix
1) Run the following command. (assuming the hostname you want to set is abc001)
# hostname abc001
2) To verify the hostname just run the following command
# hostname
it should output the hostname that we set as the output.
Proper fix
1) We need to fix this properly. In that case, we need to change the /sysconfig/network file.
open this file as sudo or super user, and change the hostname in that file to be the hostname you require.
HOSTNAME=abc001
2) Then you need to update the etc/hosts file, to reflect this change. you should map the entry
127.0.0.1 abc001
or
<local-ip> abc001
3) After that you need to restart the network service.
#service network restart
or
#/etc/init.d/network restart
4) Then verify the hostname has been set by running the command
# hostname
References
1) http://www.howtogeek.com/50631/how-to-change-your-linux-hostname-without-rebooting/
Subscribe to:
Posts (Atom)