Sunday, June 26, 2016

[IBM MQ] Create a topic using IBM MQ

Hi all,

I am configuring integration scenario of IBM MQ with WSO2 ESB. In particular scenario, used IBM MQ 8.0.0.2 and ESB 5.0.0 in the windows environment (Oracle JDK 1.8).

If you need to understand the installation mechanism from the first step, you can follow this blog post. 

Assumption: You have configured ESB with IBM MQ for Queues.

Let's begin creating a topic in IBM MQ.

1. Open IBM MQ UI (Websphere Installation1 window) through administrative mode.

How to do that ?
  • Go to window start button and search for Websphere
  • Right click on icon and  click on "Run as Administrator"
2. Create a Topic under Queue Manger





3. Add Destination




Note: Give the "Topic string" (Which was defined when creating the Topic) as the Topic in next window. (Refer next screen shot)


4. Create Topic Connection Factory if you don't specify previous connection factories as Topic Connection Factory/ Connection Factory (General term for both topics and queues)






You are done with Topic Configuration. When you are going to subscribe to Topic from external clients (Ex: ESB) you can see those subscriptions through "Subscription panel"



Happy Coding ... :)

Saturday, June 25, 2016

[WSO2][ESB Cluster] HornetQ host configuration when connecting to a cluster

When we are going to use HornetQ with  cluster environment (Ex: ESB Cluster), have to configure HorentQ hosts. If not HornetQ will not identify exact configuration.

In HornetQ run.sh script ($HorenetQ_Home/bin/) update as follows:

# Use the following line to run with different ports
#export CLUSTER_PROPS="-Djnp.port=1099 -Djnp.rmiPort=1098 -Djnp.host=localhost -Dhornetq.remoting.netty.host=localhost -Dhornetq.remoting.netty.port=5445"

Uncomment above line and update relevant parameters.

# Use the following line to run with different ports
export CLUSTER_PROPS="-Djnp.port=1099 -Djnp.rmiPort=1098 -Djnp.host=10.100.7.120 -Dhornetq.remoting.netty.host=localhost -Dhornetq.remoting.netty.port=5445"

** Update -Djnp.port and -Djnp.host with relevant values. -Djnp.host should be the IP/host_name of an instance which HornetQ installed.


[SFTP] Connecting to remote server for file sharing

You can use to connect remote server and easily share files between local and remote server using SFTP.

You can use SFTP with your "key" which is using to access the remote server OR just access without credentials if you don't have any

Using SFTP server:

sftp -i <key file> username@remote_hostname/ip   OR
sftp username@remote_hostname/ip

Ex:

 sftp -i dilshani.pem centos@192.***.**.71  

When you connected through SFTP server it can see the sftp server as follows:


 sftp>  

You can get files or send files secure manner.

Send files:

put <file_name> <remote_location>

 sftp> put test.text /home/centos/files  

** Note: If you are using above command, file should be located in same folder, which is going to start sftp server. Otherwise it can be give the path of file befor file name.

Get files:

get <remote_location/file_name> <local_location>

 sftp> get /home/centos/files/test.text  

** Note: you can navigate to any folder using cd command

cd <file_directory>

 sftp> cd /home/centos/files  

When you are done with file sharing, close the server with "exit" command.

 sftp> exit  


Happy Sharing :)



ActiveMQ Tips

How to change the default port of Active MQ ?

1. Go to activemq.xml ($ActiveMQ_Home/conf/)

2. Edit openwire port

<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="ws" uri="ws://0.0.0.0:61614?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600"/>
</transportConnectors>

By default, 61616 is port of Active MQ. You can give whatever the port you want.


Login to Active MQ dashboard 

In Active MQ, there is a web dashboard which will display every detail of topics/queues created within active mq broker. You can access to that using following url (using any browser)
  • http://<ip>: 8161
          Ex: http://localhost:8161/


Default username and password for Active MQ
  • When accessing active MQ you may not need any password unless you add it through the installation process.
  • When you login to web console (even though you did not add any password/ username), you will have to provide username and password. 
  • Use username: admin and password: admin

General JMS parameter values for Active MQ
  • java.naming.factory.initial : org.apache.activemq.jndi.ActiveMQInitialContextFactory
  • java.naming.provider.url : tcp://localhost:61616  (Note: if you are calling remote active mq broker, you should add relevant ip/host name instead of "localhost")

 Integration with WS02 ESB

  •  If you are going to configure Active MQ with WSO2 ESB, you can refer WSO2 documentations.
  • But you have to remember about the version of client jars you are going to put into ESB. Even though you are using active MQ which has higher version than 5.8, you have to put Active MQ 5.8 client jars
Client Libraries:
  • activemq-broker-5.8.0.jar
  • activemq-client-5.8.0.jar
  • geronimo-jms_1.1_spec-1.1.1.jar
  • geronimo-j2ee-management_1.1_spec-1.0.1.jar

Welcome all value additions regarding Active MQ :) 



[IBM MQ] How to make a remote connection through WSO2 ESB


Hi all,

This blog post about creating a remote connection to the IBM MQ which is located in a remote server. To do that, you have to have IBM MQ installed in remote server. For IBM MQ installation you can refer this blog post.

Let's remotely connect to IBM MQ ...

Environment : IBM MQ Version 8.0.0.2 installed in Windows 2012 R2 server
                        Client is WSO2 ESB 5.0.0 installed in ubuntu 15.10

Steps to connect:
  • First, you have to take .bindings file you used (Which is in jndiDirectory)
  • Add .bindings file in to a local file directory (Ex: /home/dilshani/Dev/jndiDirectory)
  • Edit .bindings file 
** Open .bindings file using text editor and replace all places having "localhost" with relevant IP of the server which is installed IBM MQ.
  • Add client jars into [ESB_HOME]/repository/components/lib folder   
* com.ibm.mq.allclient 
* fscontext
* jms
* providerutil

** You can find these jars inside the java folder in IBM MQ installation file directory (Ex: C:/)
  •  Remove following line from $ESB_HOME/repository/conf/etc/launch.ini. 
                            javax.jms,\
  • Add WSO2 ESB axis2.xml configs related to JMS transport (You may find axis2.xml file in [ESB_HOME]/repository/conf/axis2/)
Sample Configs:

 <transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">  
   <parameter name="default" locked="false">  
   <parameter name="java.naming.factory.initial" locked="false">com.sun.jndi.fscontext.RefFSContextFactory</parameter>  
   <parameter name="java.naming.provider.url" locked="false">file:///home/centos/esb/jndiDirectory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">myTopicConnectionFactory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>  
   <parameter name="transport.jms.Destination" locked="false">testTopic</parameter>  
   <parameter name="transport.jms.DestinationType" locked="false">topic</parameter>  
   <parameter name="transport.jms.UserName" locked="false">administrator</parameter>  
   <parameter name="transport.jms.Password" locked="false">wso2321#qa</parameter>  
  </parameter>  
  <parameter name="myQueueConnectionFactory1" locked="false">  
   <parameter name="java.naming.factory.initial" locked="false">com.sun.jndi.fscontext.RefFSContextFactory</parameter>  
   <parameter name="java.naming.provider.url" locked="false">file:///home/centos/esb/jndiDirectory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">myQueueConnectionFactory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>  
   <parameter name="transport.jms.UserName" locked="false">administrator</parameter>  
   <parameter name="transport.jms.Password" locked="false">wso2321#qa</parameter>  
   <parameter name="transport.jms.Destination">LocalQueue1</parameter>  
  </parameter>  
  <parameter name="myTopicConnectionFactory1" locked="false">  
    <parameter name="java.naming.factory.initial" locked="false">com.sun.jndi.fscontext.RefFSContextFactory</parameter>  
   <parameter name="java.naming.provider.url" locked="false">file:///home/centos/esb/jndiDirectory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">myTopicConnectionFactory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>  
   <parameter name="transport.jms.DestinationType" locked="false">topic</parameter>  
   <parameter name="transport.jms.UserName" locked="false">administrator</parameter>  
   <parameter name="transport.jms.Destination">testTopic</parameter>  
   <parameter name="transport.jms.Password" locked="false">wso2321#qa</parameter>  
  </parameter>  
 </transportReceiver>  

And also you have to add transport sender configurations too.

 <transportSender name="jms" class="org.apache.axis2.transport.jms.JMSSender">  
   <parameter name="default" locked="false">  
   <parameter name="java.naming.factory.initial" locked="false">com.sun.jndi.fscontext.RefFSContextFactory</parameter>  
   <parameter name="java.naming.provider.url" locked="false">file:///home/centos/esb/jndiDirectory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">myTopicConnectionFactory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>  
   <parameter name="transport.jms.Destination" locked="false">testTopic</parameter>  
   <parameter name="transport.jms.DestinationType" locked="false">topic</parameter>  
   <parameter name="transport.jms.UserName" locked="false">administrator</parameter>  
   <parameter name="transport.jms.Password" locked="false">wso2321#qa</parameter>  
  </parameter>  
  <parameter name="myQueueConnectionFactory1" locked="false">  
   <parameter name="java.naming.factory.initial" locked="false">com.sun.jndi.fscontext.RefFSContextFactory</parameter>  
   <parameter name="java.naming.provider.url" locked="false">file:///home/centos/esb/jndiDirectory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">myQueueConnectionFactory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>  
   <parameter name="transport.jms.UserName" locked="false">administrator</parameter>  
   <parameter name="transport.jms.Password" locked="false">wso2321#qa</parameter>  
   <parameter name="transport.jms.Destination">LocalQueue1</parameter>  
  </parameter>  
  <parameter name="myTopicConnectionFactory1" locked="false">  
   <parameter name="java.naming.factory.initial" locked="false">com.sun.jndi.fscontext.RefFSContextFactory</parameter>  
   <parameter name="java.naming.provider.url" locked="false">file:///home/centos/esb/jndiDirectory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">myTopicConnectionFactory</parameter>  
   <parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>  
   <parameter name="transport.jms.DestinationType" locked="false">topic</parameter>  
   <parameter name="transport.jms.UserName" locked="false">administrator</parameter>  
   <parameter name="transport.jms.Destination">testTopic</parameter>  
   <parameter name="transport.jms.Password" locked="false">wso2321#qa</parameter>  
  </parameter>  
 </transportSender>  
  • Start ESB server and it will create JMS connection.

** Note: If you get any error, please take following steps to solve the issue.
  •  Ping the server and see whether it can be reach
Ex:
       ping <serever ip>
  • Telenet the port and see whether it open to connect
Ex:
       telnet <server-ip> <port>
  • Restart the IBM MQ 
** Check whether "channel" is inactive and .bindings file connected. (You can do it via IBM MQ UI)

Thursday, June 9, 2016

[WSO2][ESB][JMS] Local Transactionality Sample

Hi all,

Let's consider about JMS Transport parameter [1] "jms.transport.transactionality" sample. In here I use local transactionality.

When it's comes to transactionality, it helps to establish reliability message transformation. We can use tranactions and roll back them when having any issue/error. Within WSO2 ESB, it supports main two type of transactios through JMS Transport. Those are local and jta.

Local transactions use basically for locally used wso2 ESB instances while jta using for distributed systems. JTA transactions may engage with distributed system having clusters of esb which adding data to database etc.

Lets take a simple example of local tranactions which will roll back transaction in a failure. Lets' begin...

Rrerequisite:

1. Message broker should be up and running (Ex: wso2 MB)

2. ESB should be configured with relevant message broker. Add "transport.Transactionality" parameter and "transport.jms.SessionTransacted" parameter.

Sample axis2.xml Configuration:

 <transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">   
  <parameter name="myTopicConnectionFactory" locked="false">   
     <parameter name="java.naming.factory.initial" locked="false">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter>   
     <parameter name="java.naming.provider.url" locked="false">repository/conf/jndi.properties</parameter>    
     <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">TopicConnectionFactory</parameter>
     <parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>  
     <parameter name="transport.jms.Transactionality" locked="true">local</parameter>   
     <parameter name="transport.jms.SessionTransacted" locked="true">true</parameter>  
     <parameter>   

    <parameter name="myQueueConnectionFactory" locked="false">   
     <parameter name="java.naming.factory.initial" locked="false">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter>   
     <parameter name="java.naming.provider.url" locked="false">repository/conf/jndi.properties</parameter>   
     <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter>   
     <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>   
     <parameter name="transport.jms.Transactionality" locked="true">local</parameter>   
     <parameter name="transport.jms.SessionTransacted" locked="true">true</parameter>   
    </parameter>       

    <parameter name="default" locked="false">   
     <parameter name="java.naming.factory.initial" locked="false">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter>   
     <parameter name="java.naming.provider.url" locked="false">repository/conf/jndi.properties</parameter>   
     <<parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">TopicConnectionFactory</parameter>   
     <parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>   
     <parameter name="transport.jms.Transactionality" locked="true">local</parameter>   
     <parameter name="transport.jms.SessionTransacted" locked="true">true</parameter>   
    </parameter>   
   </transportReceiver>   


3. ESB should be up and running.

4. Axis2 server should be started in port 9000

Sample:

Create a proxy with following configuration: (For proxy insequence and fault sequence, I created two seperate sequences )

 <proxy name="ErrorProxy" startOnLoad="true" transports="https http jms">  
     <target faultSequence="fault" inSequence="ErrorInSequence"/>  
     <parameter name="transport.jms.ContentType">  
       <rules>  
         <jmsProperty>contentType</jmsProperty>  
         <default>text/xml</default>  
       </rules>  
     </parameter>  
     <parameter name="transport.jms.ConnectionFactory">myQueueConnectionFactory</parameter>  
     <parameter name="transport.jms.DestinationType">queue</parameter>  
     <parameter name="transport.jms.SessionTransacted">true</parameter>  
     <parameter name="transport.jms.Destination">errorQueue</parameter>  
   </proxy>  

Error Sequence Configuration:

  <sequence name="fault">  
     <log level="full"/>  
     <property name="SET_ROLLBACK_ONLY" scope="axis2" value="true"/>  
   </sequence>  

Insequence Configuration:

   <sequence name="ErrorInSequence">  
     <call blocking="true">  
       <endpoint>  
         <address uri="http://localhost:9000/services/SimpleStockQuoteService"/>  
       </endpoint>  
     </call>  
     <description/>  
   </sequence>  

Running Sample:

* Publish messages to queue (errorQueue) in Message Broker. I used external jms message publisher.

External JMS Publisher Java Class:

 import javax.jms.*;  
 import javax.naming.Context;  
 import javax.naming.InitialContext;  
 import javax.naming.NamingException;  
 import java.util.Properties;  
 /**  
  * Created by dilshani on 5/11/16.  
  * This class will publish messages to MB Queue  
  */  
 public class QueuePublisher {  
   public static final String QPID_ICF = "org.wso2.andes.jndi.PropertiesFileInitialContextFactory";  
   private static final String CF_NAME_PREFIX = "connectionfactory.";  
   private static final String QUEUE_NAME_PREFIX = "queue.";  
   private static final String CF_NAME = "qpidConnectionfactory";  
   String userName = "admin";  
   String password = "admin";  
   private static String CARBON_CLIENT_ID = "carbon";  
   private static String CARBON_VIRTUAL_HOST_NAME = "carbon";  
   private static String CARBON_DEFAULT_HOSTNAME = "localhost";  
   private static String CARBON_DEFAULT_PORT = "5672";  
   String queueName = "QueueProxy";  
   int publishMsgCount=1;  
   public static void main(String[] args) throws NamingException, JMSException {  
     QueuePublisher queueSender = new QueuePublisher();  
     queueSender.sendMessages();  
   }  
   public void sendMessages() throws NamingException, JMSException {  
     Properties properties = new Properties();  
     properties.put(Context.INITIAL_CONTEXT_FACTORY, QPID_ICF);  
     properties.put(CF_NAME_PREFIX + CF_NAME, getTCPConnectionURL(userName, password));  
     properties.put(QUEUE_NAME_PREFIX + queueName, queueName);  
     Properties properties2 = new Properties();  
     properties2.put(Context.INITIAL_CONTEXT_FACTORY, QPID_ICF);  
     properties2.put(CF_NAME_PREFIX + CF_NAME, getTCPConnectionURL(userName, password));  
     System.out.println("getTCPConnectionURL(userName,password) = " + getTCPConnectionURL(userName, password));  
     InitialContext ctx = new InitialContext(properties);  
     // Lookup connection factory  
     QueueConnectionFactory connFactory = (QueueConnectionFactory) ctx.lookup(CF_NAME);  
     QueueConnection queueConnection = connFactory.createQueueConnection();  
     queueConnection.start();  
     QueueSession queueSession = queueConnection.createQueueSession(false,QueueSession.AUTO_ACKNOWLEDGE );//  
     Queue queue = (Queue)ctx.lookup(queueName);  
     javax.jms.QueueSender queueSender = queueSession.createSender(queue);  
     //sending 5 messages to the above created queue here  
     for(Integer i=1;i<=publishMsgCount;i=i+1){  
       TextMessage textMessage = queueSession.createTextMessage("::::Test Message:::: "+i+":::Publish to queue:::"+queueName+":::From IP:::"+CARBON_DEFAULT_HOSTNAME);  
       textMessage.setStringProperty("msgID", i.toString());  
       queueSender.send(textMessage);  
       System.out.println("Publishing Test Message "+i+"::Published From IP::"+CARBON_DEFAULT_HOSTNAME);  
     }  
     queueSender.close();  
     queueSession.close();  
     queueConnection.close();  
   }  
   public String getTCPConnectionURL(String username, String password) {  
     // amqp://{username}:{password}@carbon/carbon?brokerlist='tcp://{hostname}:{port}'  
     return new StringBuffer()  
         .append("amqp://").append(username).append(":").append(password)  
         .append("@").append(CARBON_CLIENT_ID)  
         .append("/").append(CARBON_VIRTUAL_HOST_NAME)  
         .append("?brokerlist='tcp://").append(CARBON_DEFAULT_HOSTNAME).append(":").append(CARBON_DEFAULT_PORT).append("'")  
         .toString();  
   }  
 }  

* You may see response from backend (Axis2 server on port 9000)

* Shut down axis 2 server and Publish messages to jms queue.

* Check ESB server terminal. It may print error sequence log several times. (Which is trying to rollback the transaction)

Note : As its' rollback transaction, message will be send to jms queue. But again it will be taken from proxy as it listen to that and will try to send. Then again move to fault sequence and it will be working like a round. (Untill time out)

Now you have a working sample of ESB local transactionality.