Thursday, August 17, 2017

DevOps Strategy for Oracle Fusion Middleware

Let me share the main points of a successful DevOps strategy for that large monolith as the Oracle Fusion Middleware platform.

I see the following principles for the DevOps strategy:

1. The reference configuration must be specified and stabilized. The DevOps world offers a number of Infrastructure as Code instruments such as Ansible, Chef, Puppet, Terraform, etc. Some utilities call themselves as "Orchestrators" but in any case, any environment should be created over the reference configuration only. Also, the reference configuration must be covered by tests. The tests must ensure the configuration stay in the actual state and has no leaks.

2. There is a negative phenomenon in the world of large middleware - Configuration Drift. The drift may take different forms from not matched set of patches installed on the test and the production environments, till a little bit different configuration of JDBC connection pools or JTA transaction timeouts. It's almost impossible to eliminate the configuration drift at all, but using the right DevOps strategy we can minimize it and take the control.

3. The basic step to controlling configuration drift is to prohibit manual changes to the configuration. Sometimes there may appear urgent changes without which the platform just doesn't work at all, but this changes must be immediately reflected in the reference configuration.

4. In order to bring additional reliability to the infrastructure, a playback mechanism should be introduced. If a change moves the production environment to a broken state, the operational team would have a strong mechanism to return back as quickly as possible. Another way is to check out a stable version of the infrastructure from a source code management tool since the infrastructure is a code itself but for Oracle Fusion Middleware based platforms such as SOA Suite or WebCenter Portal, this process takes excessive periods of time.

5. The next step could be the automatization of diagnostic information gathering and utilities for automation such as Chef, Puppet, Ansible and others can really help us.

Your comments are welcome!

Would you like to give a 'Like'? Please follow me on Twitter!

Tuesday, July 25, 2017

Building a Real Java EE + Oracle Coherence Application with Maven

There is an amazing Building a Real Application with Maven story published within a new book, Fusion Middleware Developing Applications Using Continuous Integration. The story tells of building a web-application which uses Oracle Coherence as a data store and a servlet as a data accessor. Apache Maven is used for compilation and building, the appropriate 'pom.xml' files are presented.

If we look deeply into the web-module's 'pom.xml' file, the following dependency (the gar-module) may be found:


A very interesting fact is there isn't any type declaration, so the gar-module is used as a usual 'jar' dependency. If a real application contains more gar-modules and dependencies, the best-practice for Maven users is to define the dependencies in the 'dependencyManagement' section inside the parent 'pom.xml' file. And the <type>gar</type> stanza must exist in this section since the dependency has to be used as the gar-type inside the ear-module's 'pom.xml' file.


Ok, we can say, let's define the dependencies inside the parent 'pom.xml' file and add the references on these dependencies into the 'pom.xml' files of the war- and ear-modules. Note! The 'groupId' and 'artifactId' attributes are modified.

Wednesday, July 12, 2017

How We Added a Spoon of Agility to a Large Oracle Service Bus Project

Several months ago I already shared how to ensure guaranteed delivery for 5.000.000 messages handled per day by Oracle Service Bus (OSB). In the case, OSB is heavy used to integrate a customer relationship management application (the CRM) in a solid environment with many regional billing systems, provisioning, ERP and ordering management system (OMS), etc. Also, the integration solution is deployed across four ex-standalone corporations, each of which has its own IT infrastructure, and this circumstance makes the project more and more interesting.

The common problem from a manager's point of view is to provide consistency among services deployed on the bus, the main client application (the CRM), and numerous 3rd party applications. Also, there is a time lag in development and end up to production these parts of the infrastructure. And the development team forever must have guaranteed stable and ready to deploy code of the service bus.

The situation is usual for almost all integration projects in the World. The releases of an enterprise service bus must be delivered in a reliable way, no one integration flow (neither the client -> service bus nor the service bus -> 3rd party application communication) must fail after the release is deployed to the production environment.

Let me explain the features of our project.
  1. A one pizza size development team, not more than 5 developers are there.

  2. Hovewer, the team has developed and delivered 13 services and 46 adapters to a wide range of enterprise applications. The services provide very different business logic - from just wrapping a stored procedure into a SOAP-based web service till a complex orchestration employing the Splig-Join component of Oracle Service Bus.

  3. SVN is utilized as a source code management tool.

  4. Oracle Service Bus out of the box provides no capability for service versioning, so it's impossible to expose numerous versions of the same service simultaneously. Although the main service client, CRM application, is developed by a team working for the same company, we are able to deliver releases of the service bus regardless of this capability.

  5. The development process is based on Agile. Sometimes it becomes a reason of inconsistent builds. In fact, this article is about how to eliminate the impact of any agile methodology on the consistency of the enterprise service bus.

Monday, July 3, 2017

The First Look at Red Hat JBoss Fuse

Hello everyone!

Today I wish to share an absolutely new thing in my blog, my impressions from using Red Hat JBoss Fuse.

What exactly is Red Hat JBoss Fuse? It is a runtime environment for an implementation of Enterprise Application Integration Patterns (EIP) - Apache Camel.


There are two available forms of the runtime environment:

  • Apache Karaf - a production ready implementation of the OSGi standard.

  • Red Hat JBoss Enterprise Application Platform - a well known open source Java EE application server with commercial support. Unfortunately, Red Hat JBoss Fuse can be installed only on top of the 6.4.0 version of the application server; this version implements Java EE 6 only and this fact makes some problems discussed below.

Thursday, June 29, 2017

How to Deploy Oracle SOA Suite JMS Resources on a Separate Cluster

Sometimes it's strong needed to deploy JMS resources within an SOA domain, for example, to speed up the server startup, on a different non-SOA cluster. Today I wish to share a short article to explain how to do it.

The starting point is the following set of WebLogic instances: AdminServer, two SOA servers combined in the soa_cluster, and two JMS servers combined in the jms_cluster as well.


In order to demonstrate the approach, a persistent store and JMS Server must be created on every member of the jms_cluster. Our JMS resources will be hosted here.


A JMS Module, which will contain JMS resource definitions, must be created and targeted to the entire jms_cluster. A subdeployment for resource targeting management has to be defined for the module.


Let's define a uniform distributed queue as well as a connection factory inside the module.


The truth is Oracle SOA Suite isn't able to look up any JNDI name defined on another cluster, so the defined before JMS resources aren't available for soa_server1 and soa_server2. The solution is to define a foreign server on the soa_cluster. This server will open access to the JNDI tree of the jms_cluster.


The foreign server must be defined within a JMS Module targeted to the soa_cluster. A connection string to the jms_cluster (t3://server1-host:server1-port,server2-host:server2-port,etc) has to be assigned as the value of the JNDI Connection URL parameter.


The foreign server must contain a pointer on the defined before JMS queue as well as connection factory.



Composite application developers should pick up in JDeveloper a queue, created on the JMS servers JMSServer_1 and JMSServer_2, as they did before. Since the adapter definition file ...JMSAdapter_jms.jca contains only the JNDI name of the queue, and the Local JNDI Name of the defined Foreign Destination is exactly the same as the Remote JNDI Name, developed composite applications still work fine.


In order to get access to the JNDI tree of a different cluster, any composite application must use the defined before connection factory exposed via the foreign server. So, the default JNDI name for the JMS Connection must be replaced.


The JMS JCA adapter, deployed on the SOA cluster, must contain an outbound connection pool with the same name as specified for the JMS Connection in the composite application.


And the JNDI name of the defined before connection factory must be assigned as the value of its ConnectionFactoryLocation attribute.


The testing composite application should be deployed on the SOA Suite cluster. If anyone touches the application, a test instance will be created and successfully executed. A new JMS message will appear in a queue instance of the uniform distributed queue. If send kill -9 to a member of the jms_cluster, messages arrive to another half of the queue.


Well, as we have seen, the SOA Suite domain can be broken into two different parts: processing and message store. If your SOA Suite cluster takes too much time to start, this approach may be considered and implemented.

Would you like to give a 'Like'? Please follow me on Twitter!

Monday, June 19, 2017

Oracle BPM Suite 12.2.1.2 Quick Start: Installation, Configuration, BPMN 2.0 Business Process Deployment and Testing

Oracle BPM Suite - a solution from Oracle for Enterprise Business Processes modeling with leveraging the BPMN 2.0 notation. Integrated development environment - JDeveloper is used for the modeling process. Oracle provides a special distributive of the solution for developers - Quick Start - which contains the Oracle WebLogic application server, BPM Suite, and JDeveloper.


This article highlights how to download Oracle BPM Suite 12.2.1.2 Quick Start from the edelivery.oracle.com web-site, install that on a developer's PC, create the first business process using JDeveloper, configure the integrated WebLogic domain (BPM Suite, SOA Suite, and Oracle Service Bus are included, and Apache Derby is here for database management), deploy the business process on this domain and start a test instance of the process.


Friday, June 9, 2017

DVM Validation on a SOA Suite Domain Causes High CPU Utilization: Analysis and Solution


Problem statement


Once upon a time, my colleagues asked me an interesting question. After each update of the MDS on a large production environment, a significant performance degradation is observed. CPU load on the SOA Suite servers is increased, respectively.

The system automated leveraging Oracle SOA Suite makes extensive use of Domain Value Maps (DVM), which is intended to map from one vocabulary used in a given domain to another vocabulary used in a different domain. For example, one domain may represent Russian rubles as 'RUR', while another domain may represent those as '810' (see ISO 4217). Business users can edit DVMs at runtime using Oracle SOA Composer.

And while DVMs are not so large everything works fine, but if there are some large DVM files (> 200-300 KB), the above strange behaviour may be observed.

Let's have a look at the thread dump collected during the observation.

"[ACTIVE] ExecuteThread: '57' for queue: 'weblogic.kernel.Default (self-tuning)'" #159 daemon prio=9 os_prio=2 tid=0x0000000066bbd800 nid=0x28ac runnable [0x0000000079188000]
java.lang.Thread.State: RUNNABLE
at oracle.xml.xpath.XPathChildAxis.getNodeList(XPathAxis.java:600)
at oracle.xml.xpath.XPathStep.evaluate(XPathStep.java:1102)
at oracle.xml.xpath.PathExpr.evaluate(PathExpr.java:808)
at oracle.xml.xpath.ComparisonExpr.evaluate(XSLExpr.java:1743)
at oracle.xml.xpath.XSLExprBase.testBooleanExpr(XSLExprBase.java:514)
at oracle.xml.xpath.AndExpr.evaluate(XSLExpr.java:524)
at oracle.xml.xpath.XSLExprBase.testBooleanExpr(XSLExprBase.java:514)
at oracle.xml.xpath.AndExpr.evaluate(XSLExpr.java:524)
at oracle.xml.xpath.XSLExprBase.testBooleanExpr(XSLExprBase.java:514)
at oracle.xml.xpath.AndExpr.evaluate(XSLExpr.java:524)
at oracle.xml.xpath.XPathPredicate.filter(XPathPredicate.java:349)
at oracle.xml.xpath.XPathChildAxis.getNodeList(XPathAxis.java:627)
at oracle.xml.xpath.XPathStep.evaluate(XPathStep.java:1102)
at oracle.xml.xpath.PathExpr.evaluate(PathExpr.java:808)
at oracle.xml.parser.v2.XMLNode.selectNodes(XMLNode.java:2762)
at oracle.xml.parser.v2.XMLNode.selectNodes(XMLNode.java:2722)
at oracle.tip.dvm.sdk.util.XMLUtil.isDVMDocumentValid(XMLUtil.java:211)
at oracle.tip.dvm.entity.DVMRTObject.validateDVM(DVMRTObject.java:202)
at oracle.tip.dvm.entity.DVMRTObject.(DVMRTObject.java:130)
at oracle.tip.dvm.DVMManagerImpl.getDVMRTObject(DVMManagerImpl.java:217)
at oracle.tip.dvm.DVMManagerImpl.lookupValue(DVMManagerImpl.java:133)
at oracle.tip.dvm.LookupValue.lookupValue(LookupValue.java:95)
at oracle.tip.dvm.LookupValue.lookupValue(LookupValue.java:252)
at sun.reflect.GeneratedMethodAccessor1815.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at oracle.xml.xpath.XSLExtFunctions.callStaticMethod(XSLExtFunctions.java:115)
at oracle.xml.xpath.XPathExtFunction.evaluateMethod(XPathExtFunction.java:422)
at oracle.xml.xpath.XPathExtFunction.evaluate(XPathExtFunction.java:347)
at oracle.xml.xslt.XSLValueOf.processAction(XSLValueOf.java:152)
at oracle.xml.xslt.XSLNode.processChildren(XSLNode.java:559)
at oracle.xml.xslt.XSLTemplate.processAction(XSLTemplate.java:278)
at oracle.xml.xslt.XSLStylesheet.execute(XSLStylesheet.java:706)
at oracle.xml.xslt.XSLStylesheet.execute(XSLStylesheet.java:665)
at oracle.xml.xslt.XSLProcessor.processXSL(XSLProcessor.java:401)
at oracle.xml.jaxp.JXTransformer.transform(JXTransformer.java:578)
at ...