Erik Hagen's review on Practical Machine Learning

Erik Hagen on Practical Machine Learning

My new book on the Best seller list and on sale just for today!

Deal of the Day| Practical Machine Learning| 50% discount on ebook

My "Getting Started with Greenplum" book on Free ebooks link ...

http://free-ebook-download-for4.blogspot.com/2014/09/getting-started-with-greenplum-for-big.html

Deep learning : Areas of interest

This concept map below is an excerpt from one of my latest/upcoming publications on Machine learning.

Informatica Power Center: Transformations and Optimization Best Practices

#	Category	Things to keep in mind
1	General Recommendations	·         Reduce the number of transformations ·         Calculate once, use many times – For e.g. Use Variable Ports ·         Only connect what’s being used ·         Facilitate reuse – Use reusable transformations, mapplets ·         Filter records early-on and set driving tables appropriately in joins ·         Utilize single pass reads – Read once and load multiple targets ·         Minimize aggregation calls – Use sorted input wherever possible ·         Reduce custom coding, stored procedures, scripting ·         Add proper comments in the Mapping description ·         Use environment variables for the services name in Unix scripts             For e.g-> pmcmd startworkflow –sv $INT_SVC –dn $DOMAIN_NAME ….
2	Lookup Transformation	·         Reduce the dataset being cached - Override if required ·         Remove un-used ports from the Lookup Transformation ·         Place all the conditions with equality operator first ·         If the number of calls can be reduced consider replacing with unconnected uncached lookup ·         Consider using persistent cache for frequently accessed lookups ·         When handling multiple matches, use “Return any matching value” setting whenever possible – Improves lookup performance
3	Aggregator Transformation	·         Group by simple columns ·         Use sorted Input ·         Filter data before aggregation ·         Limit port connection ·         For high volume, increase the index and data cache size – avoid paging to disk
4	Operations and Transformation optimization	·         Numeric operations are faster than string operations ·         Operators are faster than functions (e.g. || vs CONCAT) ·         Use Router transformation instead of multiple filter transformation ·         Use Sorter transformation before an Aggregator transformation ·         Use a Normalizer to pivot rows rather than multiple instance of the target ·         While using Joiner transformation ensure that smallest source is set as the master

Enterprise Integration : Web Services, SOA, ESB and JBI

Web Service Specifications

These define standards-based implementation technologies for many of the Integration patterns, but use the same architectural principles and guidelines.

Service Oriented Architectures 

Describe ways to build loosely-coupled systems composed from individual services. Because much of this loose coupling is a result of message-based communication between services, our patterns are extremely relevant in the SOA space.

ESB (Enterprise Service Bus)

The core of a typical service bus incorporates messaging, routing, transformation, endpoints.

Finally, the JBI (Java Business Integration)

JBI is the new integration API introduced in the J2EE world. It is a great enabler for SOA because it defines ESB architecture, which can facilitate the collaboration between services. It provides for loosely coupled integration by separating out the providers and consumers to mediate through the bus.

JSR 208 is an extension of J2EE, but it is specific for JBI Service Provider Interfaces (SPI). SOA and SOI are the targets of JBI and hence it is built around WSDL.

Case: Web Services without being SOA: Very much Possible. SOA is an Architecture and an application as such can have Web Services without really being SOA.

Case: Web Services & SOA without being an ESB: This can be a case where in you may commit to the concept of SOA but not necessarily implement ESB.

Case: Web Services, SOA & ESB

The JBI (Java Business Integration) case, following meta-model describes the constitution of the Architecture in context

Case: ESB and SOA without Web Services: Other Protocols such as JMS or Other Native Queuing forming the core messaging implementation you use.

Case: ESB without SOA: ESB implemented as Application Integration Technique without the consideration of SOA

OutOfMemory with Maven Compiler ?

When using Maven with Jdk 6 (1.6.0_14) within eclipse, OutOfMemory error is encountered
Setting MAVEN_OPTS as per the suggestions was of no help.
The following configuration setting inside the ma ven compile plugin worked like a wonder!

<configuration>
<fork>true</fork>
<meminitial>256m</meminitial>
<maxmem>512m</maxmem>
</configuration>

Upon investigation, there is a note on the attribute <fork>
<fork> attribute, when set to true, allows running the compiler in a separate process and use defined heap value as against using default built-in compiler. (By default, fork is false)

Hence, the solution is having the compiler use a separate process as against the default. Another important observation is that i haven't encountered this exception when i used jdk1.6.0_23 + version.