2013-12-26

2013-12-26 Thursday - SOA ~ API Service Bus? Enterprise Governance Value?

Let me preface this posting with one important caveat: This was written from the perspective of having spent several years working within a very large multi-national software development  effort, within a highly regulated industry, with over 150+ developers, across multiple large-scale enterprise application domains, integrating with many different externally contracted service providers - as well as dozens of internal applications (bespoke systems, as well as heavily  customized COTS packaged software)

The buzz-trend headlines of today's pundits assume we should move toward a Point-to-Point API strategy for integrations both within and without the enterprise.

An oft repeated line of argument that arises when pundits relish their prognostications of the death-knell of Service Oriented Architecture (SOA), revolves around the notion that the world-wide-web doesn't have a centralized service bus - and therefore, by inference, any corporation that has the temerity to think that they might need to provision an Enterprise Service Bus (ESB) - is implementing something that is foolishly wasteful,  extremely heavy-weight, and probably overkill.

This argument is probably quite right in many cases.

However, within any enterprise-level organizations of appreciable size - with many different applications disbursed across the organization (both politically and geographically) - there are common and recurring application infrastructure capabilities that will end up being re-implemented again-and-again within the endpoints of any Point-to-Point integration strategy, if there isn't some enterprise-class service bus capability established.

For example, does it make sense for every individual application to implement their own approach to:
- retry logic for guaranteed delivery
- exception handling
- monitoring/reporting/alerting
- caching
- fail-over routing
- implementing fan-in and fan-out patterns, with the inherent brittle dependency of canonical
mapping to each up-or-down stream application
- managed file transfers
- job scheduling
- data encryption (at-rest, and in-motion)
- complex event processing
- message queue management


The outcome of an absence of any enterprise service bus capability, cannot help but result in some trade-off decisions that have significant negative consequences for long-term maintenance, organizational agility, and software development costs.

Integration strategies within many large-scale enterprise organizations must often adhere to a much higher threshold of rigor (such as audit traceability and monitoring) - and Point-to-Point integration strategies often result in duplication of code in places that could be more centralized maintained, managed, and provisioned via a service bus (note: I'm not saying One ESB to Rule Them All - an ESB might be logical, virtual, distributed, and quite often federated).

I'm not saying Point-to-Point is a necessarily a bad integration strategy in all cases, or that an ESB is a necessary component of an enterprise-class integration strategy. But to ignore the concerns raised above, or to burden application architectures with what should be common and reusable enterprise-level shared infrastructure-like capabilities - that results in the re-implementation and redundant maintenance of such common functionality - is something that should be questioned, and strongly challenged.

The challenge/questioning of whether there is a need for Enterprise Governance is another aspect of this discussion that is often dismissed as a needless overhead to the software development process. In my experience, this is most often argued most passionately by folks who have taken the idea of Agile to an unhealthy extreme - and who seem to use it as a justification for their genetic lack of enthusiasm for documenting their work. However, within any highly regulated industry, Enterprise Governance certainly has a proper and necessary place. The trick is to do it with the lightest touch possible. Without Enterprise Governance, without a guiding vision that has continuity (i.e. from a permanent Enterprise Architecture team) beyond the typically high turn-over of all-too-often mostly outsourced development teams  - development can become a Wild West, decision making is almost invariably short-sighted, tactical, and representative of silo thinking - without any broader / longer term strategic thinking evident -and becomes a function of the strongest personalities on a development team (at best) - and  (at worst) without any cross-application team communications of decisions, tools selection is made within a vacuum...devoid of any guidance from policies, standards, or specifications.

Some possible signs that your Enterprise Governance process may be out of control:
  1. You have more than [z] developers on staff, and you have no Enterprise Governance process, where [z] is greater than the size of a small garage start-up.
  2. You have no Policies, Standards, or Specifications established to guide your development teams.
  3. Your governance process stipulates the creation of [n]X additional artifacts than your design/development process produces.
  4. Your governance process stipulates [y]X more review checkpoints than performed in your design/development process.
  5. One or more of your developers has stated the following: "The code is the {design|documentation}"
  6. One or more of your developers has stated the following; "We don't need to write unit tests"
  7. Your code base has a high cyclomatic complexity rating
  8. Your design/development process does not include frequent peer reviews.
  9. You have no visibility into code metrics
  10. You don't have an automated Continuous Integration build process established
  11. You are not running static code anlaysis tools during your Continuous Integration build process.
  12. You have no reserve capacity in your development group to review and address the technical debt that is accumulating (as found by various static code analysis tools, e.g. Sonar task executed by Jenkins/Hudson)

2013-11-26

2013-11-26 Tuesday - Deployment Optimization



A nice little nugget of a problem was handed to me today: identify ways to help an operations team reduce their system maintenance / deployment window [for production system updates] that has somehow grown to require a xx-hour window, and achieve zero downtime (or as close as possible).

The environemnt is complicated in the extreme: highly regulated industry, compliance requirements, clustered servers, high availability, PCI security zones, 3rd party software/service providers, cloud service providers/integrations (SaaS and PaaS), frequent commercial software upgrades/patches, vendor constraints on database schema changes, disaster recovery dependencies, a legion of upstream and downstream data integration dependencies.

For the last year I've been carefully planting seeds of certain ideas in various conversations with key stakeholders within an organization - to begin the gradual introduction of concepts and practices such as DevOps, Continuous Deployment, and Continuous Operations. Now that a sufficient level of pain has been experienced, there is a broad consensus and acceptance that there needs to be change.

"He was not in a hurry, 'hurry' being one human concept he had failed to grok at all. He was sensitively aware of the key importance of correct timing in all acts — but with the Martian approach: correct timing was accomplished by waiting."
Stranger in a Strange Land, by Robert E. Heinlein

I have some ideas, but as a good researcher, first order of business is to review current directions, trends, peer articles.  This posting will be a place for me to share some of the information that may be of interest to others:

Zero Downtime, Instant Deployment and Rollback
http://www.ebaytechblog.com/2013/11/21/zero-downtime-instant-deployment-and-rollback/

Jevgeni Kabanov (ZeroTurnaround)
Pragmatic Continuous Delivery, at W-JAX 2012
http://vimeo.com/79959315

Continuous Operations for Zero Downtime Deployments
http://www.virtualizationpractice.com/continuous-operations-for-zero-downtime-deployments-22680/

The Virtualization Practice
http://www.virtualizationpractice.com/

Deploying the Netflix API
http://techblog.netflix.com/2013/08/deploying-netflix-api.html


Cloud Architecture Tutorial
Constructing Cloud Architecture the Netflix Way
Gluecon May 23rd, 2012, by Adrian Cockroft
http://www.slideshare.net/adrianco/netflix-architecture-tutorial-at-gluecon

Cassandra in the Netflix Architecture, Denis Sheahan
CassandraEU London March 28th, 2012
http://www.slideshare.net/acunu/cassandra-eu-2012-netflixs-cassandra-architecture-and-open-source-efforts

Patterns for Continuous Delivery, Reactive, High Availability, DevOps and Cloud Native Open Source with Netflix OSS
Adrian Cockroft + Ben Christensen, YOW! Workshop Dec'2013
https://speakerdeck.com/adrianco/patterns-for-continuous-delivery-reactive-high-availability-devops-and-cloud-native-open-source-with-netflixoss

Best Practices for Zero Risk, Zero Downtime Database Maintenance
http://www.oracle.com/us/products/database/311390-133499.pdf

VMware vSphere High Availability 5.0 Deployment Best Practices
http://www.vmware.com/files/pdf/techpaper/vmw-vsphere-high-availability.pdf

Free Ebook: Continuous Delivery — What It Is and How to Get Started
http://info.puppetlabs.com/download-free-continuous-delivery-ebook.html

The Phoenix Project, A Novel About IT, DevOps & Helping Your Business Win
http://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/0988262592/

How Draw Something Scaled to 50 million New Users, in 50 Days, with Zero Downtime
http://www.infoq.com/presentations/games-scalability-omgpop

I Ain't Afraid of No Downtime: Scaling Continuous Deployment, by Cody Powell
http://www.codypowell.com/taods/2012/04/i-aint-afraid-of-no-downtime-scaling-continuous-deployment.html

Mandi Walls free ebook, Building a DevOps Culture [Kindle]
http://www.amazon.com/Building-DevOps-Culture-Mandi-Walls-ebook/dp/B00CBM1WFC

Daily Dose of DevOps: 27 People to Follow on Twitter
http://puppetlabs.com/blog/daily-dose-devops-27-people-follow


Selected QCON 2013 San Francisco presentations:

Adopting Continuous Delivery, Adjusting your Architecture
Rachel Laycock, ThoughtWorks
http://qconsf.com/system/files/presentation-slides/Adopting%20Continuous.pdf
 Build Your Own PaaS the Netflix Way
Sudhir Tonse, Manager, Cloud Platform Infrastructure, Netflix
http://qconsf.com/system/files/presentation-slides/BuildYourOwnPaaSTheNetflixWay-QConSF.pdf
Facebook Infrastructure
Pedro Canahuati, Director, Infrastructure Operations
http://qconsf.com/system/files/presentation-slides/ScalingtheOperationsOrganizationatFacebook.pdf

Tools:
Liquidbase:
  • Improved checksum performance
  • CORE-1509: Significantly decreased memory usage, especially with large sql files
  • CORE-1533: Performance improvements in dropAll

ZeroTurnAround's LiveRebel:

 log4j2:
  • "Log4j 2 can automatically reload its configuration upon modification"
  • "Log4j 2 contains next-generation Asynchronous Loggers based on the LMAX Disruptor library. In multi-threaded scenarios Asynchronous Loggers have 10 times higher throughput and orders of magnitude lower latency than Log4j 1.x"

  • Note the performance benchmark results recently posted on takipiblog.com


Puppet Labs:

2013-11-24

2013-11-24 Sunday - Book Review: Python In Practice



I recently had an opportunity to read Python in Practice, by Mark Summerfield as a reviewer for Pearson Education.

Two noteworthy points:
- The code samples are based on Python 3.x
- The author's web site provides sample code download

The very first thing I noticed about this book is that the sample bits of code are very concise - without extraneous fluff.

This book is an excellent addition to any software craftsman's library that needs a quick dive into practical hands-on Python coding.  Each page has a feeling of being written from the perspective of someone who has lived in the trenches producing production-quality python code for over a decade. This book has the feeling of a friendly mentor's guiding hand.

To help you understand the impression that this book left in my mind, let me share another word that arose unbidden in my mind as I made my way through the pages of this book: In Japanese, the phrase "Sensei" is often translated as "teacher", but also has a literal meaning of "One Who Has Gone Before". Mark's transmission of his depth of experience to the printed word is given in that same spirit of a Sensei teaching lessons gleaned from years of practice.

Another unexpected pleasure discovered within the pages of Mark's book: An emphasis on patterns. The first three chapters provide an excellent overview of the python-esque approach to implementing the various patterns that were described in the book Design Patterns: Elements of Reusable Object-Oriented Software.

I also found Chapter 5. Extending Python to be very interesting for focusing the discussion on Python's ctypes and Cython - techniques which are often needed to solve high performance problems.

One minor criticism is the author's choice in focusing on XML-RPC style in Chapter 6. High Level Networking [versus RESTful web service based integration strategies]


Chapter 1. Creational Design Patterns
1.1 Abstract Factory Pattern
1.2 Builder Pattern
1.3 Factory Method Pattern
1.4 Prototype Method Pattern
1.5 Singleton Pattern
Chapter 2. Structural Design Patterns
2.1 Adapter Pattern
2.2 Bridge Pattern
2.3 Composite Pattern
2.4 Decorator Pattern
2.5 Facade Pattern
2.6 Flyweight Pattern
2.7 Prox Pattern
Chapter 3. Behavioral Design Patterns
3.1 Chain of Responsibility Pattern
3.2 Command Pattern
3.3. Interpreter Pattern
3.4 Iterator Pattern
3.5 Mediator Pattern
3.6 Memento Pattern
3.7 Observer Pattern
3.8 State Pattern
3.9 Strategy Pattern
3.10 Template Method Pattern
3.11 Visitor Pattern
3.12 Case Study: An Image Package
Chapter 4. High-Level Concurrency
4.1 CPU-Bound Concurrency
4.2 I/O Bound Concurrency
4.3 Case Study: A Concurrent GUI Application
Chapter 5. Extending Python
5.1 Accessing C Libraries with ctypes
5.2 Using Cython
5.3 Case Study: An Accelerated Image Package
Chapter 6. High-Level Networking
6.1 Writing XML-RPC Applications
6.2 Writing RPyC Applications
Chapter 7. Graphical User Interfaces with Tkinter
7.1 Introduction to Tkinter
7.2 Creating Diaglogs with Tkinter
7.3 Creating Main-Window Applications with TKinter
Chapter 8. OpenGL 3D Graphics
8.1 A Perspective Scene
8.2 An Orthographic Game





2013-11-21

2013-11-21 Thursday - XML/XSD validation in Python and Installing PIP (Python Package Manager)

I'm working on developing a small utility to help automate the documentation and generation of some Enterprise Integration Architecture Context Diagrams from various spreadsheets that list the inventory of different systems and their integration (transforming the data into an XML canonical form, and then using that canonical form as the basis for generating graphviz .dot files).  This way, I will have a reusable tool that can be used across multiple projects - and simply either ask for the data in the canonical form - or limit my future effort to just getting it into the canonical form.

So I found myself needing to incorporate a few 3rd party Python packages into my solution, so I decided to spend some time this evening to finally install a decent package manager in my development environment.

First, install setuptools
https://pypi.python.org/pypi/setuptools#using-setuptools-and-easyinstall


https://pypi.python.org/pypi/pip
http://www.pip-installer.org/en/latest/installing.html

http://stackoverflow.com/questions/4750806/how-to-install-pip-on-windows

http://dubroy.com/blog/so-you-want-to-install-a-python-package/

Unofficial Windows Binaries for Python Extension Packages
http://www.lfd.uci.edu/~gohlke/pythonlibs/

http://dubroy.com/blog/so-you-want-to-install-a-python-package/



My original need was to install a package to help develop some code that requires XSD/XML manipulation.

Initially, I thought would be a good fit for my need - but the pip install process seems have some issues (at least in my environment)
http://lxml.de/installation.html
http://lxml.de/validation.html#xmlschema
http://stackoverflow.com/questions/17819884/xml-xsd-feed-validation-against-a-schema
http://code.activestate.com/recipes/578503-validate-xml-with-schemalocation/

A second possible package for consideration (although it doesn't appear to have any development since 2006...):
https://pypi.python.org/pypi/pyxsd

The xlrd package provides support for reading Excel spreadsheets:
https://pypi.python.org/pypi/xlrd
http://www.python-excel.org/ 

2013-11-07

2013-11-07 Thursday - Suggested SOA reading list

A colleague recently asked what resources I might recommend for them to begin learning about SOA (he's recently been dropped into a large enterprise organization that has a complex existing SOA environment)

Here's the reading list I prepared...

First, check InfoQ for articles and conference presentations:
http://www.infoq.com/soa/

Next, check YouTube for recent videos:
http://www.youtube.com/results?filters=year&search_query=service+oriented+architecture+soa&lclk=year


There are 10 IBM SOA Tutorials linked to here:

TechTarget.com has some

This author’s tutorials may be of interest
http://tutorials.jenkov.com/soa/index.html


The best resource for quickly getting up to speed (with a holistic and comprehensive approach) would be to read Jim Bean’s book:
(there is a kindle version available too)

Next, I would direct to you read anything on the web that is SOA-focused by Arnon Rotem-Gal-Oz, as well as his book:
(he is one of the most clear-headed people when it comes to real-world, practical SOA.

Thomas Erl is one of my favorite writers on SOA – because he is meticulous in building a body of knowledge (some might criticize his writing for be repetitive, but I see it as reinforcing the concepts)

I would start with his SOA Design patterns book:

And perhaps then his SOA with REST: Principles, Patterns & Constraints for Building Enterprise Solutions with REST

If you think you still need more foundational information, then go back to his  Service Oriented Architect: Concepts, Technology, and Design

Or his SOA: Principles of Service Design

His most recent book, which I reviewed for Prentice Hall, is also worth reading: Cloud Computing: Concepts, Technology & Architecture

For a more technical deep dive discussion of SOA service contract versioning:


I also reviewed Robert Daigneau’s book: Service Design Patterns: Fundamental Design Solutions for SOAP/WSDL and RESTful Web Services, which has a very light touch on RESTful service – but has some concise writing on service design patterns that may be of interest
http://www.amazon.com/Service-Design-Patterns-Fundamental-Solutions/dp/032154420X/

2013-09-25

2013-09-25 Wednesday - Tooling to Support Continuous Deployment

This posting is a placeholder for resources links and notes about tooling that I find interesting to support DevOps in the challenges of Configuration Management and [Continuous?] Deployment.

For some somewhat agnostic overviews of tooling and processes that can support continuous deployment  in practice...these presentations may be of interest:


While many companies have stringent regulatory constraints that require more thoughtful and methodical approaches to deployment, I thought it worth noting how Etsy.com implemented a RADICAL (!) DevOps culture - as an inspiring example to challenge status quo thinking of "but-that's-not-the-way-we-do-things-here..."

A Wikipedia article - providing a sparse summary/survey of some open source solutions for automated deployment tooling...

There is a great deal of fast-paced innovation in the tooling space for deployment & configuration management – here are some of the tools (slightly listed in order of what I find to be interesting and innovative)…


Microsoft System Center 2012

Less sophisticated, but perhaps a good example of how some shops can still make improvements over manual processes...even if it is just by doing things with bare-bones scripting…for example, via Rex

2013-09-17

2013-09-17 Tuesday - ZeroMQ


Recently, while researching some distributed computing integration patterns, I happened to come across an interesting paper: Middleware Trends and Market Leaders 2011 - produced by researchers at the CERN facility in Switzerland - that cited ZeroMQ.org as a leader.

Ilya Grigorik posted a nice write-up on ZeroMQ back in September 2010
http://www.igvita.com/2010/09/03/zeromq-modern-fast-networking-stack/

2013-09-15

2013-09-15 Sunday - Auto-Generating Documentation from DataPower Config Info?


I recently posed a question on the  IBM WebSphere DataPower SOA Appliance forum:

Auto-Generation of Documentation?

It seems that zip2html is the only option available - and it seems to be a fairly minimal / brute-force approach at that.

2013-09-15 Sunday - IBM Integration Bus

Today I'm doing some research on the relatively new IBM Integration Bus (v9.0)

IBM Integration Bus
IBM® Integration Bus formerly known as WebSphere® Message Broker is an enterprise service bus (ESB) providing connectivity and universal data transformation for service-oriented architecture (SOA) and non-SOA environments.

IBM Integration Bus Information Center v9.0

IBM Integration Bus v9.0 documentation

IBM Integration Bus introduction

What's new in IBM Integration Bus (v9.0) for WebSphere Enterprise Service Bus users?

Download IBM Integration Bus for Developers

Converting WebSphere ESB resources for use in IBM Integration Bus: Part 1

Converting WebSphere ESB resources for use in IBM Integration Bus: Part 2

Extended Structured Query Language(ESQL) is a programming language defined by IBM® Integration Bus to define and manipulate data within a message flow.

2013-09-03

2013-09-03 Tuesday - Technical Debt Conjecture


Recent observations and reflections prompted me to ponder this thought:
A Technical Debt Conjecture
For a given non-trivial project,
Given [z] budgeted sprints,
for [m] developers,
Where for sufficiently large [m],
[n] items of technical debt (or defects) are introduced - as the total for all [commits | builds],
which will subsequently require [x] additional sprints to fix.

One wonders if there are lower limits for [z], [m], [n] that would allow for approximations of [x]?

One wonders what recurring ratios of m:n might occur with some predictable frequency across different organizations, and if those ratios might vary to a greater (or lesser) degree if further considered within the specific vertical context of a given industry?

One wonders if these values are the same for on-shore vs. off-shore vs. hybrid project teams?

One wonders if there is a predictable relationship between m:n and x?

One wonders how the duration [t] of a given project might affect the trend line and predictability of these factors - and if there might be clustering of values around certain frequencies of [t]?

One wonders, given the planned / budgeted value for [z] - at what sufficiently large size of [x] do most organizations kill a project?

One wonders, what ratio of z:x does a project qualify for the 'death march' designation?

One wonders what minimum ratio of m:n should be sufficient, as a monitored metric, to serve as an indicator to management that one or more of the following might apply?

  • Requirements may not be [sufficiently | clearly] defined or finalized
  • One or more developers may need to be removed from a project team


2013-08-15

2013-08-15 Thursday - Revisiting MySQL


It has been awhile since I last checked-in on new features in MySQL.

I last installed MySQL 5.5 - since then there has been a 5.6 and 5.7 release.

Here are a few selected notes out of the many changes that I found of interest...

5.6 MySQL Release

  • MySQL now supports stronger encryption for user account passwords, available through an authentication plugin named sha256_password that implements SHA-256 password hashing
  • The EXPLAIN statement now provides execution plan information for DELETE, INSERT, REPLACE, and UPDATE statements. Previously, EXPLAIN provided information only for SELECT statements. In addition, the EXPLAIN statement now can produce output in JSON format.
  • Many interesting enhancements to the optimizer...
  • Condition handling. MySQL now supports the GET DIAGNOSTICS statement

5.7 MySQL Release
  • DDL performance for InnoDB temporary tables is improved through optimization of CREATE TABLE, DROP TABLE, TRUNCATE TABLE, and ALTER TABLE statements.
  • InnoDB now supports MySQL supported spatial data types. Prior to this release, InnoDB would store spatial data as binary BLOB data. BLOB remains the underlying data type but spatial data types are now mapped to a new InnoDB internal data type, DATA_GEOMETRY
  • Condition handling.  MySQL now supports stacked diagnostics areas.
  • Optimizer.  EXPLAIN can now be used to obtain the execution plan for an explainable statement executing in a named connection                         

2013-08-14

2013-08-14 Wednesday - Bret Victor == Inspiring




Inventing on Principle

2013-07-28

2013-07-28 Sunday - Oracle 12c R1 New Features [my notes]

I noted the following, in particular, while reading the documentation for Oracle 12c R1 New Features:
http://docs.oracle.com/cd/E16655_01/server.121/e17906/chapter1.htm#NEWFTCH1

Note that I have highlighted in red those item that were of particular interest to me:

 

1.1.1.17 Web Services Support (Oracle Application Express)
  • "Creating a PL/SQL API to interact with Web services" 
  •  "Exposing report regions, DML processes as Representational State Transfer (REST) Web services."
  • "Support of binary data types in Web services support"
  •  "Allow the inclusion of custom Simple Object Access Protocol (SOAP) headers with Web Services Description Language (WSDL) based Web services"
  • "Support of SOAP 1.2 in wizard-based Web services support."

1.1.4.3 Enhanced Oracle Native LEFT OUTER JOIN syntax
  • "...In Oracle Database 12c, the native syntax for a LEFT OUTER JOIN has been expanded to allow multiple tables on the left hand side...Merging of such views allows more join reordering and, therefore, more optimal execution plans"

1.1.4.4 JDBC Support for PL/SQL Data Types as Parameters
  • "The ability for Java and JDBC applications to bind PL/SQL package types and boolean types as parameters"

1.1.4.11 PL/SQL Functions Defined in the SQL WITH Clause
  • "You can define a PL/SQL function in the WITH clause of a subquery and use it as an ordinary function"

1.1.4.15 Temporal Validity
  • "...you can add one or more valid time dimensions to a table using existing columns, or using columns automatically created by the database"
1.1.4.16 Temporal Validity Flashback Queries
  • "Flashback Query has been extended to support queries on Temporal Validity dimensions. Users can now execute queries with the AS OF and VERSIONS BETWEEN clauses based on one or more valid time periods on the underlying tables"

1.1.6.3 IDENTITY Columns
  • "Table columns have been enhanced to support the American National Standards Institute (ANSI) SQL keyword IDENTITY"


1.1.6.4 Increased Size Limit for VARCHAR2, NVARCHAR2, and RAW Data Types
  • "The maximum size of the VARCHAR2, NVARCHAR2, and RAW data types has been increased from 4,000 to 32,767 bytes."

1.1.6.7 Native SQL Support for Query Limits and Row Offsets
  • "The FETCH FIRST and OFFSET clauses provides native SQL language support to limit the number of rows returned and to specify a starting row for the return set."

1.1.7.1 Microsoft .NET Framework 4 and 4.5 Support
  • "Oracle Data Provider for .NET supports .NET Framework 4 and 4.5, including Client Profile"

1.1.8.2 Database Support for JDK Upgrade
  • "New support in this release allows a customer to upgrade...from JDK 1.6 to JDK 1.7 and conversely downgrade from JDK 1.7 to JDK 1.6"


1.1.8.6 JDBC Support for Database Resident Connection Pool
  • "This feature allows large scale deployment of Java applications (typically hundreds or thousands of middle tier connecting to the same database). Orders of magnitude reductions of database server processes and memory are seen with this new feature"

1.2.1.1 Decision Tree Mining Text Data
1.2.1.2 Expected Maximization (EM) Clustering and Density Estimation
1.2.1.3 Feature Extraction Using Singular Value Decomposition
1.2.1.4 Feature Selection and Creation for Generalized Linear Models (GLM)
1.2.1.5 Native Double in Data Mining Functions
1.2.1.6 Native SQL Support for Row Pattern Matching
1.2.1.7 Native Text Support
1.2.1.8 On-the-Fly Models
1.2.1.9 Prediction Details and Cluster Functions

1.3.4.3 SecureFiles is the Default for the LOB Storage
  • "In this release, SecureFiles is now the default for LOB storage when the compatible initialization parameter is set to 12.1 or higher."

1.4.1.3 Multitenant Architecture
  • "The multitenant architecture is new in Oracle Database 12c Release 1 (12.1). You can have many PDBs inside a single Oracle Database occurrence. PDBs are fully backwards compatible with an ordinary pre-12.1 database."
  • Benefits:
    • " Fast provisioning of a new database or of a copy of an existing database."
    • "Fast redeployment, by unplug and plug, of an existing database to a new platform."
    • "Quickly patch or upgrade the Oracle Database version for many databases and for the cost of doing it once."
    • "Patch or upgrade by unplugging a PDB and plugging it into a different CDB in a later version."
    • "A machine can run more database instances in the form of PDBs than as individual, monolithic databases."

1.5.3.1 Global Data Services (GDS)
  • "Global Data Services (GDS) is a new capability of Oracle Database that extends the concept of services, which are only available in Oracle RAC, to a globally replicated configuration involving a combination of Oracle RAC, Active Data Guard, and Oracle GoldenGate. This allows services to be deployed anywhere within this globally replicated configuration, supporting load balancing, high availability, database affinity, and so on.
  • "Customers who have utilized the concept of services for Oracle RAC can now extend the same benefits of automatic workload management to their Active Data Guard or Oracle GoldenGate configurations. Similarly, single-instance Active Data Guard or Oracle GoldenGate customers can now fully utilize the benefits of services and automatic workload management for their replicated configurations."

 1.5.4.1 Oracle ASM Disk Scrubbing
  • "Oracle ASM disk scrubbing is a new feature that checks logical data corruptions and repairs them automatically in normal and high redundancy disk groups. This feature is designed so that it does not have any impact on normal I/O in production systems. The scrubbing process repairs logical corruptions using the mirror disks. Disk scrubbing leverages the Oracle ASM rebalancing to minimize I/O overhead."

1.11.3.1 Enhanced ANYDATA Support
  • "Restrictions have been removed from the ANYDATA implementation that prevented its use with Abstract Data Types (ADT) that contained attributes whose data type was LOB or XMLType. This enhancement increases the flexibility of the Oracle ANYDATA implementation and it can now be used with database editions."
1.11.3.3 Oracle XDK/J DOM Improvements
  •  "...adds support for W3C DOM Level 3 Core API's and reduces the memory footprint associated with using XML schemas."
 1.11.3.8 Support XQuery Full-Text Specification
"...extends Oracle's support for the W3C XQuery specification by adding support for the XQuery full text extension. This enables customers to perform XML-aware full text searches on XML content stored in the database."

 1.11.3.11 Support for XQuery Update
  •  "...allowing users to perform fragment and node-level updates using the W3C standard query language."

2013-07-28 Sunday - Development Lab Refresh


This weekend I've been updating a number of tools and applications in my development lab:

Oracle Database 12c R1
http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index-092322.html?msgid=3-8658271549

Docs
http://www.oracle.com/technetwork/database/enterprise-edition/documentation/index.html

12c R1 New Features:
http://docs.oracle.com/cd/E16655_01/server.121/e17906/chapter1.htm#NEWFTCH1


Oracle VirtualBox 4.2.16
https://www.virtualbox.org/

Ubuntu.com - 13.04 Raring Ringtail

Notepad++6.4.3
http://notepad-plus-plus.org/

I'm also planning to upgrade to Sparx Enterprise Architect 10 later this week
http://www.sparxsystems.com/


2013-07-27

2013-07-27 Saturday - Some New SOA and Cloud Links

SOA Community Blog by Jürgen Kress
http://soacommunity.wordpress.com/

An interesting new series of articles published by Oracle, entitled "Industrial SOA"

Oracle Practitioner Guide
Creating a Roadmap to Cloud Computing
Release 3.0
April 2013
http://www.oracle.com/technetwork/topics/entarch/oracle-pg-cloud-roadmap-r3-0-1940183.pdf

2013-07-23

2013-07-23 Tuesday - Book Review: Cloud Computing, Concepts, Technology & Architecture, by Thomas Erl


Book Review:

Cloud Computing
Concepts, Technologies & Architecture
By Thomas Erl, with Zaigham Mahmood and Ricardo Puttini
Pearson Education, The Prentice Hall Service Technology Series with Thomas Erl


First, let me say - this is an excellent book. The content is well organized, well written, and is useful as a reference for both business as well as technical personnel. With the information in this book, you will have established a firm foundation from which to define pilot projects to explore Cloud Computing initiatives within your own company, as well as being better informed as to what considerations should be of paramount concern if you need to work through an RFI, RFP, or RFQ process.  Consequently, you will also be much better prepared for evaluating various Cloud Vendor service offerings.

What I particularly appreciate about the Thomas Erl series of books published by Pearson Education in the Prentice Hall Series, is the obvious attention to detail and thoroughness of the the treatment for a given topic. This book continues that fine tradition.

With such a broad and deep topic as Cloud Computing, certainly no one book can be the definitive reference. However, this one is a good first step to acquiring mastery within the domain of Cloud Computing, and I highly recommend that you add it to your bookshelf.  It will help you navigate further in your exploration of the complex issues and challenges in devising cloud strategies.

Another excellent benefit of this book (for a large organization): Providing a common lexicon as the basis for establishing a common understanding of the concepts, architecture patterns, risks, and terminology of Cloud Computing.

Of particular note: Interspersed throughout the text are citations of real-world case studies.

At 487 pages, this is a great value for the price.

Chapter 1: Introduction
Chapter 2: Case Study Background

Part I: Fundamental Cloud Computing
Chapter 3: Understanding Cloud Computing
Chapter 4: Fundamental Concepts and Modules
Chapter 5: Cloud-Enabling Technology
Chapter 6: Fundamental Cloud Security

Part II: Cloud Computing Mechanisms
Chapter 7: Cloud Infrastructure Mechanisms
Chapter 8: Specialized Cloud Mechanisms
Chapter 9: Cloud Management Mechanisms
Chapter 10: Cloud Security Mechanisms

Part III: Cloud Computing Architecture
Chapter 11: Fundamental Cloud Architectures
Chapter 12: Advanced Cloud Architectures
Chapter 13: Specialized Cloud Architectures

Part IV: Working With Clouds
Chapter 14: Cloud Delivery Model Considerations
Chapter 15: Cost Metrics and Pricing Models
Chapter 16: Service Quality Metrics and SLAs

Part V: Appendices
Appendix A: Case Study Conclusions
Appendix B: Industry Standards Organizations
Appendix C: Mapping Mechanisms to Characteristics
Appendix D: Data Center Facilities (TIA-942)
Appendix E: Emerging Technologies
Appendix F: Cloud Provisioning Contracts
Appendix G: Cloud Business Case Template

2013-06-29

2013-06-29 Saturday - A solid beginning foundation for a Mobile Strategy



A solid beginning foundation for a Mobile Strategy

If you are a bit bewildered by what it means to implement a Mobile Strategy, then this book offers a solid conceptual foundation upon which to establish the beginning of a Mobile Strategy.

The first pleasant surprise on reading this book was the realization that Dirk Nicol  (Program Director for IBM Mobile Strategy and Production Management) wrote this with a true intent to educate, and while he could have used this as a platform to promote an IBM-specific agenda - he did not. Instead, he brought his obviously formidable depth of expertise - and lays out a framework for evolving a Mobile Strategy that can be applied to businesses both large and small.  No mean feat, that.

Who Should [Really] Read This Book?
- IT Executives pondering a mobile strategy
- IT Managers being tasked to implement a mobile strategy
- IT Business Analysts that need to understand the subtle transformative power of mobile to business processes, and its unique development processes

What this book is not:
- It is not an introduction to the mechanics of implementing mobile applications
- Although it touches on technical concerns, it does not intend to be an introduction to mobile technical specifications

Chapters include:
1 - Introducing Mobile Enterprise
2 - Defining Business Value
3 - Mobile Business Challenges
4 - The Mobile Framework
5 - Mobile Development
6 - Mobile Security and Management
7 - Mobile Business Transformation
8 - Planning a Mobile Project
9 - Social+Cloud+Big Data+Mobile
10 - International Considerations
11 - Case Studies and Mobile Solutions
12 - Moving Forward

Full Disclosure: I was provided a free copy of this book by Pearson Education to review

2013-06-26

2013-06-26 Wednesday - Design and Code Reviews

Recently I've been helping a large enterprise development program provide guidance to multiple teams of developers (100+ developers) via design and code reviews.

I've enjoyed the opportunity to lead these discussions - and focus on encouraging, nurturing, educating, and mentoring - versus the less productive and negative connotations of review-as-blame, as is often the unintended result of how reviews are perceived by developers.

For each build within a sprint, I typically review a SVN Diff file, and release notes - which allows me to focus on what has changed.

Although I usually don't limit my findings or comments to just the Diff details - it helps me focus and optimize the time I spend preparing for the review meeting itself (which is usually time-boxed to 30 minutes).

My first priority in the review is to identify critical sections that need refactoring (either due to observed defects, or known issues that will arise from a maintenance perspective, or inherent limitations in the implementation of the code that will affect the desired functionality)

My second priority in the review is to identify recommendations for improving the design of the solution so that it is maintainable, testable, and extensible.  "Elegance" is a subjective term, but it is the essence of what I strive to communicate.

The third priority in the review is often to help developers learn to identify anti-patterns that can be re-mediated to improve the overall performance and robustness of the code.


If your organization might benefit from having an outside independent 3rd party provide Design and Code Review feedback, please contact me to discuss your program needs.


A useful Big-O reference for developers::
http://bigocheatsheet.com/

(more to be added this weekend when I have more time to continue this post)

2013-05-27

2013-05-27 Monday - Book Review

Database Design for Mere Mortals, Third edition, by Michael J. Hernandez
Addison Weseley
http://www.amazon.com/gp/product/0321884493

You can read my review here:
http://www.amazon.com/review/RR23UR4KIVUQH/ref=cm_cr_pr_perm?ie=UTF8&ASIN=0321884493

What a pleasure to read Michael J. Hernandez's recent update to Database Design for Mere Mortals, Third Edition

I wasn't familiar with this book previously, and so had the chance to write a fresh review with no preconceived ideas from previous editions.

First, the text is well organized
Second, the examples are clear and concise
Third, the principles and underlying ideas are explained so that the reader understands not just the WHAT, but the WHY

This book is an excellent reference book for the developer who hasn't yet established a solid grounding in database design concepts. The thoroughness with which Mr. Hernandez covers the subject matter will go a long way to smoothing any gaps in one's conceptual understanding and approach to database design.

2013-04-13

2013-04-13 Saturday - Go Programming Learning Resources

I'm spending some time today exploring the Go Programming Language
http://golang.org/
http://code.google.com/p/go/
http://code.google.com/p/go/source/browse
http://code.google.com/p/go/issues/list


Go Programming Language Blog
http://blog.golang.org/

Google Group: golang-nuts
https://groups.google.com/forum/?fromgroups#!forum/golang-nuts

...and working my way through some of the various articles and documentation
http://golang.org/doc/install
http://code.google.com/p/goclipse/wiki/GettingStarted
http://golang.org/doc/effective_go.html
http://golang.org/doc/code.html
http://golang.org/doc/articles/wiki/
http://golang.org/doc/


"GoClipse is an Eclipse plugin that adds IDE functionality to the Eclipse platform for the Go programming language. The purpose of GoClipse is to create an environment in which development of Go is easy for a beginner to intermediate user"
https://code.google.com/p/goclipse/

I've installed the goclipse 0.7.6 plugin  into Eclipse Juno (4.2) SR2
http://goclipse.googlecode.com/svn/trunk/goclipse-update-site/

 3:15pm update...

After experimenting with the goclipse plugin for a few hours...I've experienced what seems to be a number of issues with it...

1) The goclipse wiki documentation appears to be woefully out of date (see references to a non-existent /cmd directory)
See: http://stackoverflow.com/questions/9957449/cant-use-packages-in-goclipse

The goclipse exemplar code in the wiki (for the clock.go module) needs to be modified as follows:

package clock

import (
    "time"
)

func IsAM() bool {
    localTime := time.Now()
    return localTime.Hour() <= 12
}

2) Moving a directory/file from the pkg folder to the src folder doesn't appear to trigger auto-rebuild(?)

3) Directories/packages created under /src are not automatically detected by goclipse

4) Adding a new directory (in either src or pkg) to the build path doesn't appear to help goclipse find the source file

5) When executing via the Run-As command, a Go console sometimes (?) didn't open

6) Finding/Importing a system library in a source file under the /src directory failed (e.g. "time" - works fine, if the source file is under the \pkg directory, but won't work if it is in the /src directory)?

7) Creating a custom package in the /pkg directory can't be found
8) Sometimes(?) unable to delete the .exe generated in the \bin directory, from within Eclipse.




As of today, there were 51open issues for the goclipse project (which is 50% of all bugs to-date)
http://code.google.com/p/goclipse/issues/list

Note: there were [5] updates to the update site (under subversion)  in all of 2012 (rev 440 ==> 0.7.6) was the last on 12/18/2012  - and no updates in 2013 (yet)
http://code.google.com/p/goclipse/source/browse/

Google User Group for goclipse
https://groups.google.com/forum/?fromgroups#!forum/goclipse
NOTE: doesn't seem to be very active...


If you prefer vim or emacs as your editor, you may want to investigate Gocode (an auto-completion daemon) http://marketplace.eclipse.org/content/goclipse

For users of the Sublime editor:
http://www.sublimetext.com/2
https://github.com/DisposaBoy/GoSublime

Notepad++
http://go-lang.cat-v.org/text-editors/notepad-plus-plus/

Another possible IDE:
https://code.google.com/p/golangide/


stackoverflow articles that reference goclipse
http://stackoverflow.com/search?q=goclipse

Other blog postings related to using goclipse
http://richitexperience.wordpress.com/2012/10/19/goclipse_preferences/
http://richitexperience.wordpress.com/2012/11/05/goclipse-project-properties-dont-forget-it/

Interesting Blogs that discuss aspects of Go programming
http://blog.natefinch.com/2013/01/go-is-for-open-source.html
http://jordanorelli.tumblr.com/post/42369331748/function-types-in-go-golang

Other Go Language Resources
http://go-lang.cat-v.org/

John Graham-Cumming's CloudFlare presentation: Go Concurrency
http://www.slideshare.net/jgrahamc/go-oncurrency

A List of Go Projects
http://code.google.com/p/go-wiki/wiki/Projects

2013-04-02

2013-04-02 Tuesday - Book Review, Database Design for Mere Mortals

Received a copy of a new book to review for Pearson Education.  Database Design for Mere Mortals by Michael Hernandez



2013-03-07

2013-03-07 Thursday - On Creativity


I've been pondering the proceses of creativity and inspiration lately...and happened to stumble upon the excellent  brainpickings.org - where I found these two gems:


John Cleese on the 5 Factors to Make Your Life More Creative [link]

...which ties in nicely with this quote...
"A wise man once said that all human activity is a form of play. And the highest form of play is the search for Truth, Beauty and Love. What more is needed? Should there be a ‘meaning’ as well, that will be a bonus?
If we waste time looking for life’s meaning, we may have no time to live — or to play."
- Arthur C. Clarke [link]

Some of the 'triggers' I usually find helpful in stimulating my own creative thought processes:

  • A long drive.  Some of my best thinking - and innovative breakthroughs to problems - have occured while driving along the coast, or through the countryside
  • Walking through a museum or gallery.  When I lived in the Washington D.C. area - a favorite Sunday activity was to walk through the National Gallery of Art - where I fell in love with one particular painting [Ships in Distress off a Rocky Coast], by Ludolf Backhuysen, a Dutch painter who lived from 1631-1708. 
  • Quiet Contemplative Meditation after a vigorous workout or 
  • A long run.
  • A long hot shower
  • A professional one hour massage, followed by a sauna
  • Reading  deeply on a complex, subject completely unrelated to the problem at hand.  Somehow the mind finds the tangents and serendipity spreads her magic dust - and Viola! - inspiration arrives with a grand entrance.
  • Music. Vivalidi's Four Seasons, for some reason, has the power to inspire me in ways that are hard to describe.




2013-03-03

2013-03-03 Sunday - OAuth 2.0 ?


This post is a reminder for me to come back and do some additional reading on a few interesting things I found this weekend...

 hapi appears to be an interesting Node.js HTTP Server framework which is under development - led by Walmart Labs

This led me to Eran Hammer's personal web site [http://hueniverse.com]

...which led me to some interesting things he has said about OAuth 2.0
http://hueniverse.com/2012/07/oauth-2-0-and-the-road-to-hell/
http://hueniverse.com/2012/11/fuckoauth-realtimeconf/


 He's also written a series of articles about his experience working with Node.js
http://hueniverse.com/2011/06/6-months-with-node-js/
http://hueniverse.com/2011/06/the-style-of-non-blocking/
http://hueniverse.com/2011/06/node-js-from-couch-to-mongo/

2013-02-17

2013-02-17 Sunday - JDK 8


link to the JDK 8 Milesstones
http://openjdk.java.net/projects/jdk8/milestones

2013-02-17 Sunday - Graphing in the Browser


I've been interested for a number of years in the Graphviz tool.

It seems to me that there are some interesting uses that this type of tool would lend itself to quite well...

Recently I wondered if a JavaScript implementation might have already been developed...and found the following that might be of interest to others:

http://code.google.com/p/canviz/

http://ushiroad.com/jsviz/

https://github.com/mdaines/viz.js


Another possibly interesting JavaScript library, along similar lines, might be jsPlumb

http://code.google.com/p/jsplumb/

https://github.com/sporritt/jsPlumb

http://jsplumb.org/doc/usage.html

Some blogs/articles that reference graphviz usage:

http://spin.atomicobject.com/2013/01/30/making-diagrams-with-graphviz/

2013-01-14

Book Review: Java EE Development with Eclipse

Updated: 2013-01-20 Sunday 

Packt Publishing has invited me to write a review on one of their new books, Java EE Development with Eclipse:
http://www.packtpub.com/java-ee-development-with-eclipse/book

A few first impressions on completing an initial read of the book:
  • 426 pages, with 10 chapters 
  • Instructions are clear
  • Screen shots are clear and helpful
  • Level of detail and complexity of the examples is appropriate
  • Code is available for download
  • Chapters are of an optimum length
  • The book will help someone who is already familiar with Eclipse learn more about the various Java EE development capabilities that are available within Eclipse (and supplemented by the Oracle Enterprise Pack for Eclipse - which includes integrated support for Oracle WebLogic Server 12c)
  • Eclipse 3.7 was used for developing the sample code and screen shots in the book
A few observations are worth mentioning as gentle criticisms:
  • Just reading the table of contents didn't convey a cohesive vision of why the topics were sequenced in the manner in which they were presented.  Perhaps a preliminary chapter could have outlined a simple enterprise application - and used the chapters to slowly demonstrated how various features of the application would be built with Eclipse and Java EE.
  • Using Ant as the build tool actually detracts from the clarity of the examples - by necessitating the discussion of where to download the required jar dependency files.  Perhaps the instructions perhaps could have been simplified by just relying on Maven or Gradle.
  • Chapter 4 (Creating a JSF Data Table) exposes a UI field for direct SQL command data entry -  which one hopes a novice programmer won't assume to be a standard best practice (for developing anything other than the most quick-and-dirty of hacks)
  • Whereas examples in Chapters 1 thru 7 seemed to be building examples or concepts presented in earlier chapters - Chapters 8 and 9 (JAX-WS Web Service and RESTful Web Services using the JAX-RS API) veered into the trivial HelloWorld examples.
  • Chapter 8 (Creating a JAX-WS Web Service) only addressed generating a web service based on annotation - and did not demonstrate Eclipse tooling capability to perform contract-first web service creation.

Chapter Outline

  • Chapter 1: EJB 3.0 Database Persistence
  • Chapter 2: O/X Mapping with JAXB 2.x
  • Chapter 3: Developing a Web Project for JasperReports
  • Chapter 4: Creating a JSF Data Table
  • Chapter 5: Templating with Facelets
  • Chapter 6: Creating Apache Trinidad User Interfaces
  • Chapter 7: Creating an AJAX Application
  • Chapter 8: Creating a JAX-WS Web Service
  • Chapter 9: RESTful Web Services Using the JAX-RS API
  • Chapter 10: Spring
Conclusion

Java EE is a large and complex topic.  Eclipse itself is a large and complex tool with a robust library of plugins.  A book such as this, in attempting to cover both - must necessarily make choices of what is included - and must maintain a stringent discipline to maintain brevity.  

This book provides a decent hand-held approach to guiding the reader through the mechanics of creating various Java EE application components.  It is not intended as an exhaustive reference, but more of a friendly guide to help you move along the trail a bit further and faster, than if you stumbled and bumbled by yourself in the dark.


 

Copyright

© 2001-2021 International Technology Ventures, Inc., All Rights Reserved.