2017-10-15 Sunday - Rabid Agile, A Gentle Criticism

One dictionary includes this definition for "Rabid": "Having or proceeding from an extreme or fanatical support of or belief in something."

This posting is intended to be a placeholder for collecting what I view as examples of fanatical Agile viewpoints held (or promoted) by others - based on my own frame of reference.


Signs You Might Be A "Rabid Agilist":


1. You Believe "The Source Code is The Design | The Documentation"

• https://intltechventures.blogspot.com/2018/07/2018-07-28-saturday-gentle-rebuke-to.html
• https://www.infoq.com/presentations/The-Code-is-the-Design
• https://less.works/less/technical-excellence/architecture-design.html
• " … the only software documentation that actually seems to satisfy the criteria of an engineering design is the source code"

2. Every Project Team Makes All Architecture Decisions Locally - No Governance

• You've misunderstood (or misread, or misapplied) the Scaled Agile Framework (SAFe)
• https://www.scaledagileframework.com/decentralize-decision-making/
  

3. You Think Every Project Can Be Forced to Fit Agile

I find the most rabid proponents of Agile (ready-fire-aim for every size project) have no appreciation for scale of when that is appropriate (perfect example of when NOT: fixed price contracts with vendors, massive development coordination between dozens of service providers, fixed deadlines - driven by market and regulatory forces) 

To an untrained eye, and based purely on that external person's observation, one might conclude that the Rabid Agilist revels in flagrant waste. 

Assuming an ∞ budget - they fail to recognize that the business has little tolerance for ∞ rework.

4. You Promote #NoEstimates Nonsense

 

• https://www.linkedin.com/in/jasonlegris/
  
• https://www.linkedin.com/posts/jasonlegris_ive-grown-to-feel-refining-an-itemstory-activity-6858089501529645056-Q4FO 
• "Estimation is reading tea leaves"
• "My plans don't have a time component."
• "Creating software is a lot like mining. It doesn't matter how much you estimate and plan, you don't have any idea what paths you will end up taking after you dig that first shaft. Reality does not bend to accomodate our wishes. Logic dictates we optimize around reality."

You are not being tasked with building a "warp drive" - most software development isn't breaking new theoretical ground - this isn't rocket science - many systems (of similar kinds) have been built (and rebuilt, and rebuilt) over many decades - so it is a poor excuse to whine it has never been done before. 

5. You Believe That The Only Participants That Matter are Developers and Customers

Fred George's "Programmer Anarchy" seems to hit all of the above, and then some:

• https://www.slideshare.net/fredgeorge/programmer-anarchy-and-managerless-processes
• GOTO 2015 • The Secret Assumption of Agile • Fred George

Caveat: Agile is a goodness, but certain practices/applications of it are not appropriate for every type/size of project.

Finally, I offer you: Meeks Software Architecture Conjecture #1:

The source code may (or may not) be a full implementation of the desired capability needed by the business - but is more likely just an approximation (constrained by permitted time, allocated budget, and available skills/talent of the team involved). Therefore it should not be confused with the actual or desired (or envisioned) design - that may require multiple years to achieve - of which the current source code may only reflect a partial (and incomplete, or inaccurate) representation.

 

Updates:

2024-05-15 Friday

An interesting 2023 post by Michael Küsters

• 10 Signs You're Facing an Agile Fanatic
  

 

2017-10-15 Sunday - Self-Organized Criticality - and Cycles of Corrective Collapse

In my reading this weekend - I happened upon the concept of "Self-Organized Criticality" - and that led me to consider the possible implications for evolutionary constraints on the design of some complex software systems.  The idea of "ongoing cycle of corrective collapse" resonates well with my casual observations of software needing to be refactored / rewritten - to simply make it maintainable and testable.

The point here is that organizations, generally, do not plan for such rework - they wait until the "corrective collapse".

"In 1987 a Danish physicist named Per Bak released a landmark paper introducing the concept of self-organized criticality. Bak observed that complex systems draw stability through an ongoing cycle of corrective collapses that keep the overall system from becoming too over-extended."

https://en.wikipedia.org/wiki/Self-organized_criticality



I've created this posting as a reminder to come back and revisit this idea in the future - I would like to explore it further in some white papers, when I have more time to write.

2017-09-17 Sunday - Cloud Decision Frameworks / Cloud Migration Strategies

On Sunday, I started collecting links to articles, checklists, and presentations that touch on the topic of providing guidance on Cloud Migration Strategies - as well as Cloud Decision Frameworks.

https://github.com/intltechventures/Lab.Architecture/blob/master/CloudComputing/Cloud_Strategy.md

Hopefully this may be of benefit to others.

2017-08-01 Tuesday - MuleSoft Performance & Best Practices

This posting is a placeholder for interesting information I happen to come across that is related to the topic of MuleSoft Performance & Best Practices.  Hopefully, others may find this of some use.

       
https://docs.mulesoft.com/mule-user-guide/v/3.8/reliability-patterns

• “Note that VM file persistency is disabled on clusters, so VM endpoints persist in-memory in a clustered topology.”
• “because we are calling a transformer, we have to allow for the possibility that it might fail and throw an exception. If it does, the file will be reprocessed, and might throw the same exception, and so on. To avoid an infinite loop, we used the redelivery policy configured at 2…”
•  “Reliability has performance implications”
• “If the outbound transport in the reliable acquisition flow is not transactional (for example, a flow from file-to-FTP), the only way to ensure message delivery is to turn off threading on the respective connector… If threading is not turned off, the caller may not notice the exception. That’s because the exception occurred in a different thread, and there is no way that one thread can see exceptions that occur in another thread.”

 

https://docs.mulesoft.com/mule-user-guide/v/3.8/tuning-performance

• “Incorrect combinations of thread configurations and maxBufferSize values can cause timeouts with no apparent cause. Counter intuitively, this issue is more probable with lower loads that don’t fully fill the buffer queue. This can cause outages. Load tests with high and low loads should be used to find and validate appropriate configurations.”

 
https://docs.mulesoft.com/mule-user-guide/v/3.8/distributed-file-polling
  

https://docs.mulesoft.com/mule-user-guide/v/3.8/file-transport-reference
https://docs.mulesoft.com/mule-user-guide/v/3.8/file-connector

• “The File connector only supports working with files on the same physical system than the running application. It doesn’t transfer files remotely over network connections. The only exception is when the operating system maps remote connections to local directories transparently”

https://docs.mulesoft.com/mule-user-guide/v/3.8/ftp-transport-reference

• “Put your login credentials in a properties file, not hard-coded in the configuration. This also allows you to use different settings between development, test, and production systems.”
• “The FTP transport does not support transactions as the File Transfer Protocol itself is not transactional. Instead you should design compensating transactions into your architecture using exception strategies in Mule”
• “If streaming is not enabled on the FTP connector, Mule attempts to read a file it picks up from the FTP server into a byte[] to be used as the payload of the MuleMessage. This behavior can cause trouble if large files need to be processed.”

https://docs.mulesoft.com/mule-user-guide/v/3.8/ftp-connector


https://docs.mulesoft.com/mule-user-guide/v/3.8/sftp-transport-reference

• “Mule 3.7 and newer supports SFTP retries. “

https://docs.mulesoft.com/mule-user-guide/v/3.8/sftp-connector

• “Mule uses the JCraft library for SFTP SSH”
• “SFTP Inbound Endpoints can only be set to a one-way exchange pattern. SFTP Outbound Endpoints can be set to either a one-way or request-response exchange pattern”

https://docs.mulesoft.com/mule-user-guide/v/3.8/configuring-reconnection-strategies

• "In addition to setting reconnection strategies on most connectors (Ajax, File, and VM are notable exceptions), you have the option to set them on global endpoints. (Once again, Ajax represents a major exception, which means that you can’t set a reconnection strategy on Ajax). However, MuleSoft recommends that whenever possible, you set your reconnection strategies on global connectors, rather than global endpoints, because this Best Practice generally allows you to reuse a once-written reconnection strategy again and again across all your flows and Mule projects."
• "The only situation in which MuleSoft recommends configuring a reconnection strategy on a global endpoint(rather than a global connector) involves Jetty, whose connector doesn’t support reconnection. This is why MuleSoft recommends the Jetty global endpoint instead."

http://www.mulesoft.org/docs/site/3.8.0/apidocs/org/mule/transport/file/ExpressionFilenameParser.html

• "ExpressionFilenameParser can use any expression language supported by Mule to construct a file name for the current message. Expressions can be xpath, xquery, ognl, mvel, header, function and more"

https://docs.mulesoft.com/mule-user-guide/v/3.8/exception-strategy-most-common-use-cases

https://docs.mulesoft.com/mule-user-guide/v/3.8/error-handling

• “Mule invokes a Messaging Exception Strategy whenever an exception is thrown within a flow…”
• “Each flow can contain only one exception strategy. However this can be a choice exception strategy that then refers to other nested exception strategies.
• “Choice exception strategies can contain one or more catch and/or rollback exception strategies. (Rollback and catch exception strategies cannot, however, contain other exception strategies.)”
• “Each exception strategy can contain any number of message processors.”
• “The exception strategy message processors should not throw exceptions, because you cannot create a nested exception strategy for these message processors”
• “Mule’s default exception strategy implicitly and globally handles all messaging exceptions that are thrown in Mule applications. You can override Mule’s default exception strategy by adding a catch, rollback, or choice exception strategy to a flow. Otherwise, when an exception is thrown in a flow, Mule automatically rolls back any pending transaction and logs the exception; if no transaction is involved, the default exception strategy simply logs the exception”

https://docs.mulesoft.com/anypoint-data-gateway/troubleshooting-data-gateway
https://docs.mulesoft.com/anypoint-data-gateway/

• "Anypoint Data Gateway allows you to easily integrate data stored in legacy back-office systems with Salesforce."
• "Data Gateway supports integration with systems such as SAP, Oracle, MS SQL, MySQL, and others"

Discussion Forum Findings…
https://forums.mulesoft.com/index.html
https://stackoverflow.com/search?q=mulesoft

For example:
https://forums.mulesoft.com/questions/68441/mule-performance.html
https://stackoverflow.com/questions/33230947/mule-ftp-polling-stops-without-error-or-warning


Some other interesting articles... 
http://www.modusbox.com/2016/01/tuning-mule-esb-for-high-performance-in-etl-scenarios/

https://www.appnovation.com/blog/mule-sftp-and-pgp-encryption 

https://dzone.com/articles/performance-tuning-in-mule-esb

https://stackoverflow.com/questions/39009139/pass-input-stream-over-http-in-mule-using-http-connector 

https://www.ricston.com/blog/batch-select-mule/

https://github.com/mulesoft/mulesoft-docs/blob/master/release-notes/v/latest/mule-runtime-3.8.0-m1-release-notes.adoc 

https://blogs.biztalk360.com/response-mulesoft-blog-10-reasons-walk-biztalk/

https://seekingalpha.com/filing/3543114 

2017-07-11 Tuesday - JavaScript Security/Audit/Vulnerability Checks

So, quite often, I have need to analyze a massively long list of JavaScript modules that are used in a vendor's solution, when I'm providing Enterprise Architecture oversight and assessment during the vendor evaluation phase of client RFP efforts.

To provide some confidence that there are no lurking serious security concerns with any of the JavaScript modules used in the vendor solution (and equally important - to identify those that are at, or beyond, end-of-life) - I would like to find a tool that would allow me to submit a file with the list of module names (with version numbers) - via a command line tool - and receive back some form of a report/analysis - identifying which ones may pose a high risk.

This posting is a placeholder for tools that I find that might be of utility in this effort - and hopefully of use to some future reader.

First, a quick survey of a Google search to help identify initial problem/solution domain articles to review:

• https://documentation.codeship.com/basic/languages-frameworks/nodejs/
• http://www.infoworld.com/article/3181432/javascript/safety-in-nodejs-nodesource-to-certify-npm-modules.html
• https://techbeacon.com/13-tools-checking-security-risk-open-source-dependencies-0
• https://blog.risingstack.com/controlling-node-js-security-risk-npm-dependencies/
• https://geekflare.com/nodejs-security-scanner/
• https://developers.redhat.com/blog/2017/04/12/using-snyk-nsp-and-retire-js-to-identify-and-fix-vulnerable-dependencies-in-your-node-js-applications/
• https://www.npmjs.com/package/snyk
• https://github.com/snyk/vulnerabilitydb
• https://nodesource.com/blog/nine-security-tips-to-keep-express-from-getting-pwned/
• https://channel9.msdn.com/coding4fun/blog/Guarding-the-code-with-Package-Security-Alerts-and-Roslyn-Security-Guard
• https://techcrunch.com/2016/10/11/facebook-partners-with-google-others-to-launch-a-new-javascript-package-manager/
• https://blogs.msdn.microsoft.com/visualstudioalmrangers/2017/01/17/manage-your-open-source-usage-and-security-in-your-pipeline/
• http://blog.myget.org/post/2016/10/14/Checking-potential-vulnerabilities-in-project-dependencies.aspx
• http://blog.bithound.io/checking-your-npm-dependencies-for-security-vulnerabilities/
• https://www.npmjs.com/package/auditjs
• https://www.npmjs.com/package/urbanjs-tool-nsp
• https://github.com/urbanjs/urbanjs-tools
• https://www.npmjs.com/package/urbanjs-tool-retire
• https://www.npmjs.com/package/run-jst-snyk
• https://www.npmjs.com/package/recink-snyk
• https://www.npmjs.com/package/nsp-audit-package
• https://www.npmjs.com/package/owdit
• https://github.com/nodesecurity/nsp

I'll post an update next week based on what I find to be of practical use from the above list.

2017-07-03 Monday - STRUTS 2 - CVE-2017-5638

https://f5.com/labs/articles/threat-intelligence/malware/from-ddos-to-server-ransomware-apache-struts-2-cve-2017-5638-campaign-25922

“this time, a “minerd” crypto coin mining program will be downloaded as well with all of its prerequisites. The attacker masquerades the malicious process and its configuration with names similar to Apache server, to make it look more innocent when the infected user will list all the running processes.”
"These cryptocoin pools appear to be hosted in France under the 'crypto-pool.fr'"
“One of the more fascinating aspects of this malware was the creative technique that the spearhead exploit uses to propagate itself. It will search for all the remote IP addresses that the administrator of the server was connecting to on this server. It searches the SSH “known_hosts” file, which keeps the IP addresses and fingerprints of all the servers to which the administrator was connecting. It also scans the Bash history file for any IP addresses used within the SSH command. Once this list of IP addresses is compiled, the script tries to connect to them via SSH. If the configured authentication was set up to use a key file instead of a username and password, the malware will successfully deploy itself on the remote machine.”
“The same attacker (surprisingly using the same IP address) behind the previously described Apache STRUTS campaign varied the campaign again during the week of March 20^th. This time, the payload infected Windows machines with the “Cerber” ransomware”

https://devcentral.f5.com/articles/apache-struts-remote-code-execution-vulnerability-cve-2017-5638-25617

https://www.alienvault.com/blogs/labs-research/apache-struts-vulnerability-being-exploited-by-attackers

https://security.berkeley.edu/news/critical-apache-struts-2x-vulnerability-cve-2017-5638

https://www.veracode.com/blog/security-news/struts-shock-vulnerability-affecting-apache-struts-2

https://www.ixiacom.com/company/blog/apache-struts-honeypot-scanning

https://www.whitehatsec.com/blog/apache-struts-cve-2017/

2017-07-01 Saturday - Continuous Delivery

I'm interested in how folks have implemented approaches to achieving Continuous Delivery, so this posting is a placeholder for interesting stories I happen to find.

Expedia (under leadership by the then Senior Release Program Manager, Sandy Anuras) initiated a project called the "Shuntless Release Project" (circa 2008-2010) - which allowed Expedia "...to reduce the downtime for a release from 8+ hours to 0 minutes...".  I would like to find out more about that specific effort's approach, but in the meantime, here are a few more recent presentation decks I found on SlideShare.net that may be of interest:

• Tech Week London 2017: How Expedia Runs Hundreds of Apps in Production Using Containers
• AWS re:Invent 2016: Serverless Computing Patterns at Expedia

2017-06-30 Friday - Dell Boomi

I'm doing some research today on the features and capabilities of Dell Boomi (as a cloud-hosted SaaS/iPaaS  integration layer).

These resource links may be of interest to others:

• https://boomi.com/integration/
• Dell Boomi Documentation
• AtomSphere User Guide
• Public Connectors
• SFTP Connector
• API Management User Guide
• Dell Boomi AtomSphere Process Shapes Visio Stencil

2017-05-16 Tuesday - HPE News: The Machine

https://news.hpe.com/a-new-computer-built-for-the-big-data-era/ 
 
"The prototype unveiled today contains 160 terabytes (TB) of memory, capable of simultaneously working with the data held in every book in the Library of Congress five times over—or approximately 160 million books."

"Based on the current prototype, HPE expects the architecture could easily scale to an exabyte-scale single-memory system and, beyond that, to a nearly-limitless pool of memory—4,096 yottabytes. For context, that is 250,000 times the entire digital universe today."

 *   160 TB of shared memory spread across 40 physical nodes, interconnected using a high-performance fabric protocol.
 *   An optimized Linux-based operating system (OS) running on ThunderX2, Cavium’s flagship second generation dual socket capable ARMv8-A workload optimized System on a Chip.
 *   Photonics/Optical communication links, including the new X1 photonics module, are online and operational.
 *   Software programming tools designed to take advantage of abundant persistent memory.

• https://www.labs.hpe.com/the-machine
• http://www.pcworld.com/article/3197054/hardware/hpe-shows-off-the-machine-prototype-without-memistors.html
  
• http://spectrum.ieee.org/computing/hardware/can-hpes-the-machine-deliver

2017-04-07 Friday - O'Reilly Early Release - Learning TensorFlow

New O'Reilly early release book on TensorFlow
http://shop.oreilly.com/product/0636920063698.do

2017-04-07 Friday - R and Plotly for Enterprise Architecture Heat Maps

I'm experimenting with v4.5.6 of the R library - working with that latest version of R v3.3.3 - crafting some Enterprise Architecture executive reporting views.https://plot.ly/ 

This weekend I will explore Shiny as well 

"A web application framework for R Turn your analyses into interactive web applications" http://shiny.rstudio.com/gallery/

http://zevross.com/blog/2016/04/19/r-powered-web-applications-with-shiny-a-tutorial-and-cheat-sheet-with-40-example-apps/ 

2017-02-23 Thursday - Flow and Deep Work

A thought-provoking article on the concepts of Flow and Deep Work - well worth the time spent to read it.
https://www.robinwieruch.de/lessons-learned-deep-work-flow/