Showing posts with label Best Practices. Show all posts
Showing posts with label Best Practices. Show all posts

2019-08-02

2019-08-02 Friday - Suggested Practices for Highly Effective EA Teams


Photo by Wayne Bishop on Unsplash
Source: https://unsplash.com/photos/7YUW7fvIYoQ,


 
Here's a quick start at creating a list of ideas for Suggested Practices for Highly Effective Enterprise Architecture (EA) Teams.

1) Asynchronous Communication: Your  team adopts and uses a web-based team collaboration/communication/chat tool. If your team is relying on email for your primary means of communication - if you are passing documents back and forth via email - you are the canonical example for why I wrote this list. Hint: You should be posting links to your repository-based artifacts.

2) Simplified Asynchronous Collaboration: You use git as your primary collaboration repository / publishing mechanism for EA artifacts - with a web UI (e.g. private Github, Bitbucket, etc.).  A wiki is an excellent choice as a complement for some categories of content (e.g. published for consumption outside of the EA team).

3) Simplified Processes for Asynchronous Authoring/Publishing: You use  markdown, stored in the git repositories, to write the majority of your EA documents.  Also see #2 above, re: Wiki.

4) Asynchronous Governance Processes: Your governance tooling and processes are based on the premise of a geographically distributed team - that operates in an asynchronous manner (i.e. artifacts are published for review, comments are collected, and voting is conducted - completely asynchronously). If the basis of your governance process is that you must interrupt team members ability to stay focused on high value tasks - by insisting on scheduling recurring mandatory governance meetings - on a very frequent basis - you are doing it wrong.  Meetings should be the exception - not the norm. Meetings for governance processes should primarily be called when there has been a lack of consensus in the voting - or there are significant questions/discussions that cannot be serviced via a discussion thread within a private (and obviously, secure) discussion forum tool.

5) Automated Reminders: Your governance tooling and processes are designed to send out timely reminders for assigned tasks to be completed.

6) Automated Search: You leverage powerful automated search tools (e.g. Apache Solr, Elastic Search, etc.) to make finding artifacts easy and painless.

7) Automated Annotation: You have processes that automate the majority of the effort to annotate, tag, and index the entire corpus of all the artifacts in your EA artifact repository.

8) Diagramming (Elements: - Root Definition/Reuse) : Whatever EA diagramming solution you adopt - supports a core capability of managing a master reference inventory of element definitions; reusing those definitions in different diagrams; easily visualizing the AS-IS/transition/TO-BE views.  (hint: As a baseline example for this type of capability, look at the Diagram Filter capability of Sparx Enterprise Architect. YouTube demonstration video)

9) Diagramming (Element Relationships/Connectivity): Creating relationships between elements; and being able to quickly and easily explore, discover, query, reuse, and report the elements in the master inventory - across different diagrams. (hint: LucidChart, Gliffy, SmartDraw, Creately, Archi, Google Draw, LibreOffice Draw, PlantUML, Umbrello, and Visio are not such solutions). Automated Dependency Impact Analysis is thus possible.

10) Asynchronous (Diagram) Repository Collaboration: EA team members are able to collaboratively work together, asynchronously, in the same repository - while crafting diagrams, components, etc.

11) A Culture of Cultivating EA Artifact Reuse: There is a process defined, resources are staffed (rotated assignment among EA team members is suggested) and effort is allocated - to continually support the creation, harvesting, management, and refresh of reusable artifacts, exemplars, patterns, templates, white papers, technology position papers, etc. - to help accelerate/optimize the efforts of the team.

12) EA Kaizen: You conduct frequent retrospectives to review WHAT you do, HOW you do it - and analyze your own EA processes for improvement. Minimally, this should be done at least quarterly. This means EA should have a BACKLOG of improvements to manage.

13) Ruthless Efficiency: The relative cost vs. value of governance processes are rigorously challenged - before adoption, and are reviewed periodically for adjustment - or elimination.

14) Secure Asynchronous External Collaboration: You leverage cloud-based, encrypted-at-rest, file storage mechanisms for collaboration with external partners (e.g. Box, DropBox, Google Drive, even private Github repositories, etc. - GPG encrypted files, if/when needed/warranted)

15) Automated Generation/Update of an EA Dashboard: You need to tell a story to your peers and stakeholders. A dashboard is a good starting point. But, you cannot really afford the luxury to assign vital resources to manually assemble/update such a dashboard. So, yours must be automated. Some ideas for possible metrics to collect (automatically), for a selected look-back period (e.g. Last Week, Last Month, Last Quarter, Last Year, vs. ~Current Period): Number of Artifacts Created, Modified; Number of Governance Reviews Scheduled, Completed; Governance Review Outcomes, by Status w/Counts; Diagrams Created, Modified; Diagram Repository Elements/Components Created, Modified; etc. If you have adopted the other recommendations in this list (in particular,  #2, #3, #8, #9, #14) - then you have a solid basis on which to simplify the automation for information collection, analysis, and publishing.

16) Internal URL Shortener:  You use an internal, enterprise-wide URL shortner. This allows you to manage updates/corrections to the final target - without having to edit/update documents everywhere. Bonus Points: A separate batch refresh process to associate a computed hash of the files that URLs that point to - so that you can identify and rationalize/consolidate references to duplicate instances of documents.

17) Daily Journals: Each EA team member publishes a Daily Journal - that is visible to the team.  Wiki or git Markdown files suggested. This does four things for team members:
1) Asynchronously catch-up on status updates - without interrupting conversations, and avoid the  n (n – 1) /2 communication channel servicing problem;  
2) Tribal knowledge is captured;  
3) Reduces the need for team meetings - members can just quickly read/scan each of the members' most recent daily journals for an update.;
4) Supports Business Continuity - in the event someone leaves the team unexpectedly/suddenly.

18) Continuous Knife Sharpening: On a rotating, periodic basis - each EA team member is tasked  with researching, organizing, and giving a one-hour demonstration / technical talk on some new/interesting area of technology, methodology, strategy, practice, etc. Suggested minimal frequency: Monthly. Invited speakers from other internal groups (or vendors, or other companies are also good variations to consider).

19) Awesome Lists: There should be an "Awesome List" git repository - in which team members can record interesting, useful new ideas, resources, articles, open source (or vendor) solutions.  This creates a valuable, persistent knowledge repository for the team - that grows over time. Hint: If you are primarily using emails (or tools like Slack) to communicate such information to the team - you are doing it wrong. The added benefit of this approach is that new team members have immediate access to the historical record of the team's growing body of knowledge (which they won't have - if you continue to just send emails to each other - and Slack sucks for scrolling back in time).

20) Tips Repository: Rationale: See "Awesome List" #19 above. Within this repository are separate Markdown files, with the following suggested naming convention: Tips.{subject area}.md (examples)

21) Automated Knowledge Dissemination: Automated publishing of content for non-IT consumption - from the repositories and automated governance processes - is greatly simplified.  This eliminates a huge cost barrier to making EA artifacts widely available across the enterprise (i.e. automated publishing of static HTML content, or Markdown files - vs. having to pay massive licensing fees for users to access a more complex commercial EA tool/repository).  Sharing is Caring.

2018-06-22

2018-06-22 Friday - Advice to a Development Team Struggling with Delivery & Defects

Recently I had a discussion with the CEO of a  .NET focused software development company - who was concerned with recurring delays in his development organization's ability to consistently meet agile sprint delivery schedules - and an attendant problem of unexpected defect rates in the code that was delivered.  

After our brief discussion, I suggested that - based on the symptoms he  described - my initial suggestion would be to ensure that the following three areas were not contributors  - and were well understood by the development teams . 

Possible Root Causes To Investigate (in the near-term):

1) Developers may not be adequately trained in agile practices and the language/tools they are using
  • Defects will tend to cluster around either specific persons and/or  specific code modules.  Root-cause analysis is essential to properly determine the appropriate mitigation actions, such as...
  • Use the 5-Whys of Six Sigma to determine the root cause for defects - often it may come down to a lack of knowledge/training - being tasked to do something outside their area of expertise - lack of awareness of good code design principles, etc.
  • For example (from my 2015 Seattle Code Camp talk, Diagnosing The Patient):
    • 1st: Why is the system slow?
      • The DB is slow
    • 2nd: Why is the database slow?
      • The app executes a lot of queries to load a page
    • 3rd: Why is the app designed to be so chatty?
      • DB was designed in an inefficient manner by [X]
    • 4th: Why didn’t [X] use an efficient DB design?
      • [X] was never trained on database design
    • 5th: Why didn’t [X] get any DB design training?
      • There is no budget for training
2) Developers may not be following Test Driven Design/Development (TDD) practices
  • This can be due to a number of possible causative factors:
    • Developers may be overwhelmed with unrealistic workloads squeezed into sprints
    • Developers may not be adequately trained/experienced to properly estimate Epics/Stories/Tasks - and therefore are simply racing as fast as they can to crank out the code for a given sprint
    • Developers may be writing trivial tests - and not focusing on the 20% that represents the `critical section` - first. 
3) Continuous Integration may not be in place (or taking advantage of all of its capabilities)


Uncle Bob Martin's book, Clean Code - is the foundation knowledge that is necessary for every developer to be successful:
https://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882



He also teaches a 3-day workshop


2017-08-01

2017-08-01 Tuesday - MuleSoft Performance & Best Practices

This posting is a placeholder for interesting information I happen to come across that is related to the topic of MuleSoft Performance & Best Practices.  Hopefully, others may find this of some use.

       
https://docs.mulesoft.com/mule-user-guide/v/3.8/reliability-patterns

  • “Note that VM file persistency is disabled on clusters, so VM endpoints persist in-memory in a clustered topology.”
  • “because we are calling a transformer, we have to allow for the possibility that it might fail and throw an exception. If it does, the file will be reprocessed, and might throw the same exception, and so on. To avoid an infinite loop, we used the redelivery policy configured at 2…”
  •  “Reliability has performance implications”
  • “If the outbound transport in the reliable acquisition flow is not transactional (for example, a flow from file-to-FTP), the only way to ensure message delivery is to turn off threading on the respective connector… If threading is not turned off, the caller may not notice the exception. That’s because the exception occurred in a different thread, and there is no way that one thread can see exceptions that occur in another thread.”
 

https://docs.mulesoft.com/mule-user-guide/v/3.8/tuning-performance
  • “Incorrect combinations of thread configurations and maxBufferSize values can cause timeouts with no apparent cause. Counter intuitively, this issue is more probable with lower loads that don’t fully fill the buffer queue. This can cause outages. Load tests with high and low loads should be used to find and validate appropriate configurations.”
 
https://docs.mulesoft.com/mule-user-guide/v/3.8/distributed-file-polling
  

https://docs.mulesoft.com/mule-user-guide/v/3.8/file-transport-reference
https://docs.mulesoft.com/mule-user-guide/v/3.8/file-connector
  • “The File connector only supports working with files on the same physical system than the running application. It doesn’t transfer files remotely over network connections. The only exception is when the operating system maps remote connections to local directories transparently”


https://docs.mulesoft.com/mule-user-guide/v/3.8/ftp-transport-reference
  • “Put your login credentials in a properties file, not hard-coded in the configuration. This also allows you to use different settings between development, test, and production systems.”
  • “The FTP transport does not support transactions as the File Transfer Protocol itself is not transactional. Instead you should design compensating transactions into your architecture using exception strategies in Mule”
  • “If streaming is not enabled on the FTP connector, Mule attempts to read a file it picks up from the FTP server into a byte[] to be used as the payload of the MuleMessage. This behavior can cause trouble if large files need to be processed.”
https://docs.mulesoft.com/mule-user-guide/v/3.8/ftp-connector


https://docs.mulesoft.com/mule-user-guide/v/3.8/sftp-transport-reference
  • “Mule 3.7 and newer supports SFTP retries. “
https://docs.mulesoft.com/mule-user-guide/v/3.8/sftp-connector
  • “Mule uses the JCraft library for SFTP SSH”
  • “SFTP Inbound Endpoints can only be set to a one-way exchange pattern. SFTP Outbound Endpoints can be set to either a one-way or request-response exchange pattern”

https://docs.mulesoft.com/mule-user-guide/v/3.8/configuring-reconnection-strategies
  • "In addition to setting reconnection strategies on most connectors (Ajax, File, and VM are notable exceptions), you have the option to set them on global endpoints. (Once again, Ajax represents a major exception, which means that you can’t set a reconnection strategy on Ajax). However, MuleSoft recommends that whenever possible, you set your reconnection strategies on global connectors, rather than global endpoints, because this Best Practice generally allows you to reuse a once-written reconnection strategy again and again across all your flows and Mule projects."
  • "The only situation in which MuleSoft recommends configuring a reconnection strategy on a global endpoint(rather than a global connector) involves Jetty, whose connector doesn’t support reconnection. This is why MuleSoft recommends the Jetty global endpoint instead."


http://www.mulesoft.org/docs/site/3.8.0/apidocs/org/mule/transport/file/ExpressionFilenameParser.html
  • "ExpressionFilenameParser can use any expression language supported by Mule to construct a file name for the current message. Expressions can be xpath, xquery, ognl, mvel, header, function and more"


https://docs.mulesoft.com/mule-user-guide/v/3.8/exception-strategy-most-common-use-cases

https://docs.mulesoft.com/mule-user-guide/v/3.8/error-handling
  • “Mule invokes a Messaging Exception Strategy whenever an exception is thrown within a flow…”
  • “Each flow can contain only one exception strategy. However this can be a choice exception strategy that then refers to other nested exception strategies.
  • “Choice exception strategies can contain one or more catch and/or rollback exception strategies. (Rollback and catch exception strategies cannot, however, contain other exception strategies.)”
  • “Each exception strategy can contain any number of message processors.”
  • “The exception strategy message processors should not throw exceptions, because you cannot create a nested exception strategy for these message processors”
  • “Mule’s default exception strategy implicitly and globally handles all messaging exceptions that are thrown in Mule applications. You can override Mule’s default exception strategy by adding a catch, rollback, or choice exception strategy to a flow. Otherwise, when an exception is thrown in a flow, Mule automatically rolls back any pending transaction and logs the exception; if no transaction is involved, the default exception strategy simply logs the exception”



https://docs.mulesoft.com/anypoint-data-gateway/troubleshooting-data-gateway
https://docs.mulesoft.com/anypoint-data-gateway/
  • "Anypoint Data Gateway allows you to easily integrate data stored in legacy back-office systems with Salesforce."
  • "Data Gateway supports integration with systems such as SAP, Oracle, MS SQL, MySQL, and others"



Discussion Forum Findings…
https://forums.mulesoft.com/index.html
https://stackoverflow.com/search?q=mulesoft

For example:
https://forums.mulesoft.com/questions/68441/mule-performance.html
https://stackoverflow.com/questions/33230947/mule-ftp-polling-stops-without-error-or-warning


Some other interesting articles... 
http://www.modusbox.com/2016/01/tuning-mule-esb-for-high-performance-in-etl-scenarios/

https://www.appnovation.com/blog/mule-sftp-and-pgp-encryption 

https://dzone.com/articles/performance-tuning-in-mule-esb


https://stackoverflow.com/questions/39009139/pass-input-stream-over-http-in-mule-using-http-connector 

https://www.ricston.com/blog/batch-select-mule/


https://github.com/mulesoft/mulesoft-docs/blob/master/release-notes/v/latest/mule-runtime-3.8.0-m1-release-notes.adoc 

https://blogs.biztalk360.com/response-mulesoft-blog-10-reasons-walk-biztalk/



https://seekingalpha.com/filing/3543114 

2015-11-01

2015-11-01 Sunday - CDN Best Practices

This posting is a placeholder for resources I find interesting - related to CDN Best Practices - that may be useful to to others...

https://en.wikipedia.org/wiki/Content_delivery_network

I'm currently evaluating Amazon's AWS CloudFront as a possible CDN solution
http://aws.amazon.com/cloudfront/

http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/DownloadDistS3AndCustomOrigins.html

http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-creating-console.html

http://docs.aws.amazon.com/AmazonCloudFront/latest/APIReference/CreateDistribution.html

http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web.html

http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ServingCompressedFiles.html

http://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html#limits_cloudfront


S3 limitation on creating directory structures for serving CDN content
 see Phil P's last response - Apr 24, 2013 11:56pm
- this invalidated an assumption I had of using a CDN release directory structure with S3 - but seems to state that it could be done with an EC2 instance as a custom origin.

I also noted this entry, re: Invalidation of paths is not (?) supported with wildcards...

A question I've raised on the AWS CloudFront forums

https://azure.microsoft.com/en-us/documentation/articles/best-practices-cdn/ 
"Versioning and cache-control You must consider how you will update static content and deploy new versions. The CDN does not currently provide a mechanism for flushing content so that new versions are available. This is a similar challenge to managing client side caching, such as in a web browser."
"Testing It can be difficult to perform local testing of your CDN settings when developing and testing an application locally or in staging."
"You cannot set up a CDN endpoint for an application deployed in Azure staging, or in the local Azure emulator in Visual Studio. This will affect unit testing, functional testing, and final pre-deployment testing. You must allow for this by implementing an alternative mechanism. For example, you could pre-deploy the content to the CDN using a temporary custom application or utility, and perform testing during the period it is cached. Alternatively, use compile directives or global constants to control where the application loads the resources from. For example, when running in debug mode it could load resources such as client-side script bundles and other content from a local folder, and use the CDN when running in release mode."

A lists of CDN Vendors...
http://blog.streamingmedia.com/2014/07/cdnvendors.html

Heroku's Fastly
https://devcenter.heroku.com/articles/fastly

Marrying CDNs with front-end optimization
http://www.webperformancetoday.com/2012/05/14/cdn-feo-front-end-optimization-web-acceleration/

WordCount

Copyright

© 2001-2026 International Technology Ventures, Inc., All Rights Reserved.