intltechventures.blogspot.com

2015-06-06

2015-06-06 Saturday - Network Bandwidth Monitoring Tools

First let me preface this posting by saying, I am not a hardware guy - nor do I specialize in network configuration. You should definitely do your own homework...and consult the appropriate experts with your respective network service providers, and network hardware vendors.

I suspect most folks are not very familiar with the overhead of the TCP protocol - and would probably be astonished at the impact that even apparently minor differences can make in the limitations on actual bandwidth availability and utilization. A simple difference such as 10ms vs 50 ms in latency can make a huge difference.

Some day you may be in a situation where the monitoring metrics across your application tiers (such as CPU and memory utilization) are within well accepted ranges - but response times may seem quite excessive in a number of cases. One possible area to explore is your available theoretical network bandwidth vs actual network bandwidth - and what percentage of the actual available is being utilized. Here are a few possible areas to examine:

Is there an inefficient application in your data center that is behaving in an excessively chatty way (e.g. sending thousands of requests to complete the display of a single page)?
How is the setting for tcp window scaling option configured?
What is the profile of transactions across your network? Is there something that is using an excessive % of your available network bandwidth - that might be a candidate for refactoring - or somehow isolating from the web application tier?
What is the latency between your application tiers / servers / external third party services?
Are all of the devices in your network using a consistently sized Network Interface Controller adapter that supports the target maximum theoretical speed of your fastest NIC device?
Are your customer facing web applications on the same network segment as your heavy-lifting back-end systems?
Will your customer facing web applications benefit by partitioning some lower-priority / high data volume / large data packet consuming applications into one or more separate network segments?
If you upgraded your network to use 10 Gigabit Ethernet, did you also upgarde the cabling from cat 5 to cat 6 (or at least cat 5e) - to mitigate interference - which can impact packet loss.?
Is your network cabling sufficiently separated by some distance from your power cabling - to mitigate interference - which can impact packet loss?

I'm doing some research today, looking at various possible tools to help monitor network performance and bandwidth utilization for applications running in a data center, or across a corporate WAN.

This posting is a placeholder for some of my background reading and research...

Some basic background material...

An excellent list curated by staff at Stanford University...

http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html

Examples of some tools to provide basic types of operational monitoring capabilities you should have in place...

http://ganglia.info/papers/science.pdf

https://www.nagios.org/

http://en.wikipedia.org/wiki/HP_OpenView

Some help with basic calculations...

Some links to possible tools and vendor specific resources...

http://networksolutionexperts.com/wan-acceleration/

http://www.silver-peak.com/products-solutions

http://www.silver-peak.com/products-solutions/wan-optimization

http://www.itadmintools.com/

http://www.cisco.com/c/en/us/solutions/collateral/service-provider/quantum/white_paper_c11-728880.html

https://www.manageengine.com/

http://www.spiceworks.com/

http://go.solarwinds.com/

http://www.cacti.net/

http://etherape.sourceforge.net/

http://www.telerik.com/fiddler

http://dumeter.net/

http://www.hageltech.com/dumeter/about

http://www.netchain.com/netcps/

MRTG graphs to display bandwidth usage. MRTG (Multi Router Traffic Grapher)

https://www.plixer.com/Scrutinizer-Netflow-Sflow/scrutinizer.html

2015-05-25

2015-05-25 Monday - Visualizing Git Logs

In preparation of needing to do some in depth analysis of 1M+ lines of legacy code for an upcoming client engagement, I'm doing some research today on resources that may be useful in helping to analyze and visualize git logs. This posting is a placeholder for things that I find of interest:

2015-05-24

2015-05-24 Sunday - PHP & MySQL Scalability Research

I'm doing some research this week, - in advance of the start-date for a new client engagement - and this post is a placeholder for notes regarding my findings on PHP & MySQL scalability.

PHP Connections:

Scaling a PHP MySQL Web Application, Part 1

http://www.oracle.com/technetwork/articles/dsl/white-php-part1-355135.html

Scaling a PHP MySQL Web Application, Part 2

http://www.oracle.com/technetwork/articles/dsl/white-php-part2-355138.html

The following MySQL related books on Amazon may be of interest:

2015-05-12

2015-05-12 Tuesday - Dream Teams

Trust, Innovation, and Ability are three of the most critical qualities in my line of work that I value the most in people. When thinking of who I would call on to tackle a major new project, there is a short list of names I mentally review from time-to-time as the core of my primary dream team. Those folks are already in my LinkedIn network - and it seems I may have need to call on them soon...

2015-05-09

2015-05-09 Saturday - Graph Databases

I'm doing some research today on Graph Databases and happened to come across this very nice concise deck by Max De Marzi that provides some good examples of use cases that illustrate the benefits of Graph Database (using Nejo4j):
http://www.slideshare.net/maxdemarzi/bootstrapping-recommendations-with-neo4j

There's also a 2nd Edition of the O'Reilly book, Graph Databases now available.

2015-05-07

2015-05-07 Thursday - Architecture Documentation with Sparx

If your organization is like most - architecture documentation is often missing, out-of-date, inaccurate, or incomplete.

I offer a three week time-boxed service to work on-site with your teams and create Sparx Enterprise Architect models to capture your current-state architecture.

Please contact me via kmeeks@intltechventures.com to discuss a convenient time to schedule an engagement.

2015-04-15

2015-04-15 Wednesday - Lambda Architecture

An interesting presentation by Helena Edelson (@helenaedelson)
Lambda Architecture with Spark, Streaming, Kafka, Cassandra, Akka, Scala

2015-04-10

2015-04-10 Friday - 2015 Stack Overflow Developer Survey

http://stackoverflow.com/research/developer-survey-2015

2015-04-10 Friday - Experimenting with Grails 3.0.1

This afternoon I'm spending time becoming familiar with Grails (current version is 3.0.1)

2015-04-09

2015-04-09 Thursday - Martin Odersky Scala Days Keynote

This year, Scala Days was held in San Francisco March 16th-18th.

Martin Odersky's presentation from his keynote is available here.

2015-04-09 Thursday - Sparx Enterprise Architect 12

I've been a long-time user of the most excellent Sparx Enterprise Architect tool - and recently upgraded to Sparx Enterprise Architect 12 - and must say, I find it to be a pleasant surprise and significant improvement over previous versions: Fast, Crisp, Elegant.

You'll notice in the Project Browser window (in the upper right-hand corner) - there is now support for doing iPhone and Android Wireframes.

I've just sent an inquiry to Sparx to request permission to develop some training programs (to offer my clients, and publicly) - and to prepare some short free video tutorials that I could upload to YouTube in the near future.

2015-04-09 Thursday - Apache Qpid AMQP Messaging

Today I'm spending some time exploring Apache Qpid's AMQP messaging features

2015-04-09 Thursday - Java SE 8 Lambda Quick Start

I happened to come across this excellent tutorial today in the Oracle Learning Library:
Java SE 8: Lambda Quick Start

This is an excellent introduction for developers to help them understand the benefit of Java SE 8 Lambda expression features.

Based on my experience leading Java code reviews over the last 5 years for large corporations - I would estimate that those companies could achieve at least a 20-30% reduction in the size of their code bases by the judicious adoption of Java SE 8 Lambda expression capabilities.

2015-04-08

2015-04-08 Wednesday - InfoQ Survey: JavaScript Frameworks in the Real World

InfoQ has an interesting real-time survey of JavaScript Frameworks in the Real World in progress...

2015-04-02

2015-04-02 Thursday - PostgreSQL 9.4 JSON Support

I'm experimenting with the JSON support features in PostgreSQL 9.4 - this post is a placeholder for interesting things I want to remember.

PostgreSQL JSON related blog posts by others:

Andrew Dunstan's presenttation: PostgreSQL 9.4 and JSON
Michael Paquier: Indexing JSON data with jsonb data type
Robert Haas: Parallel Sequential Scan for PostgreSQL 9.5

PostgreSQL documentation links:

JSON Functions and Operators
What's New in PostgreSQL 9.4: JSON Binary JSON storage
What's New in PostgreSQL 9.3: JSON: Additional functionality
What's New in PostgreSQL 9.2: JSON datatype

2015-04-02 Thursday - PHP 7.0 aka PHPNG (Next Generation)

I'm doing a bit of background reading this afternoon - to refresh myself with the current state / future direction of PHP. Over the years I've created a few web sites, tools, and frameworks with PHP for my own internal use (e.g. a portfolio analysis web application, a rapid application development framework, CRUD/DTO modules emitted by a general purpose polyglot code generator).

There's a potential opportunity that might involve some PHP related architecture work - so this is a placeholder for some of my reading today:

http://php.net/manual/en/appendices.php

https://wiki.php.net/phpng

Dmitry Stogov's presentation: PHPNG A New Core for PHP7

Some noteable discussions of the PHPNG performance improvements...

https://wiki.php.net/rfc#php_70

https://github.com/php/
https://github.com/php/php-src

2015-04-02 Thursday - Visual Studio 2015 pricing

I noted that the pricing for Visual Studio 2015 Enterprise is much better than the previous Ultimate pricing...

https://www.visualstudio.com/products/vs-2015-product-editions

There will be three main editions: Community, Professional w/MSDN, and Enterprise w/MSDN

2015-03-31

2015-03-31 Tuesday - Preparation: Areas of Focus

“By failing to prepare, you are preparing to fail.”
― Benjamin Franklin

“Give me six hours to chop down a tree and I will spend the first four sharpening the axe.”
― Abraham Lincoln

I have some time between client engagements - and want to make the best use of this time to do some deep dives into areas to "sharpen the saw" and strengthen my hands-on technical skills. I've put together the following weekly schedule as a guideline for areas in which I want to focus my time:

2015-03-28

2015-03-28 Saturday - Dev Lab Updates

This weekend I'm going through and updating a number of tools in my Dev Lab environment (Windows 7):

Scala 2.11.6 (release notes)

Julia v0.3.7

PHP 5.6.7

Note: In a Windows 7 environment, you may need to install the Visual C++ Redistributable for Visual Studio 2012 - if you get the error message that msvcr110.dll is missing

Ruby 2.2.1

Apache Camel 2.15.0

FileZilla 3.10.2

I've also added a few new things:

ZeroTurnaround Eclipse Optimizer Plugin (v1.0.1) (Eclipse update site)
SonarQube Eclipse Plugin 3.4
Gradle 2.3
BaseX 8.1 - in particular, I have found the GUI useful for testing XQUERY statements

2015-03-27

2015-03-27 Friday - Book Review: Learn Ruby the Hard Way, Third Edition

My Book Review:

The good folks at Pearson's Addison-Wesley Professional have provided me a copy of Zed Shaw's book to review - that came out in December, Learn Ruby the Hard Way, Third Edition.

I'm enjoying working my way through the book - and particularly liked the subtitle: "A Simple and Idiomatic Introduction to the Imaginative World of Computational Thinking with Code"

The book is __jammed!__ with hands-on exercises (count'em: 52 exercises) - that are brief, direct, succinct - and move the ball constantly forward in terms of raising the readers technical skill with Ruby.

Often a reader will focus solely on the chapters of a book, and ignore the Appendix. That would be a mistake with this book. Zed has provided some very useful exercises to guide the reader through learning the Command Line tools - with a good reinforcing teaching pattern of "Do this...You Learned This...Do more...")

Zed's writing is direct, pragmatic, and focuses on transferring working knowledge of the Ruby language.