Sunday, July 19, 2015

2015-07-19 Sunday - MySQL Federated Engine

I'm doing some research this weekend on MySQL's Federated Engine.  This posting is placeholder for collecting various information I find that might be of interest to others.

  • FEDERATED table does not support indexes in the usual sense; because access to the table data is handled remotely, it is actually the remote table that makes use of indexes. This means that, for a query that cannot use any indexes and so requires a full table scan, the server fetches all rows from the remote table and filters them locally. This occurs regardless of any WHERE or LIMIT used with this SELECT statement; these clauses are applied locally to the returned rows
  • Queries that fail to use indexes can thus cause poor performance and network overload. In addition, since returned rows must be stored in memory, such a query can also lead to the local server swapping, or even hanging.

  • "a number of limitations and due to network latency are rarely used...not enabled by default and is no longer maintained by Oracle and it's use is discouraged"

Saturday, July 11, 2015

2015-07-11 Saturday - Seattle Code Camp 2015 (9.12.15)

I've submitted an idea for a talk at this year's 2015 Seattle Code Camp
"Often when invited into an organization as a consultant, every new environment/situation offers numerous new challenges: symptoms of inefficiencies and performance concerns...that span systems, people, processes, infrastructure, and organizational culture. This talk will focus on sharing lessons learned from years of experience - across many different technical platforms, many different industries - and hopefully provide you with a useful, holistic, conceptual framework to help you quickly assess, diagnose, triage, and begin treatment of the system-as-patient."

2015-07-11 Saturday - DNS configuration

In my normal day-to-day architecture roles for the last 10 years, I have seldom needed to perform many hands-on System Admin and Network Configuration tasks. However, I need to be more deeply involved in reviewing many aspects of infrastructure and network design details for a current client engagement. So today I'm doing some background reading on DNS configuration as part of a new cloud architecture design effort - and deployment planning.

Saturday, July 04, 2015

2015-07-04 Saturday - SaaS Performance Tuning Notes

When exploring possible SaaS performance issues, there are a couple of possible questions/concerns that might be worth examining as possible contributing factors:
  • Is there possible saturation of server CPU % utilization?
  • Is there possible saturation of server Memory % utilization?
  • Is there possible saturation of server Disk % utilization?
  • Is there possible saturation of the available network bandwidth?
    • Check the NIC % utilization on all servers, particularly for the database
    • What is the average response time within the data center network?
  • Is there possible database thread starvation occurring?
  • Are there inefficient SQL queries excessively consuming resources?
  • Does the application leverage any SQL query caching?
  • Is there lock contention occurring on database operations?
  • Would a Master-Slave configuration help distribute some database I/O 
    • Write against Master
    • Read from Slave (need to understand latency of Master to Slave replication)
  • Is there possible excessive session management overhead being incurred?
    • How much data is being stored in session management for an average user?
  • Is there possible File I/O contention for read/write of session data files?
  • Is there possible excessive overhead being incurred for static content delivery?
    • Is static content being served up by the same domain for which cookies are specified?
    • If so, cookies will be sent for each GET of a static content file can be instrumental in helping isolate problems - as well as additional monitoring tools (e.g. such as Ganglia, SolarWinds, etc.)

In a recent experiment, moving two static content files to an externally hosted Content Delivery Network (CDN) appeared to suggest a possible 66% reduction in page load time.

2015-07-04 Saturday - R Programming Language (v3.2.1)

Experimenting today with the newest 2015-06-18 release of the R programming language (3.2.1)

exploring The R Manuals

As well as the 3.4.2 StateET for R Eclipse plugin

 ...In the new Eclipse 4.5 Mars release

Saturday, June 06, 2015

2015-06-06 Saturday - Network Bandwidth Monitoring Tools

First let me preface this posting by saying, I am not a hardware guy - nor do I specialize in network configuration.  You should definitely do your own homework...and consult the appropriate experts with your respective network service providers, and network hardware vendors.

I suspect most folks are not very familiar with the overhead of the TCP protocol - and would probably be astonished at the impact that even apparently minor differences can make in the limitations on actual bandwidth availability and utilization. A simple difference such as 10ms vs 50 ms in latency can make a huge difference.

Some day you may be in a situation where the monitoring metrics across your application tiers (such as CPU and memory utilization) are within well accepted ranges - but response times may seem quite excessive in a number of cases.  One possible area to explore is your available theoretical network bandwidth vs actual network bandwidth - and what percentage of the actual available is being utilized.  Here are a few possible areas to examine:
  • Is there an inefficient application in your data center that is behaving in an excessively chatty way (e.g. sending thousands of requests to complete the display of a single page)?
  • How is the setting for tcp window scaling option configured?  
  • What is the profile of transactions across your network? Is there something that is using an excessive % of your available network bandwidth - that might be a candidate for refactoring - or somehow isolating from the web application tier?
  • What is the latency between your application tiers / servers / external third party services?
  • Are all of the devices in your network using a consistently sized Network Interface Controller adapter that supports the target maximum theoretical speed of your fastest NIC device?
  • Are your customer facing web applications on the same network segment as your heavy-lifting back-end systems?
  • Will your customer facing web applications benefit by partitioning some lower-priority / high data volume / large data packet consuming applications into one or more separate network segments?
  • If you upgraded your network to use 10 Gigabit Ethernet, did you also upgarde the cabling from cat 5 to cat 6 (or at least cat 5e) - to mitigate interference - which can impact packet loss.?
  • Is your network cabling sufficiently separated by some distance from your power cabling - to mitigate interference - which can impact packet loss?
I'm doing some research today, looking at various possible tools to help monitor network performance and bandwidth utilization for applications running in a data center, or across a corporate WAN.

This posting is a placeholder for some of my background reading and research...

Some basic background material...

An excellent list curated by staff at Stanford University...

Examples of some tools to provide basic types of operational monitoring capabilities you should have in place...

Some help with basic calculations...

Some links to possible tools and vendor specific resources...

Tuesday, May 12, 2015

2015-05-12 Tuesday - Dream Teams

Trust, Innovation, and Ability are three of the most critical qualities in my line of work that I value the most in people. When thinking of who I would call on to tackle a major new project, there is a short list of names I mentally review from time-to-time as the core of my primary dream team. Those folks are already in my LinkedIn network - and it seems I may have need to call on them soon...