Monday, December 14, 2015

2015-12-14 Monday - Database Version Control & DevOps Deployment Processes

I'm starting some research to understand the current landscape of Best Practices around Database Version Control - as it relates to DevOps Deployment Processes.

A few years ago, on a previous client engagement, LiquidBase was heavily used - but it was not without pain & shortcomings. I haven't looked at LiquidBase recently - but plan to write a deep-dive evaluation article in January 2016.

This posting is a placeholder for information that I find - that may be of interest to others.

This recent article series came to my attention in a DZone email newsletter...

Sunday, December 13, 2015

2015-12-13 Sunday - Apache Cassandra

After reading the latest Apache Software Foundation weekly news round-up (*) - I happened to browse through the list of incubating projects - and was intrigued by Apache TinkerPop (note: Ketrina Yim created the excellent graphic characters to illustrate TinkerPop)

...which led me to the Titan Distributed Graph Database...(which was acquired by DataStax this year)
(note: Dr. Marko A. Rodriquez, Director of Engineering at DataStax, was the co-founder of TinkerPop, and founder of Aurelius LLC and the Titan Distributed Graph Database)

...which eventually led me to the excellent course material prepared by Kiyu Gabriel for the DataStax DS201: Cassandra Core Concepts

This DataStax blog post is illustrative of the concepts and power of TinkerPop

* I also noted the following, of interest to my expanding exploration of Big Data tooling, to add to the stack of things to research later: Apache Kylin™ –Open Source petabyte-scale Big Data distributed analytics engine. - The Apache Software Foundation Announces Apache Kylin as a Top-Level Project

Sunday, November 29, 2015

Friday, November 27, 2015

2015-11-27 Friday - Book: Universal Methods of Design

 Picked-up an excellent book this afternoon at the local Bellevue library: Universal Methods of Design, 100 Ways to Research Complex Problems, Develop Innovative Ideas, and Design Effective Solutions - by Bella Martin and Bruce Hanington

2015-11-27 Friday - Jason Baldridge, PhD, Scala tutorials

Today I'm spending some time exploring this...

Jason Baldridge, Associate Professor in the Department of Linguistics at the University of Texas at Austin.,
F11 Introduction to Computational Linguistics (with Scala)

2015-11-27 Friday - Go Conferences

This is a placeholder posting for Go Language Conferences...

Next year, The European Go Conference will be Oct 16th in Paris, at the Théâtre de Paris, 15, Rue Blanche

(was held in July in 2015 - in Denver, Colorado)

Sunday, November 22, 2015

2015-11-22 Sunday - Elasticsearch 2.0.0 GAA released on Oct 28th

There are some notable new features...

2015-11-22 Sunday - CentOS 7 Setup

I'm experimenting with setting-up a minimal CentOS 7 environment (within Oracle VirtualBox) on a Windows 7 laptop...this posting is a placeholder of information that others may find useful.

  • 'Sharpen the saw' with regards to my hands-on skills to configure and maintain a Linux virtual machine.
  • Create a pre-configured, reusable image, that I can use for future client engagements - and in preparation for some new product development I have planned for 2016.
  • As a foundation master/controller for experimenting with Ansible and Docker.

1) Install VirtualBox (I'm using the current 5.0.10 release)

2) Download CentOS (I'm using version 7)

3) Create Your CentOS virtual machine
  • For this experiment, I chose: CentOS-7-x86_64-Minimal-1503-01.iso

4) Update your packages
  • >yum update

5) Modify Network Interface Config Files
  • Locate the appropriate files in the /etc/sysconfig/network-scripts/ directory
    • For my particular installation environment, this happened to be ifcfg-enp0s03
    • Modify the following entries in the file to have these values:
      • ONBOOT=yes
      • BOOTPROTO=dhcp
    • Restart network service
      • >sudo systemctl restart network
    • Verify that you have network access
      • >ping -c 5

6) Configure NTP Synchronization
  • Check your current system time configuration
    • >sudo timedatectl
  • Set your desired timezone, for example:
    • >sudo timedatectl set-timezone America/Los_Angeles
  • Install ntp
    • >sudo yum install ntp
  • Start and Enable the ntp service
    • >sudo systemctl start ntpd
    • >sudo systemctl enable ntpd

7) Install some networking tools...
  • >sudo yum install net-tools
  • >sudo yum install nmap

8) Enable Outside Access (e.g. from the Windows 7 machine that is running VirtualBox)

9) Ansible install
  • {TODO}

10) Java install

11) Go install
  • >sudo yum install golang
    • currently appears to install v1.4

12) Python install
  • {TODO}

13) Git install
  • >sudo yum install git.x86_64

14) Connecting to a CentOS virtual machine with Eclipse

15) Install Apache Web Server (http)
  • >sudo yum install htttpd
  • >sudo service start httpd 
  • >sudo chkconfig httpd on

16) Enable Outside Access to Apache Web Server
  • Configure firewall
    • >sudo firewall-cmd --permanent --zone=public --add-service=http
    • >sudo firewall-cmd --permanent --zone=public --add-service=https
    • >sudo firewall-cmd --reload
  • Configure iptables 
    • >sudo iptables -I INPUT -i eth1 -p tcp -m tcp --dport 80 -j ACCEPT
    • To persist the changes, go to /etc/sysconfig/ - and add the following line to iptables
      • "-A INPUT -i eth1 -p tcp -m tcp --dport 80 -j ACCEPT"
    • >sudo service iptables restart
  •  >sudo service httpd restart
The following resource links were helpful to me, and may also be of interest to others:
You may also want to consider ordering a digital copy of this book:

Wednesday, November 18, 2015

2015-11-18 Wednesday - QConSF 2015 Trip Report

My Takeaway, as cited on the QConSF 2015 site:
"The confluence of diverse backgrounds, technologies, and business segments represented by a global breadth of attendees - merges into a synthesis of pure enthusiasm that provides one-and- all an opportunity to pause in our day-to-day professional challenges, and lift our heads up for a moment and embrace the realization that there are people doing extraordinary things in our industry - and as their knowledge is willingly shared - new plateaus of achievement are now within our own grasp."
Tonight I returned from my 4th - and have much to share.

The venue at the Hyatt Regency was excellent

I met some very interesting people on this trip - from many different countries - including: Italy, Australia, Belarus, Romania, India, England, Russia, Lebanon, and China.

I arrived on Sunday - and was able to meet up with three fellow attendees for drinks and dinner at The Chaya - an excellent Asian/Japanese Fusion style restaurant.

I will touch on just some of the more noteworthy sessions I attended this give you a flavor of the content that might appeal to someone likewise interested in the distributed architecture problem space.

Bill Buxton (Principal Researcher at Microsoft) gave an interesting talk on Monday morning's keynote: Avoiding The Big Crash.

I intended to attend as many of the sessions in the Architectures You've Always Wondered About track on Monday - but found myself often stepping out so as not to miss some very interesting sessions I note below.

My favorite session of the entire conference was given by Zhuoran Zhuang and Zhao Xu, who gave a phenomenally detailed presentation entitled Alibaba Mobile Infrastructure at "China Scale"

I also  enjoyed the talk given by Erran Berger (Senior Director of Engineering at LinkedIn) LinkedIn's Active/Active Evolution

My keen interest in the Go Language sessions led me to frequently drop in on in the Modern Languages in Practice track

For example:
  • Rick Hudson (Engineer at Google) did a session on Go GC: Prioritizing Low Latency and Simplicity. Surprisingly there is only one 'knob'  to turn with regards to Go GC. Rick hinted at some changes that are coming in Go 1.6 - which I need to research more.

On Tuesday, I spent time in the Containers in Practice track, and found these three sessions to be very notable:

I also managed to include a talk given by  Fangjin Yang's (co-founder of @Imply) Architecting Distributed Databases for Failure - in the Architecting for Failure track

On Wednesday, I particularly enjoyed the talk given by Jason Brown (an Apache Cassandra Core Committer) Hell is...other nodes - in the Taming Distributed Architectures track.

Making the time to step away from day-to-day responsibilities is never easy. But the investment I have made in making the commitment to  attend QConSF has paid off in so many ways: Ideas, insights, specific case studies that can be cited, bread crumbs of knowledge that give me the beginning of the trail to discovery, industry connections...and yes, not to be forgotten or marginalized...a renewed sense of enthusiasm for my profession.

Some additional links I picked-up from some twitter activity that may be of interest to others:

Brendan Gregg's Linux Performance page

So We Hear You Like Papers talk by Ines Sombra - Distributed Systems Engineer at @Fastly, and Caitie McCaffrey - Tech Lead of Observability at Twitter

Saturday, November 14, 2015

2015-11-14 Saturday - Reading Source Code: Google's TensorFlow

TensorFlow is an Open Source Software Library for Machine Intelligence

I'm spending some time this weekend reading through the source code for Google's TensorFlow
"TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. TensorFlow was originally developed by researchers and engineers working on the Google Brain Team within Google's Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research, but the system is general enough to be applicable in a wide variety of other domains as well."

Some recent news stories about TensorFlow

2015-11-14 Saturday - MariaDB 10.1.8

If you are running MySQL - here's a compelling reason to switch to MariaDB 10.1.8 - for table, tablespace, temporary files, temporary tables, and binary logs encryption for Data At Rest - "with an overhead of roughly 3-5%."

MariaDB 10.1 can do 1 million queries per second

MariaDB 10.1 and MySQL 5.7 performance on commodity hardware

Monday, November 09, 2015

2015-11-09 Monday - One Week until QCon San Francisco

In one week I will be in San Francisco, attending QCon (the international software development conference - that has a strong architecture theme)

2015-11-09 Monday - Saltstack

I've recently had a team experimenting with Ansible - but have also been curious to better understand the possible benefits of Saltstack. This posting is a placeholder for notes and links to resources that may be of interest to others.

Sunday, November 01, 2015

2015-11-01 Sunday - CDN Best Practices

This posting is a placeholder for resources I find interesting - related to CDN Best Practices - that may be useful to to others...

I'm currently evaluating Amazon's AWS CloudFront as a possible CDN solution

S3 limitation on creating directory structures for serving CDN content
 see Phil P's last response - Apr 24, 2013 11:56pm
- this invalidated an assumption I had of using a CDN release directory structure with S3 - but seems to state that it could be done with an EC2 instance as a custom origin.

I also noted this entry, re: Invalidation of paths is not (?) supported with wildcards...

A question I've raised on the AWS CloudFront forums 
"Versioning and cache-control You must consider how you will update static content and deploy new versions. The CDN does not currently provide a mechanism for flushing content so that new versions are available. This is a similar challenge to managing client side caching, such as in a web browser."
"Testing It can be difficult to perform local testing of your CDN settings when developing and testing an application locally or in staging."
"You cannot set up a CDN endpoint for an application deployed in Azure staging, or in the local Azure emulator in Visual Studio. This will affect unit testing, functional testing, and final pre-deployment testing. You must allow for this by implementing an alternative mechanism. For example, you could pre-deploy the content to the CDN using a temporary custom application or utility, and perform testing during the period it is cached. Alternatively, use compile directives or global constants to control where the application loads the resources from. For example, when running in debug mode it could load resources such as client-side script bundles and other content from a local folder, and use the CDN when running in release mode."

A lists of CDN Vendors...

Heroku's Fastly

Marrying CDNs with front-end optimization

Saturday, October 31, 2015

2015-10-31 Saturday - My Current Go Reading List

...and somewhat related...

Sunday, October 25, 2015

2015-10-25 Sunday - Docker Resouces

I'm experimenting with Docker Toolbox 1.8.3

2015-10-25 Sunday - New Books Ordered

The Docker Book: Containerization is the new virtualization
$10.94 for the Kindle Edition

Docker: Up & Running
$17.73 for the Kindle Edition

The Go Programming Language (Addison-Wesley Professional Computing Series) 1st Edition
$32.84 for the hardback (pre-ordered, publication date Nov 8th)

You Don't Know JS: Async & Performance
$13.68 for the Kindle Edition

Modern PHP: New Features and Good Practices
$15.87 for the Kindle Edition

Introductory Graph Theory (Dover Books on Mathematics) 
$8.94 for the Kindle Edition

Saturday, October 03, 2015

2015-10-03 Saturday - Hidden Structure of System Architecture

An interesting Harvard Business School whitepaper by Carliss Baldwin, Alan MacCormack, and John Rusnak
Hidden Structure: Using Network Methods to Map System Architecture

2015-10-03 Saturday - Groups, Roles, Permissions

Today I'm spending some time researching various solutions (Commercial, SaaS, Open Source)   that may offer some ideas for implementing a custom framework for the concept of Identity Management, Access Control, Groups, Roles, and Permissions. This posting is a placeholder for resources that I find - that may be of interest/use to others.

  • Identity Management Objects (IDM)
    • Realm
    • Group
    • Roles
    • Permission
    • User
    • Agent

- 131 packages tagged with 'permissions'
- 339 packages tagged with 'authorization'
- 82 packages tagged with 'rbac'

Commercial Solutions - Access/Identity Management...............................................................

SaaS - Identity API...............................................................


goRBAC provides a lightweight role-based access control (RBAC) implementation in Golang


A Laravel 5 Boilerplate project

A framework agnostic authentication & authorization system
- [DEPRECATED] framework agnostic authentication & authorization system

Roles & Permissions for the Laravel 5 Framework

Role-based permissions for Laravel 5
- an interesting, rudimentary capability

Authorization PHP package for Laravel 4, 5.0, and 5.1
- a port of CanCan

package for handling roles and permissions in Laravel 5 (5.1 and 5.0)
- supports inheritance of permissions...
- can attach/detach permissions to a role or directly to a specific user
- some interesting ideas/features...

Laravel 5 roles and permissions package with support for Laravel's authentication contract

Simple permission checker for Nette Framework

Role-based access control module to provide additional features on top of Zend\Permissions\Rbac

User Access Control List for Zend Framework 2

- very good overview of the symphony features...

Provides user management for your Symfony2 Project - Compatible with Doctrine ORM and ODM and Propel

Authentication, Authorization, and API Proxy Service
(An EVE online authentication and mediated API access application.)


Apache Syncope - an Open Source system for managing digital identities in enterprise

authorization module for the Play framework

- see Identity Model

Spring Security

pack4j - authenticate with Facebook, Twittter, or Google


Roles and permissions system for Nodejs


Pyramid Web Framework

list of solutions...

Authorization Gem for Ruby on Rails
simple authorization solution for Rails which is decoupled from user roles...all permissions are stored in a single location

Authority helps you authorize actions in your Rails app

Dashboard that allows you to manage roles and permissions for Auth0 users

simple authorization solution for Rails which is decoupled from user roles...all permissions are stored in a single location


Sunday, September 20, 2015

2015-09-20 Sunday - MySQL Date Handling

In preparation for some API design work I'm doing - to include exposing an API internally, to B2B partners, and potentially to a Developer Community - as well as the challenges of supporting a global deployment of a solution - I'm doing some research today on MySQL date (and timezone) handling...these are the resources I've found relevant...and which may be of interest/use to others...
  • "When the server starts, it attempts to determine the time zone of the host machine and uses it to set the system_time_zone system variable. The value does not change thereafter."
  • "You can set the system time zone for MySQL Server at startup with the --timezone=timezone_name option to mysqld_safe. You can also set it by setting the TZ environment variable before you start mysqld."
  • "The initial global server time zone value can be specified explicitly at startup with the --default-time-zone=timezone option on the command line, or you can use the following line in an option file:"
    • default-time-zone='timezone' 
  • "Per-connection time zones. Each client that connects has its own time zone setting, given by the session time_zone variable. Initially, the session variable takes its value from the global time_zone variable, but the client can change its own time zone with this statement:"
    • mysql> SET time_zone = timezone;
  • "The current values of the global and client-specific time zones can be retrieved like this:"
    • mysql> SELECT @@global.time_zone, @@session.time_zone;

*** READ THIS ***

*** AND THIS ***

  • "You shouldn't want to store time zone info in database. Store all date/time data as UTC and always make time zone offset adjustment on the application layer. –  marekful Nov 7 '13 at 18:00" 

2015-09-20 Sunday - 2015 Seattle Code Camp - Diagnosing The Patient

The deck for my presentation at the 2015 Seattle Code Camp is posted to slideshare

2015-09-20 Sunday - Go RESTful Web Services

I'm experimenting with Go 1.5 these days...evaluating it for possible adoption in a major system refactoring / renewal effort.

I'm doing some reading today on resources to leverage in developing RESTful web services in Go.  This posting is a placeholder for information that I find - and which may be of interest / use to others. 

Go: Simple, Easy, Fast – Building a Go (golang) REST Service with Gorilla

Creating a REST API using Go (GoLang) and JAS (JSON API Server)
"A simple and powerful REST API framework for Go"

Creating a REST API using Go (GoLang) and Goweb
"A lightweight RESTful web framework for Go"

Saturday, September 19, 2015

Saturday, September 05, 2015

2015-09-05 Saturday - ITV Code Generator, v0.8 beta

I spent some time today extending the data model of the ITV_Core product - now up to 112 tables.

I then ran the ITV_Code_Generator tool against that data model, and obtained the following results:

ITV Code Generator, Version, v0.8 beta
copyright 2001-2015, International Technology Ventures, Inc.

Total Tables Processed: 112
Total Files Created:        7,393
Total Lines of Code:       433,372
Run Time:                      Hours: 0  Minutes: 1  Seconds: 56  Milliseconds: 144

The tool is compiled with Java 8, within Eclipse Mars - leveraging a PostgreSQL 9.4 database

I currently have 34 target output language modules identified in scope for the tool.

The goal of the tool is to provide my consulting practice with a proprietary tool and intellectual property assets to accelerate development of custom solutions during client engagements - by eliminating a significant portion of the drudgery of foundational development work - so that more time can be spent on the creative and innovative aspects of design and development efforts.

Sunday, August 30, 2015

2015--08-30 Sunday - ES6 Notes

This posting is a placeholder for articles and documentation I'm reviewing to deepen my understanding of ES6

Saturday, August 29, 2015

2015-08-29 Saturday - JSON Schema

This posting is a placeholder for some research I'm doing related to JSON Schema features/capabilities.

"JSON Schema is an Internet Draft, currently version 4"
"Although JSON is intended solely as a data serialization format, its design as a non-strict subset of the JavaScript scripting language poses several security concerns. These concerns center on the use of a JavaScript interpreter to execute JSON text dynamically as embedded JavaScript. This exposes a program to errant or malicious scripts. This is a serious issue when dealing with data retrieved from the Internet. This easy and popular but risky technique exploits JSON's compatibility with the JavaScript eval() function"!forum/json-schema
"JSON schema validator, which is designed to be fast and simple to use. The latest IETF published draft is v4, this library is mostly v4 compatible."
JSON Schema Lint
"The schema is still under development and the progress can be tracked by comparing the versions known as “drafts”. Currently, the schema is in the 4th version."
"Much like the incoming request, Hyper-schema allows us to specify a schema for the outgoing response as well with the targetSchema keyword."
"Generate API documentation with Prmd." 

2015-08-29 Saturday - Suggestions for Developer Continued Self-Education

This week I prepared a list of suggestions for an engineering group - on some free online resources that might be of interest to those who wish to invest some time in their own continuing self-education. Perhaps others might find these of interest as well:

The AWS tutorials are free – and the examples and documentation is quite extensive

The Seattle AWS Architects & Engineers meet-up is organized by a friend of mine, Teri Radichel – and most sessions are __very__ informative

Thomas Erl’s web site provides a very good (and free) overview of much of his content

Stanford’s Video Lecture series on Startup Engineering has some very good free content

This Forbes article from last year has quite a few suggested links to free online courses that are specific to Cloud Computing

The MIT  Open Course Ware series (offered through the Sloan School of Management) has a good course on ‘Management of Services: Concepts, Design, and Delivery’

One of the best sources of free technical information (in particular, for tracking leading edge developments) is slideshare

In particular, I am almost always __very_- impressed by the decks prepared by the folks at Netflix and Etsy

One of the best values for a professional programmer is a personal professional subscription to Safari ($39/month)

There is a tremendous amount of valuable knowledge and expertise in the IBM Cloud Computing Redbooks

I also personally find the following YouTube channels to be very informative:

2015-08-29 Saturday - Language Performance Benchmarks

I've spent some time this week reviewing some language performance benchmarks - notably looking for comparisons between Java, Go, Rust, Python, Groovy, and PHP. This page is a placeholder for links to resources that I find of interest to this topic:

Monday, August 17, 2015

2015-08-17 Monday - Interviewing Tool: Candidate Self-Scoring Scale

When interviewing candidates, I like to go through a relevant list of technologies for the given role/position and have the candidate provide a self assessment score of their level of experience / knowledge / depth of technical expertise.

Over the years, I developed the following scale to help facilitate those conversations. The intent is to seek to minimize the variability in the subjective nature of having someone just assign their own weight/interpretation to a 1-10 scale.

10 - Recognized as an industry expert on the given technology, outside of their company.
  • Examples: A member of the development team of the given technology, authored books/articles, frequently invited speaker on the given technology at conferences, etc.

 9 - Recognized as one of the key experts on a given technology, within their company.

 8  - Highly Proficient
  • You are able to quickly build solutions with the given technology. 
  • You may occasionally refer back to documentation, but in general - you are quite familiar with the given technology.
  • You have built, deployed, and maintained multiple solutions with the given technology

 7 - Practical Knowledge
  • Examples: You've used the given technology on a project somewhere and have actually deployed something to production and maintained it

 6 - Played with it, have at least built a toy app, but may not have been deployed to production.

 5 - Downloaded it, look at the source code

 4 - Took a class, attended a conference session, completed brief online tutorials

 3 - Read a book

 2 - Read about it somewhere
  • Examples: Industry journal, an article, someone's blog

 1 - Aware that it exists

Copyright 2015 by Kelvin D. Meeks under the Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License