2016-12-03 Saturday - Vulkan 3D Programming

There are some interesting practical applications of 3D visualization technologies to many commercial businesses - if you have heard of OpenGL  - you might want to keep an eye on Vulkan as it continues to mature. Note the CPU and Memory utilization and lag differences in this side-by-side comparison

• https://www.youtube.com/watch?v=P_I8an8jXuM&feature=youtu.be

https://www.khronos.org/ 

• https://www.khronos.org/assets/uploads/developers/library/overview/vulkan-overview.pdf

• https://www.khronos.org/registry/vulkan/specs/1.0/xhtml/vkspec.html

https://www.toptal.com/api-developers/a-brief-overview-of-vulkan-api

https://vulkan-tutorial.com/

 
https://github.com/GPUOpen-Tools/CodeXL

2016-11-05 Saturday - Enterprise Search

I'm doing a bit of a refresh on the latest Enterprise Search options that are out there - and will use this blog post as my placeholder for my notes. Hopefully this will become a useful resource for others too.

Initially, I intend to examine the following (but may add others as time permits):

• Apache Solr (Latest v6.0 April 2016)
• http://lucene.apache.org/solr/ 
• http://lucene.apache.org/solr/resources.html#documentation
• http://lucene.apache.org/solr/quickstart.html
• https://issues.apache.org/jira/browse/SOLR/
• https://wiki.apache.org/solr/
• https://wiki.apache.org/solr/Join
• https://wiki.apache.org/solr/Solrj 
• https://wiki.apache.org/solr/UpdateXmlMessages
• https://wiki.apache.org/solr/DataImportHandler
• Use full-import command to do a full import from the database and add to Solr index
• Use delta-import command to do a delta import (get new inserts/updates) and add to Solr index  
• https://wiki.apache.org/solr/DataImportHandlerFaq

• https://cwiki.apache.org/confluence/display/solr/About+This+Guide
• https://cwiki.apache.org/confluence/display/solr/Major+Changes+from+Solr+5+to+Solr+6 
• https://cwiki.apache.org/confluence/display/solr/Introduction+to+Solr+Indexing
• https://cwiki.apache.org/confluence/display/solr/Indexing+and+Basic+Data+Operations 
• http://lucene.apache.org/solr/news.html#8-april-2016-apache-solr-600-available
• http://lucene.apache.org/solr/6_0_0/changes/Changes.html 
• https://www.openhub.net/p/solr
• https://en.wikipedia.org/wiki/Apache_Solr 
•  active community...plugins

• Elasticsearch (Latest: v5.0.0 Oct 26, 2016)
• https://www.elastic.co/products/elasticsearch
• https://www.elastic.co/guide/index.html  
• https://www.elastic.co/guide/en/elasticsearch/guide/master/index.html
• https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
• https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html
• https://www.elastic.co/guide/en/elasticsearch/reference/current/general-recommendations.html 
• https://www.openhub.net/p/elasticsearch
• https://en.wikipedia.org/wiki/Elasticsearch
• "supports multitenancy"
• "rebalancing and routing done automatically"
• "percolating"
• "gateway...recovery of index after server crash" 

• Amazon CloudSearch
• https://aws.amazon.com/cloudsearch/
• https://aws.amazon.com/documentation/cloudsearch/ 
• https://aws.amazon.com/cloudsearch/pricing/ 

• Rocana
• https://www.rocana.com/ 
• https://www.rocana.com/rocana-ops/limitless-search

General Search Articles

• http://opensourceconnections.com/blog/2016/06/29/top-mistakes-made-in-search/
• https://findwise.com/Enterprise-Search-Findability-Report-2016
• http://sunset.usc.edu/classes/cs572_2015b/
• http://sunset.usc.edu/classes/cs572_2015b/CS572_HW_SOLR_WEAPONS.pdf

 Apache Solr Related Articles

• http://www.enterprisesearchblog.com/solr/
• http://www.solrtutorial.com/
• http://www.solrtutorial.com/basic-solr-concepts.html
• http://blog.kevindoran.co/solr-tutorial/
• https://www.quora.com/How-can-I-index-multiple-tables-in-Apache-Solr
• https://wiki.apache.org/solr/Join 
• http://www.toadworld.com/platforms/oracle/w/wiki/11121.collecting-indexing-and-searching-mysql-database-table-data-in-apache-solr
• https://gist.github.com/maxivak/3e3ee1fca32f3949f052
• After successfully adding a collection to Solr you can select it and run dataimport commands:
  
• full-import - use URL: http://localhost:8983/solr/myproducts/dataimport?command=full-import
• delta-import - use URL: http://localhost:8983/solr/myproducts/dataimport?command=delta-import
• http://blog.comperiosearch.com/blog/2014/08/28/indexing-database-using-solr/
• See: "Here we have carried out a full index of our database using the full-import command parameter. To only retrieve changes since the last import, we would use delta-import instead."
• See: "Finally, to schedule reindexing you can use a simple cronjob. This one, for example, will run everyday at 23:00 and retrieve all changes since the previous indexing operation"
• http://blog.comperiosearch.com/blog/2015/04/14/solr-indexing-index-sql-databases-made-easier-part-2/
• See: "Facets, also known as filters or navigators, allow a search user to refine and drill down through search results. Before we get started with them, we need to update our data import configuration. Replace the contents of our existing db-data-config.xml with" 

• http://solr.pl/en/2010/10/11/data-import-handler-%E2%80%93-how-to-import-data-from-sql-databases-part-1/
• http://solr.pl/en/2010/11/01/data-import-handler-%E2%80%93-how-to-import-data-from-sql-databases-part-2/
• http://solr.pl/en/2010/11/22/data-import-handler-%E2%80%93-how-to-import-data-from-sql-databases-part-3/  
• http://stackoverflow.com/questions/30837300/how-could-i-properly-configure-solr-to-index-my-oracle-database
• https://jasonrogena.github.io/2014/10/03/solr-with-mysql.html
• https://community.hortonworks.com/articles/52183/solr-indexing-the-database-tables.html 
• https://www.gadgetdaily.xyz/ssh-tunnelling-on-insecure-networks/
• https://coderanch.com/t/647210/User-Solr
• https://rahuldausa.wordpress.com/2014/05/16/real-time-search-on-40-billion-records-month-with-solr/
• https://www.lullabot.com/articles/finding-related-content-faster-with-apache-solr
• Indexing MySQL database with Apache Solr 
• https://www.youtube.com/watch?v=BuSCfi7AsLU 

Apache Lucene Related Articles

• http://lucene.apache.org/core/6_2_1/changes/Changes.html
• https://www.tutorialspoint.com/lucene/index.htm
• http://www.lucenetutorial.com/techniques/indexing-databases.html
• https://dzone.com/articles/lucene-database-oracle-five 

Elasticsearch Related Articles

• https://www.tutorialspoint.com/elasticsearch/index.htm
• https://www.tutorialspoint.com/elasticsearch/elasticsearch_basic_concepts.htm
• http://www.elasticsearchtutorial.com/ 
• http://www.elasticsearchtutorial.com/basic-elasticsearch-concepts.html
• http://opensourceconnections.com/blog/2016/09/09/better-recsys-elasticsearch/
• http://stackoverflow.com/questions/31755642/how-can-i-multiply-the-score-of-two-queries-together-in-elasticsearch 
• http://joelabrahamsson.com/elasticsearch-101/
• https://aphyr.com/posts/323-call-me-maybe-elasticsearch-1-5-0
• "You can lose documents if..."
• https://www.elastic.co/guide/en/elasticsearch/resiliency/current/index.html 

 

Comparison Articles

• http://cloudacademy.com/blog/elasticsearch-vs-cloudsearch/
• http://solr-vs-elasticsearch.com/ 
• http://logz.io/blog/solr-vs-elasticsearch/
• Solr: "Very well-documented"
• Solr: "Database integration"
• ES: "Lacks in documentation "
• ES: "Intuitive"
• ES: "If you need distributed indexing, then you need to choose Elasticsearch. Elasticsearch is the better option for cloud and distributed environments that need good scalability and performance"
•  Comparing SolR/Lucene/ElasticSearch vs. Cloud based Search Providers
• https://www.youtube.com/watch?v=JiMxmkMGrbU
• https://thinkbiganalytics.com/solr-vs-elastic-search/
• http://opensourceconnections.com/blog/2016/01/22/solr-vs-elasticsearch-relevance-part-two/ 
• [ES queries are verbose, but more readable... ]
• http://opensourceconnections.com/blog/2015/12/15/solr-vs-elasticsearch-relevance-part-one/
• "ES wins on matching" 
• https://www.datanami.com/2015/01/22/solr-elasticsearch-question/
• https://sematext.com/blog/2015/05/27/side-by-side-with-elasticsearch-and-solr-performance-and-scalability/
• "see more Elasticsearch clusters with issues..."
• "despite what you just read about Elasticsearch, we rely on it in our organization for several different products"
• "Elasticsearch is often for filtering and grouping – the analytical query workload – and not necessarily text search"
• "Elasticsearch is a better choice for applications that need to do not just text search, but also complex search-time aggregations"
• http://harish11g.blogspot.in/2015/07/amazon-cloudsearch-vs-elasticsearch-vs-Apache-Solr-comparison-report.html  
• http://www.flax.co.uk/blog/2015/12/02/elasticsearch-vs-solr-performance-round-2/ 
• https://sematext.com/blog/2015/01/30/solr-elasticsearch-comparison/
• http://db-engines.com/en/blog_post//55
• https://www.quora.com/What-is-the-difference-between-Apache-Solr-and-Lucene
• http://www.hcltech.com/blogs/elasticsearch-vs-solr
• Elastic: 
• "index documents of different index structure in a single Index"
• "automatic shard rebalancing"
• "supports complex nested types"
• Percolation Queries (i.e. for alert generation)
• Solr: 
• "have to simulate that inside your application or develop a custom search component."
• "complex distributed group"

• http://www.infoworld.com/article/3119213/analytics/beyond-solr-scale-search-across-years-of-event-data.html

Search Related Books

• https://www.manning.com/books/relevant-search
• https://www.amazon.com/Lucene-Action-Second-Covers-Apache/dp/1933988177 
• https://www.amazon.com/Solr-Action-Trey-Grainger/dp/1617291021/
• https://www.amazon.com/Elasticsearch-Action-Radu-Gheorghe/dp/1617291625/ 
• https://www.amazon.com/Elasticsearch-Definitive-Guide-Clinton-Gormley/dp/1449358543/
• https://www.amazon.com/ElasticSearch-Cookbook-Second-Alberto-Paro/dp/1783554835/ 
• https://www.amazon.com/Mastering-Elasticsearch-Second-Rafal-Kuc/dp/1783553790/
• https://www.amazon.com/Learning-ELK-Stack-Saurabh-Chhajed/dp/1785887157/

2016-10-28 Friday - Weekend Reading List

http://www.businessinsider.com/mary-meeker-2016-state-of-the-web-presentation-2016-6?op=0#/#-1

http://www.kdnuggets.com/2016/08/10-algorithms-machine-learning-engineers.html

https://medium.baqend.com/building-a-shop-with-sub-second-page-loads-lessons-learned-4bb1be3ed07#.1113nb18z

http://seriot.ch/parsing_json.html#41

https://github.com/mjavascript/practical-es6

https://www.elastic.co/blog/elastic-stack-5-0-0-released 

https://aws.amazon.com/ec2/instance-types/
 
https://aws.amazon.com/ec2/pricing/reserved-instances/pricing/

2016-10-21 Friday - AI Topics, Weekend Reading

• https://www.infoq.com/news/2016/10/CommAI-Train-Test-Facebook
• https://github.com/facebookresearch/CommAI-env
• https://github.com/facebook/MazeBase

Paper: "A Roadmap towards Machine Intelligence"

• https://arxiv.org/abs/1511.08130

• https://arxiv.org/pdf/1511.08130v2.pdf

2016-10-14 Friday - The 2016 Top Programming Languages

The 2016 Top Programming Languages

http://spectrum.ieee.org/computing/software/the-2016-top-programming-languages

2016-10-14 Friday - Is REST Holding Your API Performance Hostage?

Github's recent adoption of Facebook's GraphQL has me reconsidering ways to approach API design for my own clients...

• https://www.infoq.com/news/2016/10/graphQL-GitHub
• http://githubengineering.com/the-github-graphql-api/
• http://graphql.org/

There is no silver bullet (or free lunch) to balancing the concerns of performance, complexity, and minimizing leaking-the-abstraction - but there are very powerful and compelling advantages to the GraphQL approach that are worthy of your consideration in your approach to building APIs that minimize long-term maintenance, network latency, and providing greater flexibility to your service consumers

2016-08-27 Saturday - Apache Tomcat / TomEE start-up takes forever

I've just installed Apache Tomcat 9.0.0.M9 and Apache TomEE 7.0.1 today - and encountered a curious issue with the start-up taking forever (for either one) - well, at least longer than a minute...

Some folks report much longer "hang" times...

• http://stackoverflow.com/questions/29355258/tomcat-taking-too-long-to-initialize

I've also installed Apache Tomcat 9.0.0.M9 on a different laptop (Windows 7) - and the start-up is normal (~1+ second)

On my primary development laptop (MSI, Windows 10, Java 1.8.0_102-b14) - the start-up takes ~1 minute+

I happened across this article:

• https://www.linkedin.com/pulse/understanding-tomcat-nio-connector-how-configure-tiziano-di-gennaro

which suggested changing the protocol from "HTTP/1.1" to "org.apache.coyote.http11.Http11NioProtocol"

Problem solved - for now - until I have time to dig into this more and determine root cause.

Some folks have suggested this problem is due to a lack of entropy in the OS...

• http://wiki.apache.org/tomcat/HowTo/FasterStartUp#Entropy_Source
• "Djava.security.egd=file:/dev/./urandom" 
• "replacing the blocking entropy source (/dev/random) with a non-blocking one actually reduces security because you are getting less-random data"
• http://security.stackexchange.com/questions/89/feeding-dev-random-entropy-pool
• http://www.labouisse.com/misc/2014/06/19/tomcat-startup-time-surprises 
• https://support.run.pivotal.io/entries/59869725-Java-Web-Applications-Slow-Startup-or-Failing
• http://stackoverflow.com/questions/20315022/java-slow-entropy-related-issue

• http://stackoverflow.com/questions/28201794/slow-startup-on-tomcat-7-0-57-because-of-securerandom

• http://stackoverflow.com/questions/137212/how-to-solve-performance-problem-with-java-securerandom/2325109#2325109
• "if you want a cryptographically strong random number in Java, you use SecureRandom. Unfortunately, SecureRandom can be very slow"

• https://www.digitalocean.com/community/questions/fresh-tomcat-takes-loong-time-to-start-up

Given the above, the recent Python 3.6 release included a change that caught my eye

• PEP 524 -- Make os.urandom() blocking on Linux 
• https://www.python.org/dev/peps/pep-0524/

 

2016-08-13 Saturday - Nvidia GTX Titan X

Nvidia unveils new GTX Titan X: 11 teraflops,
3584 CUDA cores, 12GB of GDDR5X memory, 480GB/s of memory bandwidth
http://arstechnica.com/gadgets/2016/07/gtx-titan-x-pascal-specs-price-release-date/

2016-08-10 Wednesday - MonkeyLearn.com Machine Learning Tutorials

In the most recent O'Reilly Media Data Newsletter - there is a link to an interesting Machine Learning tutorial (using the MonkeyLearn platform - which has a free tier of usage)
https://blog.monkeylearn.com/machine-learning-1m-hotel-reviews-finds-interesting-insights/


https://github.com/monkeylearn/hotel-review-analysis

2016-07-28 Thursday - Cloud Foundry Research

I'm putting in some effort to deepen my understanding of Cloud Foundry...and thought that others might find my 'background reading' list of some use...


https://www.cloudfoundry.org/

http://www.slideshare.net/EnikTrk/cf-summit-2016-review
see slide #6,

http://www.slideshare.net/geekclub888/multicloud-microservices-with-cloudfoundry
...a fairly good / detailed discussion...re: microservces on Cloud Foundry

http://www.slideshare.net/nicholsr/platform-as-a-service-cloudfoundry-and-ibm-bluemix
see slide #6, #16

http://www.slideshare.net/AnimeshSingh/ibm-bluemix-architecture-and-deep-dive-powered-by-cloudfoundry
http://www.slideshare.net/IanLynch1/bluemix-overview
http://www.slideshare.net/fgeorg/enterprise-paas-cloudnative-architecture-and-microservices



Video Presentations...

https://blog.pivotal.io/pivotal-cloud-foundry/features/deploying-microservice-architectures-with-pivotal-cloud-foundry

https://www.infoq.com/presentations/Cloud-Foundry-Tips

https://www.infoq.com/presentations/Cloud-Foundry-Architecture

https://www.infoq.com/presentations/Cloud-Foundry-Inside-the-Machine

https://www.infoq.com/interviews/kromhout-cloudfoundry


Cloud Foundry Youtube Videos...
https://www.youtube.com/channel/UC0ZYS0Y7b5oiVLvxGf4magw

Cloud Foundry Summit 2015 videos
https://www.youtube.com/playlist?list=PLhuMOCWn4P9g-UMN5nzDiw78zgf5rJ4gR

Cloud Foundry Summit 2016 videos
https://www.youtube.com/playlist?list=PLhuMOCWn4P9iqBDITU3bY6drETEwPMVpd


Pivotal Cloud Foundry reading...

https://pivotal.io/platform
http://docs.pivotal.io/

https://www.infoq.com/news/2015/11/pivotal-cloud-foundry-netflix

http://nordicapis.com/microservices-architecture-the-good-the-bad-and-what-you-could-be-doing-better/

2016-07-28 Thursday - Converged Infrastructure

I've been doing some research today on the topic of Converged Infrastructure, and thought these links might be of interest to others...

General Concept: Converged Infrastructure
http://www.slideshare.net/chaitanyag1/point-of-view-converged-infrastructure


VBlock

http://www.slideshare.net/Ciscodatacenter/vce-vblock-overview

http://www.slideshare.net/esteinmetz/vblock-infrastructure-platforms
see slides #10, #12

http://www.slideshare.net/EMCnederland/vce-vblock-systems
see slide #8

http://www.slideshare.net/esloof/vblock-infrastructure-packages-integrated-bestofbreed-packages-from-vmware-cisco-and-emc
see slides #5, #7, #8, #15,

http://www.slideshare.net/PuppetLabs/case-study-developing-a-vblock-systems-based-private-cloud-platform-with-puppet-and-vmware-vcloud-suite-puppetconf-2014

2016-07-12 Tuesday - Free Architecture Tool Resources

A colleague recently asked me for suggestions of possible free or Open Source tools for use in crafting the Architecture of a solution.  Here's the response I provided...

You might find this recent white paper of interest...

Journal of Object Technology, vol. 14, no. 1, 2015,
Evaluating the capabilities of Enterprise Architecture modeling tools for Visual Analysis
Department of Systems and Computing Engineering, Universidad de los Andes, Colombia
http://www.jot.fm/issues/issue_2015_01/article3.pdf

Open Source Diagramming / Architecture Tools...

Not sure about these...haven't spent any time exploring them yet...

• http://www.enterprise-architecture.org/

• http://www.enterprise-architecture.org/about/screenshots

• http://www.adoit-community.com/

• https://www.iteraplan.de/en/editionen/

I've found Graphviz to be useful for automating diagram generation from any available metadata sources...

• http://www.graphviz.org/

...but today, I would look at adopting D3.js - for some stunning diagramming capabilities...

• https://github.com/d3/d3/wiki/Gallery

SchemaSpy is a great tool for auto-generating HTML documentation of existing database schemas - and it leverages graphviz behind the scenes to auto-generate diagrams...

• http://sourceforge.net/projects/schemaspy/

this appears to be a more actively maintained type of database schema doc tool..

• http://sourceforge.net/projects/schemacrawler/

There are some useful plugins available within the Eclipse ecosystem - and the latest version of Eclipse (4.6, Neon) was just recently released

• http://www.eclipse.org

A recent Eclipse plugin that I've found useful:

• https://github.com/intltechventures/EclipseNotes/blob/master/README.md

Other Eclipse plugins, specific to modeling efforts:

• https://www.eclipse.org/modeling/mdt/
• http://projects.eclipse.org/projects/technology.epf
• https://eclipse.org/papyrus/
• https://marketplace.eclipse.org/content/uml-designer
• https://marketplace.eclipse.org/content/umlet-uml-tool-fast-uml-diagrams
• https://marketplace.eclipse.org/content/uml-designer

"Archi" for ArchiMate modeling...
http://www.archimatetool.com/

• https://marketplace.eclipse.org/content/archi-archimate-modelling-tool
• http://www.archimatetool.com/dev/eclipse-setup

Diagramming tooling...some are Open Source...some are free (or have limited-use restrictions) - some  have UML/Archimate/etc. support...

• https://docs.google.com/drawings

• https://www.openoffice.org/product/draw.html

• http://plantuml.com/

• https://www.gliffy.com/

• https://www.modelio.org/

• http://argouml.tigris.org/

• http://www.yworks.com/products/yed

• https://www.draw.io/

• https://www.gimp.org/downloads/

• https://www.lucidchart.com/users/registerLevel?tP=1&t4=A&t10=A

• http://creately.com/plans

2018-08-05 Sunday - Additions, specific to ArchiMate modeling...
re: https://www.linkedin.com/groups/50758/50758-6428603106120003584

https://www.yworks.com/products/yed

"yEd is a powerful desktop application that can be used to quickly and effectively generate high-quality diagrams.
Create diagrams manually, or import your external data for analysis. Our automatic layout algorithms arrange even large data sets with just the press of a button.
yEd is freely available and runs on all major platforms: Windows, Unix/Linux, and macOS."

https://github.com/colugo/yed-archimate

"This palette introduces archimate support to yEd"

https://sourceforge.net/projects/open-modeling/

Openmodeling has currently 5 modules available:
• Application portfolio management
• ArchiMate 3.0 architecture (including The Open Group Model Exchange)
• Demo conceptual modeling
• Efqm quality assessment
• Business and system modeling

https://about.draw.io/new-archimate-3-0-shapes/

• https://www.draw.io/?splash=0&libs=archimate3

http://plantuml.com/

• http://en.plantuml.com/archimate-diagram

https://www.eclipse.org/papyrus
https://marketplace.eclipse.org/content/archimate-papyrus

2016-07-05 Tuesday - Enterprise Architecture Heat Maps

Today I'm doing some background reading on generating Enterprise Architecture Heat Maps.

I've begun assembling my notes in this github markdown document:
https://github.com/intltechventures/Lab.Architecture/blob/master/HeatMaps/HeatMaps.md

2016-06-28 Tuesday - Eclipse Neon

The Eclipse IDE, 4.6 "Neon", was released on June 22nd...
https://www.eclipse.org/neon/noteworthy/

I've added a section to my notes for the plug-ins that I'm using with Neon
https://github.com/intltechventures/EclipseNotes

2016-06-21 Tuesday - Microsoft Project "Orleans" - Virtual Actor Abstractions

I'm spending some time today researching the Microsoft Project "Orleans"
http://research.microsoft.com/en-us/projects/orleans/

2016-06-03 Friday - Technical Proofer (TP) & Technical Development Editor (TDE)

Based on the quality of my previous contributions as a Manuscript Reviewer, I have been Invited by Manning Publications to participate in the Technical Proofer and Technical Development Editor programs. Looking forward to continuing to contributing to the process of bringing great technical books to the market.

2016-06-02 Thursday - Go 1.7 beta

Go 1.7 Beta released this week

• https://tip.golang.org/doc/go1.7

• https://golang.org/dl/#go1.7beta1

2016-06-02 Thursday - WS02 Refresh

Doing a refresh on my knowledge of the WS02 technology stack and product offerings...will continue to publish my notes here:
https://github.com/intltechventures/Lab.Architecture/blob/master/CloudComputing/WS02/WS02.md

It has been a few years since I last looked at the WS02 product offerings (previously evaluated their ESB and Registry/Repository for a client's Enterprise SOA initiative).

Here are links to my findings and JIRA issues submitted during my 2008 evaluation of WS02...

• http://intltechventures.blogspot.com/2008/02/2008-02-23-saturday.html
• http://intltechventures.blogspot.com/2008/02/2008-02-21-thursday.html

• http://comments.gmane.org/gmane.comp.java.wso2.registry.devel/1399
• http://comments.gmane.org/gmane.comp.java.wso2.registry.devel/3824 
• https://wso2.org/jira/browse/REGISTRY-233
• https://www.wso2.org/jira/browse/REGISTRY-459
• https://www.wso2.org/jira/browse/REGISTRY-549

It was around this time that I also evaluated the MuleSoft Registry/Repository offering (Galaxy)

• https://www.mulesoft.org/jira/browse/GALAXY-393
• http://apsblog.burtongroup.com/2008/01/mulesource-rele.html 

2016-05-25 Wednesday - Fixing my GDB (and Eclipse CDT Debugging capability)

I previously had my Windows 10 development machine configured to successfully develop and debug C++ programs from within Mars Eclipse - leveraging CDT and Cygwin.

I recently made some changes to my system configuration - and broke my Cygwin GDB debugging functionality.

After spending a few hours exploring the problem, I thought sharing this post might help someone else minimize their cycle time if they have a similar problem...

The first clue was in a Windows Command Prompt, I got the following error when trying to launch GDB:

"ImportError: No module named site"

Also, in Eclipse, when attempting to launch the debugger for a C++ program, I encountered this error message:

 "Could not determine DGB version using command gdb --version"

I  read several forum postings - and these were the relevant takeaways:

• http://stackoverflow.com/questions/5599872/python-windows-importerror-no-module-named-site
• https://docs.python.org/2/using/windows.html
• https://docs.python.org/3/using/windows.html
• "The PYTHONPATH variable is used by all versions of Python 2 and Python 3, so you should not permanently configure this variable unless it only includes code that is compatible with all of your installed Python versions."
• "he Python launcher for Windows is a utility which aids in locating and executing of different Python versions"
• "System-wide installations of Python 3.3 and later will put the launcher on your PATH. The launcher is compatible with all available versions of Python," 

Now, in my environment, I had setup Python 2.7.11 and Python 3.5.1 - and since GDB relies on Python 2.7 - I thought, "ah, perhaps one of my library installs / updates has resulted in a broken dependency situation". So, I uninstalled and re-installed them both. 

No joy.

I then uninstalled and re-installed Cygwin.

No joy.

Then I tried alternating where PYTHONHOME and PYTHONPATH pointed...to 2.7 vs 3.5

No joy.

Finally, I removed the System Variables for PYTHONHOME and PYTHONPATH.

Success. 

When in the Cygwin bash shell (or Windows Command Prompt window) - the Windows System Variables for PYTHONHOME and PYTHONPATH were conflicting with the Python 2.7 version that is bundled with Cygwin - and a dependency for GDB.

So I then just added the dir for Python 3.5 directly to the PATH environment variable. 

Problem solved.

2016-05-22 Sunday - Open Broadcaster Software

An interesting bit of Open Source software - possibly a good tool for recording instructor-led training material?

• https://obsproject.com/
• https://github.com/jp9000/obs-studio

OBS Studio (formerly known as OBS Multiplatform) is a complete rewrite of the original OBS from the ground up, with the main goals being multiplatform support, a more thorough feature set, and a much more powerful API. While still in its early stages, releases are currently available for Windows, Mac and Linux.

OBS Studio will eventually support many of the advanced requested features not present in the original OBS, such as multiple stream outputs and scene previewing, the latter of which is now available in the current release.

Open Broadcaster Software is free and open source software for video recording and live streaming. Supported features include:

• Encoding using H264 (x264) and AAC.
• Support for Intel Quick Sync Video (QSV) and NVENC.
• Unlimited number of scenes and sources.
• Live RTMP streaming to Twitch, YouTube, DailyMotion, Hitbox and more.
• File output to MP4 or FLV.
• GPU-based game capture for high performance game streaming.
• DirectShow capture device support (webcams, capture cards, etc).
• Windows 8 high speed monitor capture support.
• Bilinear or lanczos3 resampling.

2016-05-22 Sunday - Exploring Lattice Graphs in R

I'm exploring the Lattice Graphs library in R today...

Some background reading:

• http://www.statmethods.net/advgraphs/trellis.html
• R in Action, Second Edition 
• Chapter 23: Advanced graphics with the lattice package ~ is available on the Manning Publications web site, for free: 
• https://manning-content.s3.amazonaws.com/download/e/d13d6ca-337d-4c01-8260-02f13a32f2fd/BonusChapter-23.pdf
• http://master.bioconductor.org/help/course-materials/2008/advanced_R/latticeLab.pdf
• http://polisci.msu.edu/jacoby/uic/lattice/UIC%20Workshop,%20lattice%20%20Outline.pdf

2016-05-18 Wednesday - My Protocol for Learning New Technologies

A former colleague pinged me today, asking how do I go about learning new technologies...here's the slightly edited version of the response that I wrote: 

Depending on the specific technology, and whether you are going for depth or breadth, might well change how I would answer that question. Here's my general approach though: 

1) I usually first read the formal specification, the relevant RFCs, standards

2) I often assemble a Twitter List of the noteworthy companies and individuals that are contributing or are the thought leaders in that given technology 

3) I create a github project as my laboratory for experimenting with the technology (showing your work, and making it visible helps to motivate me to keep plugging away). Within the github project - usually using a naming convention of "Lab.xxxxxx..." I create a README file with links to references, tutorials, interesting articles, blogs, etc.  

4) I usually attend 1-2 conferences per year (https://qconsf.com/ is one of my favorites - it gives me the greatest opportunity to hear about a wide diversity of technologies - from a globally represented attendee roster. http://thestrangeloop.com/ in St. Louis is another good conf. for leading edge technology trends that may not be widely adopted yet - but are going to be very popular in 3-5 years - but some sessions can be a bit esoteric. 

5) I spend quite a bit of money every year buying books - and not just on the specific thing I'm learning - but on peripheral areas as well. My Kindle probably has over 100+ technical books on it - as well as whitepapers, references, etc. 

6) If a technology has a special niche in the area of algorithms, I usually read 4-6 academic research papers on the relevant current research for that area. 

7) I've found membership in the IEEE and ACM to be very useful to stay abreast of leading edge research and trends.  

8) I'm constantly downloading and experimenting with new Open Source software  

9) I read a lot of source code for Open Source software projects 

10) I usually attend 3-5 tech meetups in the Seattle area per month  

11) I have a few small personal software development projects that I continue to nurture privately, as a test bed for my experimentation with new technologies - for example, one is a code generation tool that can be pointed at a database schema and it will generate all of the SQL stored procedures, base object classes, etc. for over a dozen languages (~650K+ lines of code generated in less than 90 seconds)

12) Using Coding Exercises, for example, 99 Scala Problems
http://aperiodic.net/phil/scala/s-99/

2016-05-15 Sunday - Oden Functional Language for the Go Ecosystem

This looks like an interesting talk scheduled for the Curry On conference in Rome (July 18-19, 2016), by Oskar Wickström, entitled: "Oden - A Functional Programming Language for the Go Ecosystem"

Abstract "..Oden, an experimental, statically typed, functional programming language being built for the Go ecosystem.. aims to leverage the great features of Go — static linking, cross-compilation, goroutines, channels and the great set of libraries and tools — and enable higher-level abstractions, generics and a safer yet more flexible type system."

http://www.curry-on.org/2016/sessions/oden-a-functional-programming-language-for-the-go-ecosystem.html

2016-05-15 Sunday - 99 Problems

How do you go about learning a new language?

If you are like most folks, you probably have some simple app that you build for any given language that you might want to learn.

This is an interesting article, that articulates 99 problems - which is an interesting consistent way of approaching learning any new language...in particular, this specific article is Scala focused - which is convenient given my current research focus...
http://aperiodic.net/phil/scala/s-99/

2016-05-14 Saturday - AWS Command Line Interface (CLI)

I'm experimenting today with some of the capabilities of the AWS CLI. This post is a placeholder for my notes - that may be of use to others.

• https://aws.amazon.com/cli/
• http://docs.aws.amazon.com/cli/latest/reference/
• https://aws.amazon.com/documentation/cli/
• http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html 
• Amazon AWS Documentation  (for Kindle)

My intention is to leverage the AWS SDK for Go

• https://github.com/aws/aws-sdk-go/wiki

2016-05-14 Saturday - Laptop Setup (Installed Software)

I've updated the list of software that I typically have installed on my primary development laptop:
https://github.com/intltechventures/Consulting.Project.Tools/blob/master/LaptopSetup.md

2016-05-03 Tuesday - GCC 6.1

http://www.infoq.com/news/2016/05/gcc61-released

Today I've pulled the latest source code for the GNU GCC 6.1 compiler - reading great code is critical to becoming a better engineer.

• https://gcc.gnu.org/svn.html

Here are some links that may be of interest to others...

• https://gcc.gnu.org/gcc-6/changes.html
• https://gcc.gnu.org/onlinedocs/index.html#current
• https://gcc.gnu.org/onlinedocs/gcc-6.1.0/gcc/ 

2016-05-02 Monday - Time Series Analysis and Its Applications

During my current exercise to master the R programming language (MITx: 15.071x The Analytics Edge) - I happened to come across the web site for an interesting book today...
http://www.stat.pitt.edu/stoffer/tsa3/

Time Series Analysis and Its Applications: With R Examples, Third Edition
by Robert H. Shumway
and David S. Stoffer, Professor in the Department of Statistics at the University of Pittsburgh

The R package for the text is called asta (Applied Statistical Time Series Analysis)

You may be interested in the change log notes (latest version appears to be 1.3, Nov 2014)
http://www.stat.pitt.edu/stoffer/tsa3/xChanges.htm

They have also made available a free 200 page EZ Edition of the material
http://www.stat.pitt.edu/stoffer/tsa3/tsa3EZ.pdf

David S. Stoffer also collaborated with Randal Douc and Eric Moulines to write: Nonlinear Time Series: Theory, Methods, and Applications with R Examples
also available on Amazon

2016-05-01 Sunday - glibc DNS bug (CVE-2015-7547)

Dan Kaminsky has written an excellent post about the glibc DNS bug (CVE-2015-7547)

This is a critical security issue that should be high on everyone's list of patches to apply - to their Linux OS systems.

2016-04-30 Saturday - Suggested Machine Learning Books

On the recommendation of a friend, I've ordered a book today:

Understanding Machine Learning: From Theory to Algorithms, 1st Edition, Cambridge University Press
by Shai Shalev-Shwartz and Shai Ben-David
http://www.amazon.com/Understanding-Machine-Learning-Theory-Algorithms/dp/1107057132


Also, I've ordered this one too...

Foundations of Machine Learning (Adaptive Computation and Machine Learning series, MIT Press)
by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar
http://www.amazon.com/Foundations-Machine-Learning-Adaptive-Computation/dp/026201825X

2016-04-29 Friday - Windows 2016 (and Nano) Technical Preview 5

I'm spending some time this weekend exploring Windows 2016 Technical Preview 5 (and Windows Nano Server).

Here's a github project that I've started to help me organize notes and links to resources - that may also be of interest to others.
https://github.com/intltechventures/Lab.Windows2016

2016-04-27 Wednesday - A High-Level Design (HLD) Template

https://github.com/intltechventures/Consulting.Project.Tools/blob/master/templates/HighLevelDesign.md

Rationale: Numerous times, in my travels as a consultant, I've encountered organizations in which there is no established standard/template for what should be included in an HLD. Quite often (even in organizations that ostensibly have a template), the artifacts that architects and engineers produce devolve into a wild-wild west of anything-goes. Consistency engenders repeat-ability - a worthy goal. And therefore, the intended purpose in sharing this...at least as a starting point for further customization.

The goal of an HLD is to enable communication and coordination, both within teams and organizations - as well as serving as a tool for communication and coordination with external partners.

The HLD provides a consistent format for teams to assemble details - which support a goal of being able to reuse design artifacts - and is intended to provide sufficient information to Program/Project Management to be able to plan and coordinate large-scale development efforts. Additionally, an HLD serves as an effective mechanism to support Design Review and Architecture Governance efforts.
The target level of detail is to scope the effort, provide sufficient input to the estimation process, and to clearly articulate the How of an approach.

Investing effort in the creation of the HLD is intended for larger-scoped tasks - that have some non-trivial number of unknowns, technical complexity, a high number of coordination points, or any external integration/coordination requirements.

2016-04-24 Sunday - TiKV, a distributed KV database powered by Rust

Much goodness to grok in TiKV,
...a distributed KV database powered by Rust
...uses Raft to support Geo-Replication
https://github.com/pingcap/tikv

"TiKV is a Distributed Key-Value Database which mainly refers to the design of Google Spanner and HBase, but much simpler (Don't depend on any distributed file system). We've implemented the Raft consensus algorithm in Rust and stored consensus state in RocksDB. It not only guarantees consistency for data but also makes use of placement driver to implement sharding (split && merge) and data migration automatically. The transaction model is similar to Google's Percolator, and with some performance improvements."

2016-04-23 Saturday - Taxonomy of Artifact Classification Codes - updated

I've spent some time today doing a bit more organizing and extending of my working draft of a taxonomy of classification codes for various types of artifacts (e.g. proposal development, requirements, high level design, detail level design, architecture, testing, operations, training, etc.)

https://github.com/intltechventures/Consulting.Project.Tools/blob/master/taxonomies/ClassificationCodes.md

Still a work-in-progress, but it feels like it is beginning to congeal...

I fear I still have quite a bit of re-organization and re-structuring to do before I will be satisfied with the final product - but with what is there already - it is a decent foundation upon which others might yet build.

And so, it is worth publishing this update.  

2016-04-23 Saturday - MSI GT72 6QE Dominator Pro G Notebook Review

I've been working with a new notebook since January, an MSI GT72 6QE Dominator Pro G-034 (GTX 980M) with the 6th Generation Intel Core i7-6700HQ Skylake processor - running Windows 10.

I chose this particular model based on price, performance, features, and local store availability. 

I've had enough experience working with this new notebook to have reached some observations / conclusions that I'm willing to share...

• This seems like a rock-solid development machine
• I've been very pleased with the performance of the machine
• The keyboard is comfortable, although the position of the Insert, Delete, Home, and End keys has taken a bit getting used to...
• The screen is well lit - and resolution is fine
• The wireless/network connectivity is good - I'm lucky to have a 1GB internet connection through Wave G Fiber
• Disk access speeds are quite good
• Running various Linux distributions in Oracle's VirtualBox has been snappy
• Boot-up time is excellent
• It is much heavier than I expected - and quite bulky - and doesn't quite fit into most of my backpacks - but 'just' fits into a rolling bag
• The power unit is quite bulky and heavy
• The various MSI utilities / add-ons are quite handy / nice

Given that my work often requires me to travel - the size and weight of this machine is a definite drawback. However, given that much of my software engineering interest is focused on container- based distributed architectures - this platform gives me enough horsepower to run multiple virtual machines - and is an excellent platform for GPU parallel programming experimentation (1536 CUDA Cores).  If you are interested in the software that I have installed, click here.

Note:
Udacity offers a free NVIDIA created course, Intro to Parallel Programming, Using CUDA to Harness the Power of GPUs

Available On Amazon
(although I picked mine up at a local Frys.com)

• Intel® Core™ i7-6700HQ Processor(6M Cache, 2.6 GHz  - up to 3.50 GHz)
• 4 processors
• 24 GB SO-DIMM, GDDR5
• 256GB M.2 SATA 
• 1TB 7200 RPM
• 17.3 display, 1920 x 1080 
• NVIDIA Geforce GTX980M
• G-SYNC THUNDERBOLT 
• 6 USB 3.0 ports 
• DVD
• 8.4 pounds 
• 16.85 x 11.57 x 1.89 inches 

Additional Links:

• http://www.hardwareluxx.com/index.php/reviews/hardware/notebooks/36907-reviewed-msi-gt72-with-skylake-and-gtx-980m.html
• Youtube: MSI GT72 Dominator Pro Skylake Detailed Review and Benchmarks
• http://www.ultrabookreview.com/8839-intel-core-i7-6700hq/
• http://www.notebookcheck.net/Skylake-for-Notebooks-Core-i7-6700HQ-and-i7-6820HK-Review.150864.0.html
• http://laptopmedia.com/review/intel-skylake-core-i7-6700hq-and-core-i7-6500u-review-features-above-all/
• http://laptopmedia.com/highlights/intel-core-i7-6700hq-one-of-the-top-shelf-mobile-skylake-cpus-gets-benchmarked-and-added-to-our-ranking-table/ 
• http://processors.specout.com/l/1893/Intel-Core-i7-6700HQ
• http://www.notebookcheck.net/Intel-Core-i7-6700HQ-Notebook-Processor.149420.0.html
• "The Intel Core i7-6700HQ is a quad-core processor based on the Skylake architecture, that has been launched in September 2015. In addition to four CPU cores with Hyper-Threading clocked at 2.6 - 3.5 GHz (4 cores: max. 3.1 GHz, 2 cores: max. 3.3 GHz), the chip also integrates an HD Graphics 530 GPU and a dual-channel DDR4-2133/DDR3L-1600 memory controller." 

• http://www.geforce.com/hardware/notebook-gpus/geforce-gtx-980m
• http://www.geforce.com/hardware/notebook-gpus/geforce-gtx-980m/features
• http://www.geforce.com/hardware/technology/cuda/faq  
• http://www.nvidia.com/object/cuda_home_new.html
• https://www.youtube.com/user/NVCUDA/videos
• http://www.geforce.com/hardware/notebook-gpus/geforce-gtx-980m/specifications 
• Youtube: NVIDIA GeForce GTX 980M and 970M
• http://gpu.userbenchmark.com/Compare/Nvidia-GTX-980M-vs-Intel-HD-530-Desktop-Skylake/m15596vsm33102  
• http://www.digitaltrends.com/computing/nvidia-reveals-maxwell-based-geforce-gtx-980m-970m-laptop-gpus-980m-benchmarked-tested/
• http://www.notebookcheck.net/NVIDIA-GeForce-GTX-980M.126692.0.html
• "1536 shader ALUs and 96 TMUs are active. In addition, there are 64 ROPs in four clusters with 16 units each" 
• " memory interface has a bandwidth of 256-bits (4x 64 bit)"
• "Displays can be connected with a maximum resolution of 3840 x 2160 pixels via DisplayPort 1.2 or HDMI 2.0 (no HDCP @ 4K)"
• "video processor VP6 supports the usual formats like H.264 and can also decode and encode H.265 with a resolution of up to 4K via the NVENC API"
• 5200 Million Transistors

2016-04-23 Saturday - Visual Studio 2015 on Windows 10 - Screen Flicker

2016-04-30 Update:
After trying a number of things, I discovered that I just needed to uncheck the box for G-SYNC


My primary development machine is an MSI GT72 6QE Dominator Pro G (GTX-980M), 6th Generation notebook with the new Skylake processor.

When launching any of the Visual Studio 2015 tools, the screen flickers constantly.

If you are experiencing the same thing, it may be due to a a bug with the new Intel HD graphics drivers for the new Skylake processors.

You may be able to resolve the issue by disabling Hyper-V with this command:

> dism.exe /Online /Disable-Feature:Microsoft-Hyper-V

2016-04-22 Friday - Jo: a shell command to create JSON

A very nice (and elegant) command line utility (Jo, a shell command to create JSON) by Jan-Piet Mens (@jpmens)
http://jpmens.net/2016/03/05/a-shell-command-to-create-json-jo/

2016-04-22 Friday - Book Review: Docker In Action by Jeff Nickoloff

If you are interested in Docker, there is a new book out by Jeff Nickoloff, published by Manning that you should check out.

Docker In Action
https://www.manning.com/books/docker-in-action

I'm waiting for Manning to send me my copy of the book to review - will update this post after I've completed my read.

2016-04-20 Wednesday - Are You Serious About Innovating?

https://www.linkedin.com/pulse/you-serious-innovating-kelvin-meeks

I've been pondering this question lately - and thought that sharing this small sample of my own introspective questions might benefit others who struggle with nurturing the seed of innovation within organizations.

This is written from the specific perspective of a software engineering discipline – but is applicable to so many different types of organizations and teams.

Further, these thoughts are based on a slightly constrained premise that you have an existing product/service - and that stagnation has set in - and your innovation muscles have atrophied...

Questions that you might want to consider, in assessing whether your organization is serious about innovation…

1. Does your culture allow for errors and mistakes? If your organization operates under a threat of punishment and fear – how will your engineers ever have the courage to try something new and potentially innovative? I recently heard a CTO proudly state: “We have a perfect track record of tech choices and implementation so far, and we don't want to risk that”. A search of review comments on his company’s Glassdoor page – appears to reflect what I expected - a culture of fear and punishment. Which begs the question – are his engineers too afraid to try something new? Are yours?

2. Do you plan – and give your engineers time to “throw the first one away”, especially for critical new  components? If you are racing to the finish on every release – how much of your solution is designed (for maintenance, for scalability, for extensibility) – and how much is simply hacked together? How much innovation can be introduced – if there is no luxury of time for trial-and-error?

3. As an engineering organization, do you periodically do collective code reviews and discussions of innovative Open Source software projects to glean great ideas? More specifically, for the individual, I think there are five essential practices to becoming a world-class innovative engineer:

• Read great code written by others;
• Write a lot of code – with intent - focused on solving meaningful problems;
• Be willing to experiment;
• Constantly be re-learning your craft (patterns, algorithms, new developments, etc.);
• Share what you learn

4. If you did a poll of your engineering group – and ask them what books they are reading, how many would have a ready answer? What percentage are focused on your current technology stack – and what percentage are looking further afield? Innovation benefits from cross-pollination of ideas.

5. What are you doing to support access to innovative ideas and continued professional development within your engineering group? Do you have a corporate Safari account program in place? Why not? When was the last time you invited an innovative leader to give a talk to your engineering group?

6. How many of your engineering group are members of the IEEE and ACM? Those two organizations provide access to their Digital Libraries – and are a wealth of ideas for innovation. Here’s a tip: Provide your engineering group with reimbursement for their membership dues. But that doesn’t go far enough. You need to close the circle. Institute a monthly discussion forum in which a rotating set of people present innovative ideas that they have gleaned from the latest papers, journal
articles, etc.

7. Do you have a wiki for collecting links to innovative articles and Open Source projects? Sometimes the flood of new things is overwhelming – but you can at least have a process for organizing information and curating it for later re-discovery.

8. When is the last time you sent any of your engineering group to a conference? It isn’t about picking-up new skills – it is about the exposure to ideas – and the invigorating surge of energy that is shared by being exposed to a large collection of people that are doing innovative leading-edge things. Sure, they could watch conference talks on Youtube – but that isn’t nearly as effective in achieving the transmission of enthusiasm...nor does it help you in spotting potential new candidates to recruit for your engineering group – or communicate with enthusiasm some of the interesting new problems your group is tackling.

9. Is there some bit of code, tooling, etc. that you could Open Source – that might be beneficial to the Open Source community at large? Trust me, being supportive of your engineers’ effort to Open Source something (that is truly meaningfully useful) will repay itself in all kinds of ways that you cannot imagine. Second best – lend some portion of your engineering team’s efforts to support an existing Open Source project. Again, the payback in innovation, enthusiasm, pride, etc. is beyond what you can imagine.

10. How much of your engineering development effort is allocated to re-factoring? Unless your code base is of a trivial size – planning for re-factoring efforts needs to be part of your technology road-map. That’s also where innovation can seep back into your solution.

11. Do you have a formal Innovation Reward Program? Why not? If your business is driven by software, or derives value from the software that runs it – you are insane to not have a formal Innovation Reward Program. Establishing a culture of financially rewarding people for game-changing innovative ideas is moral, equitable, and in your best interest. Your business is people. Your best, most vital asset, is people. Take care of your people, and they will take care of you. Invest in them. Reward your people in proportion to the value they create. If someone in your engineering group creates a solution that will reduce costs by 10% - why not give them 1% of that reduction as an Innovation Reward? Reward the behavior that is beneficial – and eventually, it will stifle the behavior that isn’t.

12. Are your key business processes documented? Without first assessing what you do - and how you do it, and understanding the underlying cost/value of that activity - how will you prioritize and drive change with a focus on First Things First?

13. Have you done time-motion studies of your key critical process flows? Why not? How could they be reworked to dramatically improve efficiency and eliminate waste - to provide an innovative competitive advantage?

14. Do you actively listen to your end users and HEAR what they need and want? I was once brought in to consult to a major software engineering group on the East Coast - to help alleviate some amorphous and undefined problem that Executive Management didn't want to specify. The Director of Engineering was quite proud of the fact that they had Nobel Prize winners in their organization - and that his group did 26 releases a year. But when I interviewed their end users (in another State) - their biggest complaint was that they were underwhelmed by the slow/buggy behavior of the application - and were not pleased with 26 changes foisted upon them per year. Sometimes the Art of Listening with Intent - is the missing ingredient to achieving breakthrough innovation.

A critical component of sustained innovation is establishing a virtuous feedback loop of positive reinforcing behavior. Change doesn't happen overnight, but with the right incremental steps, support, and nurturing - any organization, in any field of endeavor, can be changed into an innovation dynamo.

My name is Kelvin Meeks, and I help software engineering organizations achieve breakthroughs in changing their culture, envisioning  new products and capabilities, introducing innovative new architectures, and achieving breakthroughs in performance & scalability.

2016-04-20 Wednesday - Exploring Apache SINGA 0.3.0, A General Distributed Deep Learning Platform

Exploring today's Apache SINGA 0.3.0 release, A General Distributed Deep Learning Platform
http://singa.apache.org/docs/overview.html

"SINGA is a general distributed deep learning platform for training big deep learning models over large datasets. It is designed with an intuitive programming model based on the layer abstraction. A variety of popular deep learning models are supported, namely feed-forward models including convolutional neural networks (CNN), energy models like restricted Boltzmann machine (RBM), and recurrent neural networks (RNN). Many built-in layers are provided for users. SINGA architecture is sufficiently flexible to run synchronous, asynchronous and hybrid training frameworks. SINGA also supports different neural net partitioning schemes to parallelize the training of large models, namely partitioning on batch dimension, feature dimension or hybrid partitioning."

2016-04-19 Tuesday - VirtualBox Host-Only Ethernet Adapter Problem

2016-04-19 3:35pm Update:
Uninstalling 5.0.16, rebooting, and doing a clean install of 5.0.18 appears to have resolved my issue.

I'm researching an issue setting up a VirtualBox Host-Only Ethernet Adapter in a Mint 17.3 Linux Guest OS (on Windows 10). Note: I have successfully configured this in a prior version (4.3.x) of VirtualBox.

You can see the error below.

I've been using VirtualBox 5.0.16 - but will uninstall and upgrade to the latest (5.0.18) today.

It is worth noting that I do have a Host-Only Ethernet Adapter installed - and Windows reports that the driver is up-to-date (confirmed, after I removed the device, and had it re-install the drivers). However, it appears that there is no entry in the Properties list for a Host-Only Ethernet Adapter connection item.

A number of posts have described a similar problem - going back to several prior releases of VirtualBox.

• https://forums.freebsd.org/threads/7631/

• https://github.com/docker/machine/issues/1521

• https://forums.virtualbox.org/viewtopic.php?f=6&t=36730 
• https://forums.virtualbox.org/viewtopic.php?f=6&t=68590
• https://forums.virtualbox.org/viewtopic.php?f=6&t=70199
• https://forums.virtualbox.org/viewtopic.php?f=6&t=74601 
• https://forums.virtualbox.org/viewtopic.php?f=6&t=74691
• "You have to enable "VirtualBox NDIS6 Bridged NetworkingDriver" in "VirtualBox Host-Only Ethernet Adapter" 's properties"? 
• https://caffinc.github.io/2015/11/fix-vbox-network/
• http://stackoverflow.com/questions/33725779/failed-to-open-create-the-internal-network-vagrant-on-windows10 
• http://superuser.com/questions/1002160/failed-to-start-virtual-device-in-oracle-virtual-box
• "Don't use NDIS6, it's known bug with this driver, you can install old NDIS5 driver which will solve this issue."
• http://ram.kossboss.com/virtualbox-host-only-adapter-error-verr_intnet_flt_if_not_found/
• http://www.codingepiphany.com/2015/11/22/virtualbox-5-host-only-error-failed-to-attach-the-network-lun/ 

• https://www.virtualbox.org/ticket/13713
• https://www.virtualbox.org/ticket/14040
• https://www.virtualbox.org/ticket/14832
•  "Installed the new Windows 10 update "threshold 2" via Windows Update, and VirtualBox is no longer able to use Host Only adapters"
• "Current workaround is to enable "NDIS6 Bridging" on the HostIF network device."
• "After re-installing VB I switched on "VirtualBox NDIS6 Bridged Networking Driver" in properties network connection."
•  

Helpful References:

• http://download.virtualbox.org/virtualbox/5.0.18/UserManual.pdf
• https://www.virtualbox.org/manual/UserManual.html 
• https://wiki.archlinux.org/index.php/VirtualBox

Some conjectures on how this problem might arise:

• Not properly shutting down a Guest OS before upgrading to a new version of VirtualBox
• Not doing a clean uninstall of VirtualBox before upgrading
• Not physically deleting the left-over contents of the VirtualBox directory after an uninstall
• A default Windows setting of [8] in the Windows Registry Key for MaxNumFilters
• Possibly needing to specify this as a parameter when installing some older release of VirtualBox? 
• -msiparams NETWORKTYPE=NDIS6
• Not running the Windows MSI installer as Admin?
• ...?

Error Encountered..

Failed to open/create the internal network 'HostInterfaceNetworking-VirtualBox Host-Only Ethernet Adapter' (VERR_SUPDRV_COMPONENT_NOT_FOUND).

Failed to attach the network LUN (VERR_SUPDRV_COMPONENT_NOT_FOUND).

Result Code:	E_FAIL (0x80004005)
Component:	ConsoleWrap
Interface:	IConsole {872da645-4a9b-1727-bee2-5585105b9eed}

2016-04-16 Saturday - We Make Seattle - A Short Film

2016-04-14 Thursday - Building Lua on Ubuntu

While trying to build Lua...
https://www.lua.org/download.html

curl -R -O http://www.lua.org/ftp/lua-5.3.2.tar.gz
tar zxf lua-5.3.2.tar.gz
cd lua-5.3.2
make linux test

I encountered the following error:

fatal error: readline/readline.h: No such file or directory

All that I needed to do was:  

sudo apt-get install libreadline-dev 

This blog post is also worth reviewing, as it illustrates how to find the various 

version of readline

(i.g. "apt-cache search readline")

Note the distinction between the lib version for running an application, versus 

for developing applications.