2019-04-27 Saturday - LaTeX learning resources

I've been mildly curious about learning LaTeX for a long time - and while reading many articles this weekend by Jeremy Kun, PhD - and being intrigued by his use of it to assemble both his dissertation - as well as self-publishing his book, "A Programmer's Introduction to Mathematics" - I spent a bit of time assembling this Tips.LaTeX.md file.

In particular, I spent some time assembling a list of what appear to be some of the more useful video tutorials available on YouTube.  If you have additional resource suggestions - you can leave a comment below.

2019-04-20 Saturday - Eclipse IDE v4.11 Release 2019-03 JSON Schema Validation Issues

During some recent testing of the latest release of the Eclipse IDE, v4.11, Release 2019-03 - I discovered some issues with JSON Schema Validation:

First:

While looking for a better JSON editor - I installed this plugin:

• Eclipse JSON Editor Plugin 
• https://marketplace.eclipse.org/content/json-editor-plugin 

But, I observed that

• It appears that when this plugin is installed - JSON Schema Validation (i.e. after registering the file pattern and schema URL via Window > Preferences > JSON > JSON Catalog) no longer works.

• I repeated this test several times, by uninstalling the plugin - confirming that Schema Validation worked - and then re-installing the plugin and confirming that Schema Validation no longer worked.

• I did not see any errors being logged to the Eclipse log file.

• Tested with Java JDK 12, on a Windows 10 laptop

 Issue #26 Submitted

• "Blocks JSON Schema Validation in Eclipse v4.11 (release 2019-03)"

• https://github.com/boothen/Json-Eclipse-Plugin/issues/26\

Second:

While experimenting with the JSON Schema Validation capabilities of the latest Eclipse release (v4.11, 2019-03) - I noted an error in the log (see my stackoverflow submission) - even though validation appeared to be working.

• relequestual replied soon after: "JSON Schema core dev here. Yeah this looks fine in terms of JSON Schema. No idea why an error is being throw" 

 Issue #546541 Submitted:

• "UnsupportedOperationException: Not a string: {"type":"string","format":"uri-reference"} thrown by JsonValue.java"
• https://bugs.eclipse.org/bugs/show_bug.cgi?id=546541
• (see my follow-up bug comments for additional information I found)

Subsequently:

• By changing the @schema string value from
•  "$schema": "http://json-schema.org/draft-07/schema#", 

• To:
•  "$schema": "http://json-schema.org/draft-04/schema#",

• It appears that the exception is no longer thrown. 

• However, when I change the string to either of the following: 
•  "$schema": "http://json-schema.org/draft-05/schema#",

• or
• "$schema": "http://json-schema.org/draft-06/schema#",

• ....a different error messages shows in the log - with a different exception is thrown. 

References:

• http://www.eclipse.org/webtools/
• http://git.eclipse.org/c/?q=webtools 
• http://git.eclipse.org/c/sourceediting/webtools.sourceediting.git/ 
• http://git.eclipse.org/c/sourceediting/webtools.sourceediting.git/tree/
• https://git.eclipse.org/c/sourceediting/webtools.sourceediting.git/tree/json/bundles/org.eclipse.json/src/org/eclipse/json/provisonnal/com/eclipsesource/json 
• https://git.eclipse.org/c/sourceediting/webtools.sourceediting.git/tree/json/bundles/org.eclipse.json/src/org/eclipse/json/provisonnal/com/eclipsesource/json/JsonValue.java

Observation: 

There are quite a few open bugs within the Eclipse WTP Source Editing and wst.json component - that have not been addressed in years...

• https://bugs.eclipse.org/bugs/buglist.cgi?bug_status=__open__&component=wst.json&list_id=18649565&product=WTP%20Source%20Editing

2019-04-15 Monday - IBM Developer Artificial Intelligence: Code Patterns

There appears to be a lot of great (and, recent) AI/ML/DL tutorial content on the IBM Developer portal - within the categories of Artificial Intelligence, Machine Learning, and Deep Learning - under "Code Patterns"

 Also see the Model Asset Exchange.

2019-04-12 Friday - Some Recent ImageNet ML Training Benchmarks

A few notes I've collected on recent ImageNet ML performance benchmark achievements - useful to help illustrate some architecture considerations - through a lens of infrastructure requirements - when trying to balance the potentially conflicting project constraints of accuracy/cost/performance.

This posting is a place-holder to provide a handy link for sharing some of those types of examples in the future.

Context:

• http://www.image-net.org/
• https://en.wikipedia.org/wiki/ImageNet

2019:

• New Technique Cuts AI Training Time By More Than 60 Percent
• https://news.ncsu.edu/2019/04/new-technique-cuts-ai-training-time-by-more-than-60-percent/
• "Adaptive Deep Reuse cut training time for AlexNet by 69 percent; for VGG-19 by 68 percent; and for CifarNet by 63 percent – all without accuracy loss."
• "The paper, “Adaptive Deep Reuse: Accelerating CNN Training on the Fly,” will be presented at the 35th IEEE International Conference on Data Engineering, being held April 8-11 in Macau SAR, China. The work was done with support from the National Science Foundation under grant numbers CCF-1525609, CNS-1717425 and CCF-1703487." 

• SenseTime Trains ImageNet/AlexNet In Record 1.5 minutes

• https://medium.com/syncedreview/sensetime-trains-imagenet-alexnet-in-record-1-5-minutes-e944ab049b2c
• "Researchers from Beijing-based AI unicorn SenseTime and Nanyang Technological University have trained ImageNet/AlexNet in a record-breaking 1.5 minutes, a significant 2.6 times speedup over the previous record of 4 minutes."
• "...a single NVIDIA M40 GPU requires 14 days to complete 90-epoch ResNet-50 training"
• "Researchers used 512 Volta GPUs for ImageNet/AlexNet training and achieved 58.2 percent accuracy in 1.5 minutes, with a corresponding training throughput of 1514.3k images/s and a 410.2 speedup ratio."
• "The previous record was held by a Tencent Machine Learning (腾讯机智, Jizhi) team, which used 1024 GPUs to train AlexNet on the ImageNet dataset in 4 minutes." 

2018:  

• Tencent ML Team Trains ImageNet In Record Four Minutes
• https://medium.com/syncedreview/tencent-ml-team-trains-imagenet-in-record-four-minutes-d3d85eff2062
• ""trained ResNet-50 in 6.6 minutes and AlexNet in just 4 minutes on the ImageNet dataset" \

• ImageNet Training in Minutes 
• https://arxiv.org/abs/1709.05011
• "Finishing 90-epoch ImageNet-1k training with ResNet-50 on a NVIDIA M40 GPU takes 14 days. This training requires 10^18 single precision operations in total. On the other hand, the world's current fastest supercomputer can finish 2 * 10^17 single precision operations per second (Dongarra et al 2017, this https URL). If we can make full use of the supercomputer for DNN training, we should be able to finish the 90-epoch ResNet-50 training in one minute. However, the current bottleneck for fast DNN training is in the algorithm level. Specifically, the current batch size (e.g. 512) is too small to make efficient use of many processors. For large-scale DNN training, we focus on using large-batch data-parallelism synchronous SGD without losing accuracy in the fixed epochs. The LARS algorithm (You, Gitman, Ginsburg, 2017, arXiv:1708.03888) enables us to scale the batch size to extremely large case (e.g. 32K). We finish the 100-epoch ImageNet training with AlexNet in 11 minutes on 1024 CPUs. About three times faster than Facebook's result (Goyal et al 2017, arXiv:1706.02677), we finish the 90-epoch ImageNet training with ResNet-50 in 20 minutes on 2048 KNLs without losing accuracy. State-of-the-art ImageNet training speed with ResNet-50 is 74.9% top-1 test accuracy in 15 minutes. We got 74.9% top-1 test accuracy in 64 epochs, which only needs 14 minutes. "  

• Now anyone can train Imagenet in 18 minutes
• https://www.fast.ai/2018/08/10/fastai-diu-imagenet/ 
• "...train Imagenet to 93% accuracy in just 18 minutes, using 16 public AWS cloud instances, each with 8 NVIDIA V100 GPUs, running the fastai and PyTorch libraries. This is a new speed record for training Imagenet to this accuracy on publicly available infrastructure, and is 40% faster than Google’s DAWNBench record on their proprietary TPU Pod cluster. Our approach uses the same number of processing units as Google’s benchmark (128) and costs around $40 to run."

• Training Imagenet in 3 hours for $25; and CIFAR10 for $0.26 
• https://www.fast.ai/2018/04/30/dawnbench-fastai/

• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour 
• https://arxiv.org/pdf/1706.02677.pdf
• "In this paper, we empirically show that on the ImageNet dataset large minibatches cause optimization difficulties, but when these are addressed the trained networks exhibit good generalization. Specifically, we show no loss of accuracy when training with large minibatch sizes up to 8192 images. To achieve this result, we adopt a hyperparameter-free linear scaling rule for adjusting learning rates as a function of minibatch size and develop a new warmup scheme that overcomes optimization challenges early in training. With these simple techniques, our Caffe2-based system trains ResNet-50 with a minibatch size of 8192 on 256 GPUs in one hour, while matching small minibatch accuracy. Using commodity hardware, our implementation achieves ∼90% scaling efficiency when moving from 8 to 256 GPUs."

2015: 

• Training Deep Net on 14 Million Images by Using A Single Machine
• http://dmlc.ml/mxnet/2015/10/27/training-deep-net-on-14-million-images.html
• "...how to train a neural network on Full ImageNet Dataset [1] with 14,197,087 images in 21,841 classes...using 4 GeForce GTX 980 cards on a single machine in 8.5 days"

Additional Useful Resources/References:

• https://benchmarks.ai/cifar-10

Other Interesting Articles:

2018:

• Review of Deep Learning Algorithms for Image Classification

2019-04-09 Tuesday - Proceedings from the 2018 NeurIPS Conference

Proceedings are available from the 2018 Conference on Neural Information Processing Systems (NIPS)

• http://papers.nips.cc/book/advances-in-neural-information-processing-systems-31-2018

Also, note the awards:

• https://nips.cc/Conferences/2018/Awards

2019-04-08 Monday - Laptop Upgrade: New Samsung 860 Evo SSD NAND Flash

Due to a bad block error that developed last week - over the weekend, I replaced the original 1 TB  Western Digital HGST Travelstar 7K1000  (1TB 7200RPM SATA, Model: HGST HTS721010A9E630 - datasheet) in my  MSI GT72 Dominator Pro G-034 laptop  (original specifications)- with a new SSD (Samsung MZ-76E1T0, 1 TB, 860Evo, 3D NAND FLASH) - that I purchased at a local Frys.com for $139.99 (after asking for the internet comparison shopping discount)

Here are the noteworthy performance improvements, per the CrystalDiskMark benchmark utility.

2019-04-06 Saturday - Visual Studio Code, Useful Marketplace Extensions

Useful Marketplace Extensions:

• Current File Path

• https://marketplace.visualstudio.com/items?itemName=YoshinoriN.current-file-path

• "Display current file's path from absolute (root directory) or workspace highest directory in StatusBar by Unix style or Windows style."