2023-02-16 Thursday - Preparing for a Deep Dive into Bitcoin Source Code

[image credit: natassa64 on pixabay.com]

 

This post is a placeholder, as I gather, organize, and plan - a deep dive into Bitcoin's source code. 

I'm interested in developing a deep understanding of not just the concepts and architecture behind Bitcoin - but more importantly - the actual code.

Today's Wall Street Journal article (2023-02-16) spurred my interest:

WSJ: Bitcoin’s Future Depends on Six Mysterious Coders

 

Useful References/Resources:

1. https://bitcoincore.org/
1. https://twitter.com/bitcoincoreorg
   
2. https://bitcointalk.org/
3. https://bitcoin.stackexchange.com/ 
4. https://bitcoincore.reviews/
1. https://bitcoincore.reviews/meetings/ 
2. https://twitter.com/BitcoinCorePRs 
   
5. https://podcast.chaincode.com/
6. https://bitcoinops.org/ 
   
7. https://www.bitcoinfoundation.org/
   
1. https://twitter.com/BTCFoundation 
   

 

Github Resources:

1. https://github.com/bitcoin
   
1. https://github.com/bitcoin/bitcoin
2. https://github.com/bitcoin-core
   

2019-06-06 Thursday - Brent Ozar's SQL Server First Responder Kit

I happened across Brent Ozar's SQL Server consulting site  today (he's a Microsoft Certified Master) - who focuses on SQL Server, or in his own words: "I make SQL Server faster and more reliable."

Checkout some of his SQL Server Script resources:

• https://www.brentozar.com/first-aid/
• https://www.brentozar.com/first-aid/sp_blitzwho/
• https://www.brentozar.com/blitz/
• https://www.brentozar.com/blitzcache/
• https://www.brentozar.com/askbrent/
• https://github.com/BrentOzarULTD/SQL-Server-First-Responder-Kit

And, or course, his blog:

• https://www.brentozar.com/blog/
• For example: 
• https://www.brentozar.com/archive/2019/06/whats-better-ctes-or-temp-tables/
• https://twitter.com/BrentOzarULTD

As well as his YouTube Channel: Brent Ozar Unlimited

• https://www.youtube.com/user/BrentOzar

2019-04-12 Friday - Some Recent ImageNet ML Training Benchmarks

A few notes I've collected on recent ImageNet ML performance benchmark achievements - useful to help illustrate some architecture considerations - through a lens of infrastructure requirements - when trying to balance the potentially conflicting project constraints of accuracy/cost/performance.

This posting is a place-holder to provide a handy link for sharing some of those types of examples in the future.

Context:

• http://www.image-net.org/
• https://en.wikipedia.org/wiki/ImageNet

2019:

• New Technique Cuts AI Training Time By More Than 60 Percent
• https://news.ncsu.edu/2019/04/new-technique-cuts-ai-training-time-by-more-than-60-percent/
• "Adaptive Deep Reuse cut training time for AlexNet by 69 percent; for VGG-19 by 68 percent; and for CifarNet by 63 percent – all without accuracy loss."
• "The paper, “Adaptive Deep Reuse: Accelerating CNN Training on the Fly,” will be presented at the 35th IEEE International Conference on Data Engineering, being held April 8-11 in Macau SAR, China. The work was done with support from the National Science Foundation under grant numbers CCF-1525609, CNS-1717425 and CCF-1703487." 

• SenseTime Trains ImageNet/AlexNet In Record 1.5 minutes

• https://medium.com/syncedreview/sensetime-trains-imagenet-alexnet-in-record-1-5-minutes-e944ab049b2c
• "Researchers from Beijing-based AI unicorn SenseTime and Nanyang Technological University have trained ImageNet/AlexNet in a record-breaking 1.5 minutes, a significant 2.6 times speedup over the previous record of 4 minutes."
• "...a single NVIDIA M40 GPU requires 14 days to complete 90-epoch ResNet-50 training"
• "Researchers used 512 Volta GPUs for ImageNet/AlexNet training and achieved 58.2 percent accuracy in 1.5 minutes, with a corresponding training throughput of 1514.3k images/s and a 410.2 speedup ratio."
• "The previous record was held by a Tencent Machine Learning (腾讯机智, Jizhi) team, which used 1024 GPUs to train AlexNet on the ImageNet dataset in 4 minutes." 

2018:  

• Tencent ML Team Trains ImageNet In Record Four Minutes
• https://medium.com/syncedreview/tencent-ml-team-trains-imagenet-in-record-four-minutes-d3d85eff2062
• ""trained ResNet-50 in 6.6 minutes and AlexNet in just 4 minutes on the ImageNet dataset" \

• ImageNet Training in Minutes 
• https://arxiv.org/abs/1709.05011
• "Finishing 90-epoch ImageNet-1k training with ResNet-50 on a NVIDIA M40 GPU takes 14 days. This training requires 10^18 single precision operations in total. On the other hand, the world's current fastest supercomputer can finish 2 * 10^17 single precision operations per second (Dongarra et al 2017, this https URL). If we can make full use of the supercomputer for DNN training, we should be able to finish the 90-epoch ResNet-50 training in one minute. However, the current bottleneck for fast DNN training is in the algorithm level. Specifically, the current batch size (e.g. 512) is too small to make efficient use of many processors. For large-scale DNN training, we focus on using large-batch data-parallelism synchronous SGD without losing accuracy in the fixed epochs. The LARS algorithm (You, Gitman, Ginsburg, 2017, arXiv:1708.03888) enables us to scale the batch size to extremely large case (e.g. 32K). We finish the 100-epoch ImageNet training with AlexNet in 11 minutes on 1024 CPUs. About three times faster than Facebook's result (Goyal et al 2017, arXiv:1706.02677), we finish the 90-epoch ImageNet training with ResNet-50 in 20 minutes on 2048 KNLs without losing accuracy. State-of-the-art ImageNet training speed with ResNet-50 is 74.9% top-1 test accuracy in 15 minutes. We got 74.9% top-1 test accuracy in 64 epochs, which only needs 14 minutes. "  

• Now anyone can train Imagenet in 18 minutes
• https://www.fast.ai/2018/08/10/fastai-diu-imagenet/ 
• "...train Imagenet to 93% accuracy in just 18 minutes, using 16 public AWS cloud instances, each with 8 NVIDIA V100 GPUs, running the fastai and PyTorch libraries. This is a new speed record for training Imagenet to this accuracy on publicly available infrastructure, and is 40% faster than Google’s DAWNBench record on their proprietary TPU Pod cluster. Our approach uses the same number of processing units as Google’s benchmark (128) and costs around $40 to run."

• Training Imagenet in 3 hours for $25; and CIFAR10 for $0.26 
• https://www.fast.ai/2018/04/30/dawnbench-fastai/

• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour 
• https://arxiv.org/pdf/1706.02677.pdf
• "In this paper, we empirically show that on the ImageNet dataset large minibatches cause optimization difficulties, but when these are addressed the trained networks exhibit good generalization. Specifically, we show no loss of accuracy when training with large minibatch sizes up to 8192 images. To achieve this result, we adopt a hyperparameter-free linear scaling rule for adjusting learning rates as a function of minibatch size and develop a new warmup scheme that overcomes optimization challenges early in training. With these simple techniques, our Caffe2-based system trains ResNet-50 with a minibatch size of 8192 on 256 GPUs in one hour, while matching small minibatch accuracy. Using commodity hardware, our implementation achieves ∼90% scaling efficiency when moving from 8 to 256 GPUs."

2015: 

• Training Deep Net on 14 Million Images by Using A Single Machine
• http://dmlc.ml/mxnet/2015/10/27/training-deep-net-on-14-million-images.html
• "...how to train a neural network on Full ImageNet Dataset [1] with 14,197,087 images in 21,841 classes...using 4 GeForce GTX 980 cards on a single machine in 8.5 days"

Additional Useful Resources/References:

• https://benchmarks.ai/cifar-10

Other Interesting Articles:

2018:

• Review of Deep Learning Algorithms for Image Classification

2017-08-01 Tuesday - MuleSoft Performance & Best Practices

This posting is a placeholder for interesting information I happen to come across that is related to the topic of MuleSoft Performance & Best Practices.  Hopefully, others may find this of some use.

       
https://docs.mulesoft.com/mule-user-guide/v/3.8/reliability-patterns

• “Note that VM file persistency is disabled on clusters, so VM endpoints persist in-memory in a clustered topology.”
• “because we are calling a transformer, we have to allow for the possibility that it might fail and throw an exception. If it does, the file will be reprocessed, and might throw the same exception, and so on. To avoid an infinite loop, we used the redelivery policy configured at 2…”
•  “Reliability has performance implications”
• “If the outbound transport in the reliable acquisition flow is not transactional (for example, a flow from file-to-FTP), the only way to ensure message delivery is to turn off threading on the respective connector… If threading is not turned off, the caller may not notice the exception. That’s because the exception occurred in a different thread, and there is no way that one thread can see exceptions that occur in another thread.”

 

https://docs.mulesoft.com/mule-user-guide/v/3.8/tuning-performance

• “Incorrect combinations of thread configurations and maxBufferSize values can cause timeouts with no apparent cause. Counter intuitively, this issue is more probable with lower loads that don’t fully fill the buffer queue. This can cause outages. Load tests with high and low loads should be used to find and validate appropriate configurations.”

 
https://docs.mulesoft.com/mule-user-guide/v/3.8/distributed-file-polling
  

https://docs.mulesoft.com/mule-user-guide/v/3.8/file-transport-reference
https://docs.mulesoft.com/mule-user-guide/v/3.8/file-connector

• “The File connector only supports working with files on the same physical system than the running application. It doesn’t transfer files remotely over network connections. The only exception is when the operating system maps remote connections to local directories transparently”

https://docs.mulesoft.com/mule-user-guide/v/3.8/ftp-transport-reference

• “Put your login credentials in a properties file, not hard-coded in the configuration. This also allows you to use different settings between development, test, and production systems.”
• “The FTP transport does not support transactions as the File Transfer Protocol itself is not transactional. Instead you should design compensating transactions into your architecture using exception strategies in Mule”
• “If streaming is not enabled on the FTP connector, Mule attempts to read a file it picks up from the FTP server into a byte[] to be used as the payload of the MuleMessage. This behavior can cause trouble if large files need to be processed.”

https://docs.mulesoft.com/mule-user-guide/v/3.8/ftp-connector


https://docs.mulesoft.com/mule-user-guide/v/3.8/sftp-transport-reference

• “Mule 3.7 and newer supports SFTP retries. “

https://docs.mulesoft.com/mule-user-guide/v/3.8/sftp-connector

• “Mule uses the JCraft library for SFTP SSH”
• “SFTP Inbound Endpoints can only be set to a one-way exchange pattern. SFTP Outbound Endpoints can be set to either a one-way or request-response exchange pattern”

https://docs.mulesoft.com/mule-user-guide/v/3.8/configuring-reconnection-strategies

• "In addition to setting reconnection strategies on most connectors (Ajax, File, and VM are notable exceptions), you have the option to set them on global endpoints. (Once again, Ajax represents a major exception, which means that you can’t set a reconnection strategy on Ajax). However, MuleSoft recommends that whenever possible, you set your reconnection strategies on global connectors, rather than global endpoints, because this Best Practice generally allows you to reuse a once-written reconnection strategy again and again across all your flows and Mule projects."
• "The only situation in which MuleSoft recommends configuring a reconnection strategy on a global endpoint(rather than a global connector) involves Jetty, whose connector doesn’t support reconnection. This is why MuleSoft recommends the Jetty global endpoint instead."

http://www.mulesoft.org/docs/site/3.8.0/apidocs/org/mule/transport/file/ExpressionFilenameParser.html

• "ExpressionFilenameParser can use any expression language supported by Mule to construct a file name for the current message. Expressions can be xpath, xquery, ognl, mvel, header, function and more"

https://docs.mulesoft.com/mule-user-guide/v/3.8/exception-strategy-most-common-use-cases

https://docs.mulesoft.com/mule-user-guide/v/3.8/error-handling

• “Mule invokes a Messaging Exception Strategy whenever an exception is thrown within a flow…”
• “Each flow can contain only one exception strategy. However this can be a choice exception strategy that then refers to other nested exception strategies.
• “Choice exception strategies can contain one or more catch and/or rollback exception strategies. (Rollback and catch exception strategies cannot, however, contain other exception strategies.)”
• “Each exception strategy can contain any number of message processors.”
• “The exception strategy message processors should not throw exceptions, because you cannot create a nested exception strategy for these message processors”
• “Mule’s default exception strategy implicitly and globally handles all messaging exceptions that are thrown in Mule applications. You can override Mule’s default exception strategy by adding a catch, rollback, or choice exception strategy to a flow. Otherwise, when an exception is thrown in a flow, Mule automatically rolls back any pending transaction and logs the exception; if no transaction is involved, the default exception strategy simply logs the exception”

https://docs.mulesoft.com/anypoint-data-gateway/troubleshooting-data-gateway
https://docs.mulesoft.com/anypoint-data-gateway/

• "Anypoint Data Gateway allows you to easily integrate data stored in legacy back-office systems with Salesforce."
• "Data Gateway supports integration with systems such as SAP, Oracle, MS SQL, MySQL, and others"

Discussion Forum Findings…
https://forums.mulesoft.com/index.html
https://stackoverflow.com/search?q=mulesoft

For example:
https://forums.mulesoft.com/questions/68441/mule-performance.html
https://stackoverflow.com/questions/33230947/mule-ftp-polling-stops-without-error-or-warning


Some other interesting articles... 
http://www.modusbox.com/2016/01/tuning-mule-esb-for-high-performance-in-etl-scenarios/

https://www.appnovation.com/blog/mule-sftp-and-pgp-encryption 

https://dzone.com/articles/performance-tuning-in-mule-esb

https://stackoverflow.com/questions/39009139/pass-input-stream-over-http-in-mule-using-http-connector 

https://www.ricston.com/blog/batch-select-mule/

https://github.com/mulesoft/mulesoft-docs/blob/master/release-notes/v/latest/mule-runtime-3.8.0-m1-release-notes.adoc 

https://blogs.biztalk360.com/response-mulesoft-blog-10-reasons-walk-biztalk/

https://seekingalpha.com/filing/3543114 

2009-07-20 Monday - JavaRebel

David Booth (with zeroturnaround.com) pinged me this morning with the link to the recents suvery results they published: Java EE Containers – Heaven or Hell?