2020-07-31 Friday - Tools for Continuous Learning

Photo by 🇸🇮 Janko Ferlič on Unsplash
https://unsplash.com/photos/sfL_QOnmy00

Today, I received the following question via LinkedIn from a new connection: "...do you still believe in learning via reading books?..."

I believe in continuous Kaizen (改善) - and that also touches on self-directed personal learning. I am probably relatively high, among most cohorts, on any Autodidacticism spectrum.
 
Well-written books are a great treasure. When a competent author - who is also a subject-matter expert - has taken the time to think deeply about a subject - and has invested (usually) 12-18 months in researching, preparing, outlining, drafting, revising, writing, rewriting, reviewing, and rewriting some more - the finished work is usually a holistic view/treatment of the subject matter.

My personal library includes over 3,000 books - on a broad and deep cross-section of topics - including art, literature, writing, sales & marketing, business management, consulting, start-ups, entrepreneurship, mentoring/coaching, psychology, history, languages, philosophy, game theory, many different areas of science specialization (medicine, surgery, pharmacology, computer science, information technology, many areas of mathematics, physics, astronomy, geology, biology, chemistry, ...), several different areas of mechanical and electrical engineering disciplines, archaeology, travel, biographies, fiction, many different types of technical/craftsmanship skilled trade manuals, sailing, adventure, wilderness/pioneering skills, architecture, house building, farming and gardening, survival manuals, solo sports that rely on the individual's skill and performance, many different religions, several types of dictionaries, and a full set of Encyclopedia Britannica - as well as the 60 volume set of Great Books of the Western World.

As a solo consultant practitioner - knowledge management is a critically important skill. I must be able to develop (or expand, or refresh) strong  and deep domain and technical expertise quickly - and maintain selected areas of expertise on an on-going basis. What follows are some of the essential components of my personal strategy.

My commitment to learning also involves working on a freelance basis with Manning Publications - as a Technical Development Editor (TDE) for new books that are in development.

I also read quite a few blogs, articles, and papers on a weekly basis (my favorite source for research papers: arxiv.org).
 
Additional resources that I leverage:
- coursera.com
- YouTube.com - for conference talks, tech channels, etc.
- DZone.com
- InfoQ.com

- SlideShare.net
- HighScalability.com
 
- Dev.to
- Hackernoon.com
...are also sometimes good sources for interesting articles
 
- My "short list" of  useful links includes 1,935 entries, organized by many specialized topics and sub-topics...in addition to many technical podcasts that I monitor with iTunes...
 
I also spend quite a bit of time doing hands-on experimenting with new/leading edge technologies - which I leverage my 99 github repositories - or this research notebook blog - as simple knowledge management tools to capture lessons-learned, useful notes, reminders, tips, how-to's...
 
Following 3,500+ Twitter.com profiles (with many dozens of Lists defined to organize into specialized topics), and subscribing to 60+ Meetup.com groups, and 50+ LinkedIn.com groups - gives me additional vectors from which to maintain (or, quickly develop) a current situational-awareness of interesting technologies, discussions, and topics that are trending/emerging.

 

I've also invested time in building a carefully selected LinkedIn network of 5,000+ thought leaders, creators, innovators, entrepreneurs, researchers, inventors, executives, and highly skilled technologists - that I can quickly tap for their insights, experience, and expertise.
 
I often leverage StackOverflow.com  - when I need short/quick/urgent answers
 
For broader/deeper efforts to keep an eye on the horizon, and a bit further out...I leverage the following:
 
Conferences:
The Strange Loop conference held in St. Louis
QConSF.com San Francisco
 
Membership with the ACM.org and IEEE.org are also excellent resource for keeping your eyes on the prize, and a weather eye on the horizon. Their library of articles, as well as their journals - are top shelf.

2020-07-26 Sunday - Researching: Distributed Application Runtime (Dapr)

https://dapr.io/
An event-driven, portable runtime for building microservices on cloud and edge

Background Reading

https://github.com/dapr
https://github.com/dapr/dapr

https://github.com/dapr/docs
"A building block is as an HTTP or gRPC API that can be called from user code and uses one or more Dapr components. Dapr consists of a set of building blocks, with extensibility to add new building blocks."

https://github.com/dapr/docs/blob/master/concepts/bindings/README.md
"Using bindings, you can trigger your app with events coming in from external systems, or invoke external systems. This building block provides several benefits for you and your code:"

https://github.com/dapr/docs/blob/master/concepts/bindings/README.md#supported-bindings-and-specs

• Generic
• Cron (Scheduler)
• HTTP
• Kafka
• Kubernetes Events
• MQTT
• RabbitMQ
• Redis    
• Twilio    
• Twitter
• SendGrid
• AWS Services (see link above)
• Google Services (see link above)
• Azure Services (see link above)

Articles

• 2020
• Most commonly asked questions about Dapr 
  
  • https://medium.com/@abhishek1987/most-commonly-asked-questions-about-dapr-cd11e5cee555
• Dapr is “An event-driven, portable runtime for building microservices on cloud and edge.”
• "Service meshes are closer to infrastructure layer and operator concerns"
• "Dapr is closer to applications layer which developers are [responsible] for"
• "There is an overlap between Dapr and Service meshes …"
• "It is a `Runtime` that operates along with your application using a sidecar architecture — your application does not run `inside it`. In standalone mode, Dapr simply runs as a different process and in Kubernetes, it runs as a (sidecar) container in the same Pod as your application"
• "(Myth) Dapr depends on Docker"
•  Dapr Distributed Application Runtime with Azure CTO Mark Russinovich
• https://hanselminutes.com/718/dapr-distributed-application-runtime-with-azure-cto-mark-russinovich

• 2019
•  Announcing Distributed Application Runtime (Dapr), an open source project to make it easier for every developer to build microservice applications
• https://cloudblogs.microsoft.com/opensource/2019/10/16/announcing-dapr-open-source-project-build-microservice-applications/

  

2020-07-25 Saturday - Researching: Neural Autonomic Transport System (NATS)

image source: https://nats.io/

Background Reading

Neural Autonomic Transport System (NATS)

• https://nats.io/

• https://nats.io/about/#roadmap

• https://twitter.com/nats_io

https://en.wikipedia.org/wiki/NATS_Messaging

• NATS is an open-source messaging system (sometimes called message-oriented middleware). 
• The NATS server is written in the Go programming language. 
• Client libraries to interface with the server are available for dozens of major programming languages. The core design principles of NATS are performance, scalability, and ease of use.
• NATS was originally developed by Derek Collison as the messaging control plane for Cloud Foundry and was written in Ruby. NATS was later ported to Go.

• Microservices frameworks such as Micro, Mainflux, and Hemera rely on NATS as their messaging backbone.

NATS Documentation

https://docs.nats.io/

• Secure by default communications for microservices, edge platforms and devices
• Secure multi-tenancy in a single distributed communication technology
• Transparent location addressing and discovery
• Resiliency with an emphasis on the overall health of the system
• Ease of use for agile development, CI/CD, and operations, at scale
• Highly scalable and performant with built-in load balancing and dynamic auto-scaling
• Consistent identity and security mechanisms from edge devices to backend services

At Most Once and At Least Once Delivery
   
Support for Observable and Scalable Services and Event/Data Streams
   
Cloud Native, a CNCF project with Kubernetes and Prometheus integrations

NATS Streaming is also known as STAN, which is just NATS backward

Use Cases:

• Cloud Messaging
• Services (microservices, service mesh)
• Event/Data Streaming (observability, analytics, ML/AI)

• Command and Control
• IoT and Edge
• Telemetry / Sensor Data / Command and Control

• Augmenting or Replacing Legacy Messaging Systems   

https://docs.nats.io/nats-server/configuration

"While the NATS server has many flags that allow for simple testing of features, the NATS server
products provide a flexible configuration format that combines the best of traditional formats and
newer styles such as JSON and YAML."

• https://docs.nats.io/nats-server/configuration#configuration-properties   

https://docs.nats.io/developing-with-nats/events/slow

"NATS is designed to move messages through the server quickly. As a result, NATS depends on the applications to consider and respond to changing message rates. The server will do a bit of impedance matching, but if a client is too slow the server will eventually cut them off by closing the connection. These cut off connections are called slow consumers."

https://docs.nats.io/faq

• What is the difference between Request() and Publish()?

• Publish() sends a message to nats-server with a subject as its address, and nats-server delivers the message to any interested/eligible subscriber of that subject. Optionally, you may also send along a reply subject with your message, which provides a way for subscribers who have received your message(s) to send messages back to you.

• Request() is simply a convenience API that does this for you in a pseudo-synchronous fashion, using a timeout supplied by you. It creates an INBOX (a type of subject that is unique to the requestor), subscribes to it, then publishes your request message with the reply address set to the inbox subject. It will then wait for a response, or the timeout period to elapse, whichever comes first.

• Can multiple subscribers receive a Request?
• NATS is a publish and subscribe system that also has distributed queueing functionality on a per subscriber basis.

• Monitoring?
• Prometheus NATS Exporter Use Prometheus to configure metrics and Grafana to create a visual display.

• Size Limitations?

• NATS does have a message size limitation that is enforced by the server and communicated to the client during connection setup. Currently, the limit is 1MB.
• https://github.com/nats-io/nats.py/issues/46
• max_payload defaults to 1MB, but it can be configured in the server: 
• http://nats.io/documentation/server/gnatsd-config/
• https://github.com/nats-io/nats.go/issues/63
• KM Comment: As of 2020-07-25, this issue is still "Open"
• "The server enforces, but will send the error in an async fashion. The client should register this from the info and enforce it at the client API level."
• https://stackoverflow.com/questions/55368487/send-a-message-larger-than-1mb-using-nats-streaming
• "Messaging systems are not supposed to be used for file transfer. Use a distributed storage service to hold files and pass file ID in the message."
• "You can change the message size for nats while configuring the server."

• $ nats-server --config /path/to/nats.config

• # Override message size limit (bytes):
  max_payload: 100000000

• https://groups.google.com/forum/#!topic/natsio/Uwly62MRCzg
• "suggest...using NATS to pass around file "pointers", e.g. an S3 bucket URL, and transferring the actual files out-of-band. NATS is better suited to smaller messages, so I would advise against sending large BLOBs through it since you'd potentially be sequencing lots of 1MB messages (the default max payload size) and reassembling them."
• KM Comment: Read this post - there is a very good discussion in several other responses that elaborate on why pushing large files through NATS is a bad idea.

• Does NATS offer any guarantee of message ordering?
• NATS implements source ordered delivery per publisher. That is to say, messages from a
  given single publisher will be delivered to all eligible subscribers in the order in which
  they were originally published. There are no guarantees of message delivery order
  amongst multiple publishers.    

   

Github Resources

• https://github.com/nats-io
• https://github.com/nats-io/nats-general
• https://github.com/nats-io/nats-general/blob/master/architecture/DESIGN.md
• https://github.com/nats-io/nats-streaming-server
• https://github.com/nats-io/nats-streaming-server/issues
• https://github.com/nats-io/stan.go
• https://github.com/nats-io/nats-server
• https://github.com/nats-io/nats-server/issues

Docker Resources

• https://hub.docker.com/_/nats

Articles

• 2020
• ...
• 2018
• https://simplydistributed.wordpress.com/2018/03/30/kafka-nats-random-notes/
• "Kafka clients use a poll-based technique in order to extract messages as opposed to NATS where the server itself routes messages to clients (maintains an interest-graph internally)"
• "NATS can act pretty sensitive in the sense that it has the ability to cut off consumers who are not keeping pace with the rate of production as well as clients who don’t respond to heartbeat requests."
• "NATS doesn’t seem to have a notion of partitioning/sharding messages like Kafka does"
• https://medium.com/@philipfeng/modern-open-source-messaging-apache-kafka-rabbitmq-nats-pulsar-and-nsq-ca3bf7422db5
• "NATS was originally built with Ruby and achieved a respectable 150k messages per second. The team rewrote it in Go, and now you can do an absurd 8–11 million messages per second." 
• "Fire and forget, no persistence: NATS doesn’t do persistent messaging; if you’re offline, you don’t get the message."
• "NATS is not HTTP, it’s its own very simple text-based protocol, RPC-like. So it does not add any header to the message envelope."
• "NATS doesn’t have replication, sharding or total ordering"
• "With NATS, queues are effectively sharded by node. If a node dies, its messages are lost."
• "To fully guarantee that a message won’t be lost, it looks like you need to declare the queue as durable + to mark your message as persistent + use publisher confirms. And this costs several hundreds of milliseconds of latency."
• "NATS is useful when raw performance is a priority."
• vs. Kafka:
• "The broker barely knows anything about the consumer. All that’s really stored is an “offset” value that specifies where in the log the consumer left off."
• "Topics are arranged in partitions (for parallelism), and partitions are replicated across nodes (for high availability)."
• "Kafka is more matured compared to Nats and performs very well with huge data streams."
• "The only queue or pub/sub system that is relatively safe from partition errors is Kafka"
• "Kafka is just a really solid piece of engineering when you need 5–50 servers. With that many servers, you can handle millions of messages per second"
• "Kafka is binary over TCP"
• "Consumer cannot acknowledge message from a different thread"
• "No multitenancy"
• "Management in a cloud environment is difficult."
• https://www.quora.com/Is-NATS-IO-a-real-alternative-to-Kafka-Who-used-it-in-a-production
  
• 2017
• https://storageos.com/nats-good-gotchas-awesome-features
• "Over the last two years, I have used NATS extensively in many projects"
• "Lack of proper authentication. The current authentication method in NATS is not intended for end-users, and there is no way to dynamically add/remove users to NATS server." 
• [KM TODO: Follow-up and see if this is still true]
• "If you want a battle tested messaging system between your cluster nodes, I would definitely recommend NATS, which despite being in the community for a long time has done a great job of avoiding feature creep and bloat."
• 2016
• https://blog.greta.io/a-signaling-system-for-10m-concurrent-connections-10d327fd6837
• https://news.ycombinator.com/item?id=11284489
  

2020-07-24 Friday - What is your understanding of Enterprise Architecture?

This question was posed on LinkedIn recently:

"What is your understanding of Enterprise Architecture? What are the three the most important and the most prominent outcomes?"

https://www.linkedin.com/posts/nedimdedic_ea-enterprisearchitecture-eam-activity-6689164727978901504-Y2hO

These were my answers:

1) Alignment of the enterprise information technology capabilities  - and the technology roadmap with investment for innovation - to support the organization's business strategy, vision, priorities, growth, and opportunities.

2) Enabling the accelerated delivery of new innovative products and services - to grow the business - and serve the customer.

3) Maximizing the leveraging and investment of technology to ensure that the business is able to execute its mission - with reliable information technology platforms - that meet or exceed the performance, availability, scalability, and security requirements of the business.

2020-07-20 Monday - Grokking Artificial Intelligence Algorithms officially published today

• https://www.manning.com/books/grokking-artificial-intelligence-algorithms
• https://github.com/microservices-security-in-action/samples

2020-07-19 Sunday - Reseaching - Authentication Strategies for B2B Azure hosted APIs (i.e. North-South)

Photo by Jen Theodore on Unsplash
https://unsplash.com/photos/CiMITAJtb6I

This post is a a bit of a brain dump of research I'm doing this weekend on Azure Cloud API Authentication strategies - when you wish to expose a public B2B API for your business partners to consume (i.e. a North-South API Authentication problem).

For the moment, this post will be a bit of a mess - truly a brain dump. But, there are some excellent tidbits in this initial posting to give you a clear path to finding the right answer.  I'lll hopefully have time to come back and clean this up later - but no promises.

For the moment, consider this posting to be guidance for "recommended background reading".

Note: as a convention in writing notes for my personal use, I designate the very interesting bits with three asterisks (***)

Important, Read First

***
Microsoft identity platform and the OAuth 2.0 client credentials flow

https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-client-creds-grant-flow

"You can use the OAuth 2.0 client credentials grant specified in RFC 6749, sometimes called two-legged OAuth, to access web-hosted resources by using the identity of an application. This type of grant is commonly used for server-to-server interactions that must run in the background, without immediate interaction with a user. These types of applications are often referred to as daemons or service accounts."

"The OAuth 2.0 client credentials grant flow permits a web service (confidential client) to use its own credentials, instead of impersonating a user, to authenticate when calling another web service. In this scenario, the client is typically a middle-tier web service, a daemon service, or a web site."

"For a higher level of assurance, the Microsoft identity platform also allows the calling service to use a certificate (instead of a shared secret) as a credential."

"In the more typical three-legged OAuth, a client application is granted permission to access a resource on behalf of a specific user. The permission is delegated from the user to the application, usually during the consent process. However, in the client credentials (two-legged OAuth) flow, permissions are granted directly to the application itself. When the app presents a token to a resource, the resource enforces that the app itself has authorization to perform an action and not the user."

***
"An app typically receives direct authorization to access a resource in one of two ways:

• Through an access control list (ACL) at the resource
• Through application permission assignment in Azure AD"

"These two methods are the most common in Azure AD and we recommend them for clients and resources that perform the client credentials flow. "

"A resource can also choose to authorize its clients in other ways. Each resource server can choose the method that makes the most sense for its application."

***

Microsoft identity platform application authentication certificate credentials

https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-certificate-credentials

 

Relevant RFCs

RFC 6749 The OAuth 2.0 Authorization Framework
https://tools.ietf.org/html/rfc6749

RFC 6750 The OAuth 2.0 Authorization Framework: Bearer Token Usage
https://tools.ietf.org/html/rfc6750

RFC 7519 JSON Web Token (JWT)
https://tools.ietf.org/html/rfc7519

RFC 8725 JSON Web Token Best Current Practices
https://tools.ietf.org/html/rfc8725

RFC 5246 The Transport Layer Security (TLS) Protocol Version 1.2
https://tools.ietf.org/html/rfc5246

Additional Background Reading

Quickstart: Register an application with the Microsoft identity platform
https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app

Azure API Management
https://docs.microsoft.com/en-us/rest/api/apimanagement/

Azure API Management REST API Authentication
https://docs.microsoft.com/en-us/rest/api/apimanagement/apimanagementrest/azure-api-management-rest-api-authentication

Authorization Server
https://docs.microsoft.com/en-us/rest/api/apimanagement/2019-12-01/authorizationserver

Certificate
https://docs.microsoft.com/en-us/rest/api/apimanagement/2019-12-01/certificate

Gateway
https://docs.microsoft.com/en-us/rest/api/apimanagement/2019-12-01/gateway

Microsoft Azure Well-Architected Framework
https://docs.microsoft.com/en-us/azure/architecture/framework/

Cloud Design Patterns
https://docs.microsoft.com/en-us/azure/architecture/patterns/

Security patterns
https://docs.microsoft.com/en-us/azure/architecture/patterns/category/security

Federated Identity pattern
https://docs.microsoft.com/en-us/azure/architecture/patterns/federated-identity

"Implement an authentication mechanism that can use federated identity. Separate user authentication from the application code, and delegate authentication to a trusted identity provider. This can simplify development and allow users to authenticate using a wider range of identity providers (IdP) while minimizing the administrative overhead. It also allows you to clearly decouple authentication from authorization."

"The trusted identity providers include corporate directories, on-premises federation services, other security token services (STS) provided by business partners, or social identity providers that can authenticate users who have, for example, a Microsoft, Google, Yahoo!, or Facebook account."

https://docs.microsoft.com/en-us/azure/architecture/patterns/_images/federated-identity-overview.png

"This model is often called claims-based access control. Applications and services authorize access to features and functionality based on the claims contained in the token. The service that requires authentication must trust the IdP. The client application contacts the IdP that performs the authentication. If the authentication is successful, the IdP returns a token containing the claims that identify the user to the STS (note that the IdP and STS can be the same service). The STS can transform and augment the claims in the token based on predefined rules, before returning it to the client. The client application can then pass this token to the service as proof of its identity."

Valet Key pattern
https://docs.microsoft.com/en-us/azure/architecture/patterns/valet-key

"Use a token that provides clients with restricted direct access to a specific resource, in order to offload data transfer from the application. This is particularly useful in applications that use cloud-hosted storage systems or queues, and can minimize cost and maximize scalability and performance."

how expose a public api in azure
Quickstart: Configure an application to expose web APIs
https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-configure-app-expose-web-apis

Quickstart: Register an application with the Microsoft identity platform
https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app

"Your app is integrated with the Microsoft identity platform by registering it with an Azure Active Directory tenant. Enterprise developers and software-as-a-service (SaaS) providers can develop commercial cloud services or line-of-business applications that can be integrated with Microsoft identity platform. Integration provides secure sign-in and authorization for such services."
- Search for and select Azure Active Directory. Under Manage, select App registrations.
- Select New registration.
- In Register an application, enter a meaningful application name to display to users.

Quickstart: Configure a client application to access web APIs
https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-configure-app-access-web-apis

Calling an ASP.NET Core Web API from a WPF application using Azure AD V2
https://docs.microsoft.com/en-us/samples/azure-samples/active-directory-dotnet-native-aspnetcore-v2/calling-an-aspnet-core-web-api-from-a-wpf-application-using-azure-ad-v2/


API Management
Hybrid, multi-cloud management platform for APIs across all environments
https://azure.microsoft.com/en-us/services/api-management/

https://azure.microsoft.com/en-us/services/api-management/#security

- "Keep all your APIs behind a single static IP or domain, and help protect them with keys, tokens, and IP filtering."
- "Enforce flexible and fine-grained quotas and rate limits. Modify the shape and behavior of your APIs using policies. And improve latency and scale your APIs with response caching."
- "Connect on-premises APIs to cloud services by creating a façade that lets you safely integrate on-premises and cloud environments."

***
Architect API integration in Azure
https://docs.microsoft.com/en-us/learn/paths/architect-api-integration/

"Learn how to architect API integration in Azure, and provide secure, scalable API access for your applications."

Import and publish an API
https://docs.microsoft.com/en-us/learn/modules/publish-manage-apis-with-azure-api-management/4-import-and-publish-an-api

"To make an API available through an API gateway, you need to import and publish the API."

***
"There are various API frameworks and standards. API Management provides you with several options for importing APIs."
- Blank API
- Open API
- WADL
- WSDL
- Logic App    
- API App    
- Function App

"There are several ways to import an API into Azure API Management."

"Using the Azure portal, you select APIs, and then + Add API. You start by selecting the API framework you'd like to import."

Control authentication for your APIs with Azure API Management
https://docs.microsoft.com/en-us/learn/modules/control-authentication-with-apim/

- "Use API keys to secure your APIs"
- "Use client certificate authentication to secure your APIs"

***
Use client certificates to secure access to an API
https://docs.microsoft.com/en-us/learn/modules/control-authentication-with-apim/4-secure-access-client-certs

"Certificates can be used to provide TLS mutual authentication between the client and the API gateway." 

"You can configure the API Management gateway to allow only requests with certificates containing a specific thumbprint. The authorization at the gateway level is handled through inbound policies." 

"Here, you will learn how to configure API Management to accept client certificates." 

"With TLS client authentication, the API Management gateway can inspect the certificate contained within the client request and check for properties like:"

"Client certificates are signed to ensure that they are not tampered with. When a partner sends you a certificate, verify that it comes from them and not an imposter. There are two common ways to verify a certificate:"

"Check who issued the certificate. If the issuer was a certificate authority that you trust, you can use the certificate. You can configure the trusted certificate authorities in the Azure portal to automate this process."

"If the certificate is issued by the partner, verify that it came from them. For example, if they deliver the certificate in person, you can be sure of its authenticity. These are known as self-signed certificates."

"Create these policies in the inbound processing policy file within the API Management gateway:"

***
Check the thumbprint of a client certificate
"Every client certificate includes a thumbprint, which is a hash, calculated from other certificate properties. The thumbprint ensures that the values in the certificate have not been altered since the certificate was issued by the certificate authority. You can check the thumbprint in your policy. The following example checks the thumbprint of the certificate passed in the request:"

***
Check the thumbprint against certificates uploaded to API Management
"In the previous example, only one thumbprint would work so only one certificate would be validated. Usually, each customer or partner company would pass a different certificate with a different thumbprint. To support this scenario, obtain the certificates from your partners and use the Client certificates page in the Azure portal to upload them to the API Management resource. Then add this code to your policy:"

Check the issuer and subject of a client certificate
"This example checks the issuer and subject of the certificate passed in the request:"

Protect your APIs on Azure API Management
https://docs.microsoft.com/en-us/learn/modules/protect-apis-on-api-management/

"Create an Azure API gateway"
"Import a RESTful API into the gateway"
"Implement policies to secure and throttle the requests"
"Call an API to test the applied policies"

AZ-400: Develop a security and compliance plan
https://docs.microsoft.com/en-us/learn/paths/az-400-develop-security-compliance-plan/

"Build strategies around security and compliance that enable you to authenticate and authorize your users, handle sensitive information, and enforce proper governance."

Secure your identities by using Azure Active Directory
https://docs.microsoft.com/en-us/learn/modules/intro-to-azure-ad/

Microsoft Azure Well-Architected Framework - Security
https://docs.microsoft.com/en-us/learn/paths/secure-your-cloud-apps/


Encryption
https://docs.microsoft.com/en-us/learn/modules/azure-well-architected-security/5-encryption

Encrypting secrets
"Azure Key Vault is a cloud service that works as a secure secrets store. Key Vault allows you to create multiple secure containers, called vaults."

***

"Azure Key Vault can handle requesting and renewing Transport Layer Security (TLS) certificates, providing the features required for a robust certificate lifecycle management solution."

Network security
https://docs.microsoft.com/en-us/learn/modules/azure-well-architected-security/6-network-security

Using API gateways in microservices
https://docs.microsoft.com/en-us/azure/architecture/microservices/design/gateway

"If you don't deploy a gateway, clients must send requests directly to front-end services. However, there are some potential problems with exposing services directly to clients:"

***
Gateway Routing pattern
"Route requests to multiple services using a single endpoint. This pattern is useful when you wish to expose multiple services on a single endpoint and route to the appropriate service based on the request."
https://docs.microsoft.com/en-us/azure/architecture/patterns/gateway-routing

Choosing a gateway technology:
"Reverse proxy server. Nginx and HAProxy are popular reverse proxy servers that support features such as load balancing, SSL, and layer 7 routing. "

"Service mesh ingress controller."

"Azure Application Gateway. Application Gateway is a managed load balancing service that can perform layer-7 routing and SSL termination. It also provides a web application firewall (WAF)"

"Azure API Management. API Management is a turnkey solution for publishing APIs to external and internal customers. It provides features that are useful for managing a public-facing API, including rate limiting, IP restrictions, and authentication using Azure Active Directory or other identity providers. API Management doesn't perform any load balancing, so it should be used in conjunction with a load balancer such as Application Gateway or a reverse proxy."

   
   
***
Integrate API Management in an internal VNET with Application Gateway
https://docs.microsoft.com/en-us/azure/api-management/api-management-howto-integrate-internal-vnet-appgateway

***
"Azure Application Gateway and API Management are managed services."

***
Azure API Management and Application Gateway integration (2019)
https://medium.com/azure-architects/azure-api-management-and-application-gateway-integration-a31fde80f3db

"API Management deployed in “internal” VNET mode"
"Application Gateway (WAF) for exposing a subset of API’s externally"

   
Microsoft: API Management and App Gateway integration
https://miro.medium.com/max/955/1*1mBS7UOcQU2O8j3et9v7ig.png

***
"How are the API’s segregated so that only the ones deemed “external” are accessible via the Internet? Is it configured on API-M or the App Gateway?"
"It turns out the solution is a combination of both and is relatively simple -"

- "Within API-M, APIs are created with separate base URL’s i.e. /external and /internal"
- "Within Application Gateway, a path-based routing rule is created that redirects any API requests that contain /external to the API-M back-end"
"The same routing rule drops requests to any other API requests including /internal"

Publishing internal APIs to external consumers
https://docs.microsoft.com/en-us/azure/architecture/example-scenario/apps/publish-internal-apis-externally

"In this scenario, an organization has hosted multiple APIs using Application Service Environments(ILB ASE) and would like to consolidate these APIs internally using Azure API Management (APIM) deployed inside a Virtual Network. The internal API Management instance could also be exposed to external users to allow for utilization of the full potential of the APIs. This external exposure could be achieved using an Application Gateways forwarding requests to the internal API Management service, which in turn consumes the APIs deployed in the ASE."

***
See diagram:        

https://docs.microsoft.com/en-us/azure/architecture/example-scenario/apps/media/architecture-publish-internal-apis-externally.png

The data flows as follows:
- Developers check in code to a GitHub repository connected to CI/CD pipeline Agent installed on an Azure VM
- The agent pushes the build to the API application hosted on ILB ASE
-API Management consumes the above APIs via HOST Headers specified in API Management policy    
- API Management uses the App Service Environment's DNS name for all the APIs
- Application Gateway exposes API Management's developer and API portal
- Azure Private DNS is used to route the traffic internally between ASE, API Management, and 

Application Gateway
"External Users uses exposed Dev Portal to consume the APIs via Application Gateway's public IP"

"Azure Virtual Network enables Azure resources to securely communicate with each other, the internet, and on-premises networks."

"Azure Private DNS allows domain names to be resolved in a virtual network without needing to add a custom DNS solution."

"Azure API Management helps organizations publish APIs to external, partner, and internal developers to use their data and services."

"Application Gateway is a web traffic load balancer that enables you to manage traffic to your web applications."

"Internal Load Balancer App Service Environment is an Azure App Service feature that provides a fully isolated and dedicated environment for securely running App Service apps at high scale."

"Azure DevOps is a service for managing your development lifecycle and includes features for planning and project management, code management, build, and release."

"Application Insights is an extensible Application Performance Management (APM) service for web developers on multiple platforms."

"Azure Cosmos DB is Microsoft's globally distributed, multi-model database service."

KM Note: I submited Issue #2245 - the documentation states "SSL", but it should be "TLS"
https://github.com/MicrosoftDocs/architecture-center/issues/2245

       
       
How to Expose Services with Azure API Management
https://spr.com/how-to-expose-services-with-azure-api-management/

Azure Tips and Tricks
https://microsoft.github.io/AzureTipsAndTricks/

Getting started with Azure API Management       
https://microsoft.github.io/AzureTipsAndTricks/blog/tip197.html

Azure REST APIs with Postman
https://microsoft.github.io/AzureTipsAndTricks/blog/tip223.html

Taking a peek at Azure Key Vault Part 1 of 2
https://microsoft.github.io/AzureTipsAndTricks/blog/tip180.html

Taking a peek at Azure Key Vault Part 2 of 2
https://microsoft.github.io/AzureTipsAndTricks/blog/tip181.html

   

2020-07-19 Sunday -Weekend Reading - Microservices Security In Action

This weekend I'm reading the excellent "Microservices Security In Action", published by Manning Publications Co. - written by Prabath Siriwardena and Nuwan Dias
(both with WSO2)

Imminently practical. Well written. Concise. Love the inclusion of fairly recent citations and use cases - and in particular, the excellent footnote annotations to articles, papers, and other books.