Saturday, July 25, 2020

2020-07-25 Saturday - Researching: Neural Autonomic Transport System (NATS)

image source: https://nats.io/

Background Reading

Neural Autonomic Transport System (NATS)

  • NATS is an open-source messaging system (sometimes called message-oriented middleware). 
  • The NATS server is written in the Go programming language. 
  • Client libraries to interface with the server are available for dozens of major programming languages. The core design principles of NATS are performance, scalability, and ease of use.
  • NATS was originally developed by Derek Collison as the messaging control plane for Cloud Foundry and was written in Ruby. NATS was later ported to Go.
  • Microservices frameworks such as Micro, Mainflux, and Hemera rely on NATS as their messaging backbone.

NATS Documentation

    • Secure by default communications for microservices, edge platforms and devices
    • Secure multi-tenancy in a single distributed communication technology
    • Transparent location addressing and discovery
    • Resiliency with an emphasis on the overall health of the system
    • Ease of use for agile development, CI/CD, and operations, at scale
    • Highly scalable and performant with built-in load balancing and dynamic auto-scaling
    • Consistent identity and security mechanisms from edge devices to backend services
At Most Once and At Least Once Delivery
   
Support for Observable and Scalable Services and Event/Data Streams
   
Cloud Native, a CNCF project with Kubernetes and Prometheus integrations

NATS Streaming is also known as STAN, which is just NATS backward

Use Cases:
  • Cloud Messaging
    • Services (microservices, service mesh)
    • Event/Data Streaming (observability, analytics, ML/AI)
  • Command and Control
    • IoT and Edge
    • Telemetry / Sensor Data / Command and Control
  • Augmenting or Replacing Legacy Messaging Systems   

"While the NATS server has many flags that allow for simple testing of features, the NATS server
products provide a flexible configuration format that combines the best of traditional formats and
newer styles such as JSON and YAML."

"NATS is designed to move messages through the server quickly. As a result, NATS depends on the applications to consider and respond to changing message rates. The server will do a bit of impedance matching, but if a client is too slow the server will eventually cut them off by closing the connection. These cut off connections are called slow consumers."

https://docs.nats.io/faq
  • What is the difference between Request() and Publish()?
    • Publish() sends a message to nats-server with a subject as its address, and nats-server delivers the message to any interested/eligible subscriber of that subject. Optionally, you may also send along a reply subject with your message, which provides a way for subscribers who have received your message(s) to send messages back to you.
    • Request() is simply a convenience API that does this for you in a pseudo-synchronous fashion, using a timeout supplied by you. It creates an INBOX (a type of subject that is unique to the requestor), subscribes to it, then publishes your request message with the reply address set to the inbox subject. It will then wait for a response, or the timeout period to elapse, whichever comes first.
  • Can multiple subscribers receive a Request?
    • NATS is a publish and subscribe system that also has distributed queueing functionality on a per subscriber basis.
  • Monitoring?
    • Prometheus NATS Exporter Use Prometheus to configure metrics and Grafana to create a visual display.
  • Size Limitations?
    • NATS does have a message size limitation that is enforced by the server and communicated to the client during connection setup. Currently, the limit is 1MB.
  • Does NATS offer any guarantee of message ordering?
    • NATS implements source ordered delivery per publisher. That is to say, messages from a
      given single publisher will be delivered to all eligible subscribers in the order in which
      they were originally published      . There are no guarantees of message delivery order
      amongst multiple publishers.          
   

Docker Resources

Articles

  • 2020
    • ...
  • 2018
    • https://simplydistributed.wordpress.com/2018/03/30/kafka-nats-random-notes/
      • "Kafka clients use a poll-based technique in order to extract messages as opposed to NATS where the server itself routes messages to clients (maintains an interest-graph internally)"
      • "NATS can act pretty sensitive in the sense that it has the ability to cut off consumers who are not keeping pace with the rate of production as well as clients who don’t respond to heartbeat requests."
      • "NATS doesn’t seem to have a notion of partitioning/sharding messages like Kafka does"
    • https://medium.com/@philipfeng/modern-open-source-messaging-apache-kafka-rabbitmq-nats-pulsar-and-nsq-ca3bf7422db5
      • "NATS was originally built with Ruby and achieved a respectable 150k messages per second. The team rewrote it in Go, and now you can do an absurd 8–11 million messages per second." 
      • "Fire and forget, no persistence: NATS doesn’t do persistent messaging; if you’re offline, you don’t get the message."
      • "NATS is not HTTP, it’s its own very simple text-based protocol, RPC-like. So it does not add any header to the message envelope."
      • "NATS doesn’t have replication, sharding or total ordering"
      • "With NATS, queues are effectively sharded by node. If a node dies, its messages are lost."
      • "To fully guarantee that a message won’t be lost, it looks like you need to declare the queue as durable + to mark your message as persistent + use publisher confirms. And this costs several hundreds of milliseconds of latency."
      • "NATS is useful when raw performance is a priority."
      • vs. Kafka:
        • "The broker barely knows anything about the consumer. All that’s really stored is an “offset” value that specifies where in the log the consumer left off."
        • "Topics are arranged in partitions (for parallelism), and partitions are replicated across nodes (for high availability)."
        • "Kafka is more matured compared to Nats and performs very well with huge data streams."
        • "The only queue or pub/sub system that is relatively safe from partition errors is Kafka"
        • "Kafka is just a really solid piece of engineering when you need 5–50 servers. With that many servers, you can handle millions of messages per second"
        • "Kafka is binary over TCP"
        • "Consumer cannot acknowledge message from a different thread"
        • "No multitenancy"
        • "Management in a cloud environment is difficult."
    • https://www.quora.com/Is-NATS-IO-a-real-alternative-to-Kafka-Who-used-it-in-a-production
  • 2017
    • https://storageos.com/nats-good-gotchas-awesome-features
      • "Over the last two years, I have used NATS extensively in many projects"
      • "Lack of proper authentication. The current authentication method in NATS is not intended for end-users, and there is no way to dynamically add/remove users to NATS server." 
        • [KM TODO: Follow-up and see if this is still true]
      • "If you want a battle tested messaging system between your cluster nodes, I would definitely recommend NATS, which despite being in the community for a long time has done a great job of avoiding feature creep and bloat."
  • 2016
Posted by

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Copyright

© 2001-2021 International Technology Ventures, Inc., All Rights Reserved.