Why Reactive Principles Ensure System Scalability: A Conversation with Josh Long
The Reactive Principles, a new set of guidelines and techniques published by the Reactive Foundation, incorporate the ideas and patterns from both Reactive Programming and Reactive Systems into a set of practical software design concepts. These principles distill the experiences of leading experts from the broader distributed systems communities into a collection of principles and patterns for building Cloud Native, Edge Native, and Internet of Things (IoT) applications.
In this post, I catch up with Josh Long, Spring developer advocate, Java Champion, and a contributor to The Reactive Principles. We discuss his introduction to Reactive programming, why Reactive principles are so important for building distributed systems, and the practical challenges these principles solve in everyday enterprise environments.
How did you get introduced to Reactive programming/Reactive systems?
JL: Project Reactor! We on the Spring team developed it and started talking to the world about it in 2011, and I vaguely had some sense about its applicability, but I didn't understand the big picture. I didn't appreciate how Reactor, or projects like RxJava, could be scaled up into the day to day business of delivering software to production. I'd also seen what our friends at Lightbend were doing with actors and Akka, but I had trouble seeing the finish line.
I knew that Node.js and Vert.x gave us an event loop. I knew that Reactor gave me a ring buffer implementation and supported super high-speed dispatch. I knew that Akka gave me actors. But how was I supposed to connect all of this to production? I never cared about reactive streams. I cared about production. We talked about Reactor, but things only got interesting for me when we participated in the Reactive Streams work. It finally became clear: all of this stuff was foundational, org.reactivestreams.Publisher<T> could be the new java.util.Collection<T> as far as Spring developers would be concerned, and Spring was how I connected all of this stuff to production.
What makes Reactive principles so important for building distributed systems?
JL: There are three aspects to Reactive principles that mean a lot to me in building distributed systems.
The first concern is backpressure. It's not a new idea. Quite the contrary: we've had to write software to account for flow control from the very moment we had two computers on the same network and ran the risk of one computer going faster or slower than another. Flow control is a core concern when modeling any protocol between services. It's _always_ been a core concern. Whether people addressed it explicitly in their code–or even understood that they should address it–was quite another thing. Reactive programs make explicit this concern and provide developers a ubiquitous metaphor for working with flow control.
The second concern is the ease of integration and composition. In the wide and wonderful world of application integration and composition, consistency is key. Reactive Streams give us a consistent paradigm with which to describe... everything. SQL databases? Use a Publisher<T>. NoSQL databases? Use a Publisher<T>. Message queues? Use a Publisher<T>. HTTP web services? Use a Publisher<T>. WebSockets? Use a Publisher<T>. RSocket services? Use a Publisher<T>.
You get the idea. In a non-reactive world, all of the aforementioned applications (and many others besides) manifest in just as many incompatible APIs. You'll find it a lot easier to think about higher-order integration and composition when you're not constantly worrying about adapting one API to another.
The third concern is the decoupling of assembly time from execution time. In a reactive world, the pipelines that you build are a bit akin to the pipes through which water will flow. Defining a reactive pipeline is _not_ the same as turning on the faucet. That happens later, in execution time, through an operator like Publisher<T>#subscribe().
This is a _very_ simple idea, but it has some potent applications because the streams' final routing is up in the air until the very moment the stream starts executing. This leaves plenty of opportunities for operators to intervene on our behalf after we've defined our pipeline and before the pipeline executes.
What practical challenges do Reactive principles solve in everyday enterprise environments?
JL: I see several benefits to Reactive principles in everyday enterprise environments. First: they significantly reduce the incidence and complexity of disparate APIs and corresponding code for:
Reliability concerns
Error handling
Threading
Scalability
Reactive libraries like Reactor ship with a _ton_ of useful operators that obviate the need for several other things shipped as standalone libraries. Things like retries, timeouts, and rate limiting all simplify API integration.
Reactive moves error handling from thread-bound try/catch blocks to error propagation. The result is that errors become just another kind of data handled in a pipeline, simplifying things considerably.
Reactive principles also give runtimes every opportunity to repurpose threads, leading to more efficient thread resource use. This promotes scalability, allowing our services to handle more requests without extra hardware spend.
Reactive APIs provide us a DSL with which to describe data flow. The runtime takes care to ensure the desired (possibly parallel, possibly asynchronous) threaded execution without exposing the developer to subtly incorrect threading code risks.
In your opinion, what makes Reactive principles such a good fit for cloud native applications?
JL: Reactive principles are more natural fits in the context of a cloud native system because they promote easier integration and composition and because they are more scalable, able to better deal with the increased service demands implied in a service-centric architecture.
How do you recommend software architects and developers get started using these principles?
JL: I find it most straightforward to think about production when endeavoring to learn Reactive principles. Don't think about Reactive for Reactive's sake. Focus instead on using Reactive to build more natural HTTP endpoints, do more scalable data access, or perform service orchestration and composition more naturally.
Mark your calendar for Josh’s talk, “Bootiful RSocket,” on Tuesday, November 10th at the Reactive Summit, a one-day virtual conference hosted by the Reactive Foundation and focused on Reactive principles, patterns, and projects.