Intro to Virtual Actors by Microsoft Orleans
Recently I wanted to dig deeper into what Microsoft Orleans is about and how it’s working and I had a painful experience because even if Orleans is a Microsoft project, it lacks plenty of publicly available resources to learn about it.
So, because of my experience, I’ve decided to add some new blog posts to my collection for helping people like me that are new to Orleans to speed up the learning process.
Actor Model vs Virtual Actor Model
First of all, let’s remember that there exists the Actor Model pattern, that’s implemented by various frameworks and platforms like Akka.io.
By definition, the Actor Model is:
The actor model in computer science is a mathematical model of concurrent computation that treats actor as the universal primitive of concurrent computation. In response to a message it receives, an actor can: make local decisions, create more actors, send more messages, and determine how to respond to the next message received. Actors may modify their own private state, but can only affect each other indirectly through messaging (removing the need for lock-based synchronization).
If you want a more complete overview of this brilliant idea, please check the following talk by Carl Hewitt, the inventor of the concept.
Now, we can ask ourselves what’s the Virtual Actor Model, and what is the difference between this concept and the previous one. More information about this concept can be found in this research paper.
The main difference between actors and virtual actors is that the latter it’s totally abstracting away the physical instantiation of actors by leaving this to be managed by the runtime.
Some of the implementations of the virtual actors' concept are:
- Microsoft Orleans
- dapr.io
What is Microsoft Orleans
As can be already concluded from the above intro, Microsoft Orleans is a Virtual Actor Model implementation.
More specifically, Microsoft Orleans is a .NET platform that allows developers to build distributed applications. It can scale from single on-premises servers to globally distributed services in the cloud.
Microsoft Orleans uses the same concepts (classes, interfaces, async/await or TPL, try/catch error handling, etc) that a developer is using for building a simple on-premise application and uses them in a distributed environment. This allows developers to be able to easily build distributed systems by using the same familiar concepts.
Why use Microsoft Orleans
If it is not yet obvious why to use such a platform, then I will state it clear: for building scalable and fault-tolerant distributed systems.
Main concepts of Microsoft Orleans
Grain
According to Microsoft Orleans official documentation:
The fundamental building block in any Orleans application is a grain. Grains are entities comprising user-defined identity, behavior, and state. Grain identities are user-defined keys which make Grains always available for invocation. Grains can be invoked by other grains or by external clients such as Web frontends, via strongly-typed communication interfaces (contracts). Each grain is an instance of a class which implements one or more of these interfaces.
For people with experience when it comes to the Actor Model, a Grain is the equivalent of an Actor. If in the Actor Model world, everything is an Actor, then in Microsoft Orleans world, everything is a Grain.
In order to summarize what a Grain is about, we should keep in mind that the Grain is the sum of three characteristics:
- identity
- behavior
- state
Silo
If we think about where a farmer will keep its grains, then this is a Silo for sure. The same is true for Orleans Grains, too.
Microsoft Orleans Runtime is implementing the programming model for applications and the main component of it is the so-called Silo. The responsibility of a Silo is to host one or more Orleans Grains.
Below is how we should visualize a Microsoft Orleans Silo: a container that’s hosting multiple virtual actors called Grains.
We can see that in a silo, we can have multiple grains. Keep in mind that each grain is different from another, because it has a different identity in the first hand, but it can have different behavior or different state.
Some of the best two features/characteristics provided by Microsoft Orleans are:
- scalability
- fault-tolerance
For achieving the above-mentioned characteristics we will need to run a cluster of Silos.
If we will run a group of silos together, then those will form a cluster that will increase the scalability and fault-tolerance degrees. At the same time, running a cluster of Microsoft Orleans silos will distribute the work and will allow detection and recovery from errors.
The Microsoft Orleans runtime will enable grains from within the cluster to communicate with each other as if they are hosted by the same silo/process.
Microsoft Orleans — major features
Microsoft Runtime provides a set of services that can be accessed in each silo from the grains within it. In the following, we will detail some of the major features that grain can benefit from.
Persistence
We can leverage the persistent grains by using a simple persistence model provided by Microsoft Orleans.
For using persistent grains, we need to use a storage provider. There are multiple out-of-the-box storage providers for multiple databases, available via Orleans.Providers namespace, but at the same time, we can build a custom one.
Timers & Reminders
Microsoft Orleans runtime provides built-in support for using a scheduling mechanism for grains. We can use timers & reminders to ensure that a certain action will take place in the upcoming future, even if the grain is not activated at this point in time.
This feature is very useful, and usually, when building services we need to use an external scheduling library like Quartz.NET.
Distributed ACID Transactions
Microsoft Orleans provides a simple persistence model, as we have already mentioned, but as an addition, grains can have a transactional state even in distributed environments. By this mean, multiple grains can participate in executing an ACID transaction regardless of where their state is stored.
An important aspect when it comes to this feature of Microsoft Orleans is that transactions are distributed, because the state of the grains is distributed, and decentralized because there isn’t any central point of coordination (such as a transaction manager or transaction coordinator). Distributed transactions provide a serializable isolation level.
Replication
Replication is a complex distributed system topic that needs to be discussed in large, but in the following, we will try to explain how Microsoft Orleans is providing support for this feature.
First of all, when we deal with replication, generally, we will benefit from:
- the highest degree of availability because each node (silo) in the cluster can be used for serving data;
- we can scale up and allow increasing the workloads because the nodes (silos) in the cluster can be used for doing parallel processing;
- fast/instantaneous failure recovery due to the fact that multiple nodes )(silos) are storing an instance of our replicated grain.
Specifically, when it comes to Microsoft Orleans and Virtual Actor Model, using replicated grain instances, will take care of the following aspects:
- consistent versioning: a grain can have multiple instances, each of them having a proprietary state that will be created using the same sequence of events. This implies the fact that if two grain instances are having the same version number, then they are having the same state;
- racing events: an event can be raised by multiple grain instances and this will require resolving a race condition and an agreement for deciding on how the sequence of events should look like;
- notifications/reactivity: a notification will be raised to all grain instances, whenever an event is raised by one specific grain instance.
Partitioning
Like replication, partitioning is another complex distributed systems topic that is tackled by Microsoft Orleans.
Generally speaking, when we are using partitioning, we will take advantage of the following:
- automatic workload scalability because the data is partitioned across multiple cluster members and we can use those members in parallel for doing computations;
- increased resilience and simplified recovery from failures achieved by using partition backups;
- lower latency and less load on the data stores are achieved by keeping the grain state in memory while the grain is active.
In Microsoft Orleans, grains are implicitly partitioning their application state, leading us to benefit from all the above-mentioned advantages.
!!! More details about Microsoft Orleans’ features are available here.
Example project
In the following, we will see a real example of how to use Microsoft Orleans for developing a distributed counter. We will use some of the advanced features provided by the platform, like:
- clustering
- persistence
- client-server
High-level architecture
The High-level architecture looks like in the following diagram:
System’s components
The system developed consists of:
- One component that will be instantiated for starting a new Microsoft Orleans Silo. You can find it here.
- One component that is containing the Microsoft Orleans Grains. This is basically the type-safe API by which we can communicate with the Microsoft Orleans cluster. You can check it out here.
- An HTTP Web API built by using .NET 6 minimal web API feature. This will allow triggering the operations of increment and decrement of the distributed counter.
Microsoft Orleans Grains
As we have already mentioned, the project that will contain the Grains will be used as a type-safe API to communicate with the Silos.
Because of this, we will create a Class Library for .NET or .NET Standard project and will add to it the following Nuget packages as its dependencies:
- Microsoft.Extensions.Caching.Abstractions
- Microsoft.Extensions.DependencyInjection
- Microsoft.Extensions.Options
- Microsoft.Orleans.Client
- Microsoft.Orleans.CodeGenerator.MSBuild
- Microsoft.Orleans.Core.Abstractions
For understanding how all the things are working, we will start by having a look at the following interface, which is basically the Grain type-safe API.
In the above code, we can see that our custom interface is extending the IGrainWithStringKey interface, that’s allowing us to create a Grain where its key will be of type String and that the Grain is exposing four methods. It’s worth mentioning that our grain will be a persisted one, so all the methods will interact in one way or another with the persistent storage.
First of all, this is how the class that it’s implementing the above interface looks like:
It’s very important to note the [StorageProvider] class attribute that will allow us to transform our grain into a persistent one. One of the features provided by the Microsoft Orleans Silo is that each of the silos can point to different data stores, so by using this attribute we declare how a grain depends on a specific storage provider.
Also, another important aspect is the fact that the class is deriving the Grain<T> class, which will allow us to access the grain’s state. So, by this mean, our grain will be a stateful one.
A constraint imposed on stateful grains is that those must declare the type of stateful data that will manage. In our case, the type of stateful data is Immutable<T>, as can be seen in the extension clause.
The state of the grain can be accessed by using the State property exposed by Grain<T>.
Set() Method
In the following, we will see how the Set() method will be implemented.
The responsibility of this method is to set the State property of the grain to a new value provided as a method parameter.
Also, the State will be persisted to the persistent store if the PersistWhenSet flag will be set to true.
Another very important aspect of this method implementation is the fact that it calls the DelayDeactivation() base method that will set a minimum amount of time before the silo will deactivate the grain.
Get(), Refresh() & Clear() Methods
Because all the above-mentioned methods are having very simple implementations, we will discuss them all at once.
As can be seen above, the first of them is simply returning the current value of the State property of the grain.
The second one is reading the state in an async way from the persistent store and basically refreshing it in memory. This implies calling a base class method ReadStateAsync().
Finally, the third one is just calling another base class method ClearStateAsync(), this setting the State property to a null value and then deactivates the grain immediately by calling DeactivateOnIdle().
DistributedCounterService
We have already mentioned that the business logic is composed of two functionalities: increment and decrement of a distributed counter. These two functionalities are triggered from and via an HTTP Web API.
Because the HTTP Web API is built using .NET, which knows nothing of Orleans grains, we need a way to interact with the Microsoft Orleans Cluster. That’s why we have created the IDistributedCounter interface and its implementation DistributedCounterService.
The interface exposes two methods that will produce the business logic. It’s worth mentioning that both operations are asynchronous and will receive as a single parameter, the key/identifier of the Counter that we will want to increment.
The implementation of this interface will use .NET Dependency Injection and will simply implement the interface above.
Please note the fact that the IClusterClient is injected via the constructor. This, as its name suggests, will be used for communicating with the Microsoft Orleans cluster.
Let’s have a look at how the IncrementAsync method is implemented in our codebase.
In the first place, we can see how we obtain a reference to the DistributedCounterGrain by using the _clusterClient property of this.
By using this grain reference, we are reading the state of the counter identified by the value of the key parameter. We are building a new instance of a simple POCO class, named DistributedCounter and set its Value property to the old counter value incremented by one or simply initialize it with value 1.
After the incrementedCounterValue is being initialized, then we will want to set the DistributedCounterGrain value to the new one and persist it to the persistent store, by using the Set() method on the grain reference, then we will return the value and exit the method.
The other method, DecrementAsync, is working exactly the same as the IncrementAsync, but decrementing the value of the old counter, instead of incrementing it.
The DistributedCounter extension
We will follow the .NET Options pattern and will create an extension class that will allow us to use the DistributedCounter feature in our services, by a simple method call:
The extension is implemented as in the following snippet:
These are all the major components of the grains library project. Let’s move forward and discuss how the Silo and the client will look like.
Microsoft Orleans Silo
For building the Microsoft Orleans Silo, we will create a simple .NET Console Application and then add to its dependencies the following Nuget packages:
- Microsoft.Extensions.Hosting
- Microsoft.Extensions.Logging
- Microsoft.Orleans.Clustering.AdoNet
- Microsoft.Orleans.Persistence.AdoNet
- Microsoft.Orleans.Server
- Orleans.AdoNet.SqlServer
- System.Data.SqlClient
Also, we will add a project reference to our DistributedCounterGrains library, presented earlier.
After that, we will edit the Program.cs file of this newly created project. We will discuss the main components of this file in the following paragraphs.
First of all, let’s have a look at the Main() method.
Here, we can see that we take the first argument from the args command-line arguments and pass it as a parameter of the RunMainAsync() method. That’s all.
Now, let’s follow the stream and see how RunMainAsync() method looks like.
The above method is simply safely trying to start the Microsoft Orleans Silo by awaiting the StartSilo() method. Please note that the siloPort is passed around as a parameter.
Please be aware of the fact that this is not a production-ready implementation of how a Microsoft Orleans Silo should be used in production.
Now, last but not least, we have the StartSilo() method, this being the place where the Silo is being configured in such a way to fit our needs.
Silo follows the builder pattern proposed by the .NET, so the way to create and configure a silo is by adding various extensions to it by using the fluent lambda configuration model. In the end, we will call the Build() method for obtaining an ISiloHost instance that will be subsequently used for calling StartAsync(). This last method is the one that will start our silo instance.
There is a lot of code above, but let’s take it step by step. First, we will use the SiloHostBuilder helper method, on which we will add the ADO.NET Clustering, by calling the UseAdoNetClustering extension method. This method needs an AdoNetClusteringSiloOptions instance, that’s initialized by a lambda method. We need to provide a database connection string and an Invariant property set to the value of StorageProviderNamespace, that is having the value:
The ADO.NET Clustering will provide us clustering on top of a database integrated into our solution by using ADO.NET.
Second, we will add another extension that will allow us to use grain storage. This will be done by calling the AddAdoNetGrainStorage extension method. Because of the fact that this is also using ADO.NET under the hood, we need to use a storage provider, this being the DistributedCounterService.OrleansDistributedCounterStorageProvider. Also, we will need to provide a connection string to the database and an invariant, too.
After, we will configure the cluster and the endpoints exposed by the Silo, by providing one ClusterOptions instance and one EndpointOptions instance.
The ClusterId is the identifier of the cluster, while the ServiceId will identify the current version of the silo. Both of them are stored somewhere in the database and are used for various operations that can be done at the cluster or service/application level.
The values of those are below:
The endpoints are configured by using the Loopback interface/localhost address, for accessing our service/application. The silo will be accessible on the value provided via the siloPort parameter, while the silo-to-client communication will be done on port 30000.
In the end, we will add some services to our silo and, also a reference to our Grain.
That’s all when it comes to how a Microsoft Orleans demo silo should be configured.
The database
The database used in this specific scenario will be SQL Server. You can install your own instance of SQL Server locally by using SQL Server Express 2022.
After installation, use Microsoft SQL Server Management Studio (SSMS) for connecting to the newly-installed SQL Server database.
Create your own custom database, called Orleans.
The ConnectionString used in our demo is the following one:
The Microsoft Orleans Client and Web API
For the client-side, we will create a .NET 6 minimal Web API, called DistributedCounterWebApi, to which we will add the following Nuget packages:
- Microsoft.Orleans.Clustering.AdoNet
- Swashbuckle.AspNetCore
- System.Data.SqlClient
Also, we need to add a project reference to the DistributedCounterGrains project library.
The most interesting part of this project is the following one, which is part of the Program.cs source file.
The above code is basically creating an IClusterClient instance that is subsequently used for connecting to the Microsoft Orleans Cluster and added as a service for being injected by the .NET DI Container into the DistributedCounterService.
We can see that for creating the IClusterClient instance we need to call UseAdoNetClustering() extension method with the same options and configure the cluster, by providing the ClusterOptions instance.
Here, after the builder for the WebApplication is created, we will call the AddOrleansDistributedCounter() extension method for using our custom-developed feature.
Conclusion
We will end here for this article since it is already too long, but we can make some conclusions:
- we learned what Microsoft Orleans is and what features it’s providing;
- we saw how to create and use Microsoft Orleans Grains;
- we created and used a Microsoft Orleans Silo and a Microsoft Orleans Client;
- we integrated a persistent store into our solution and persisted the state of our grains into it.
In the next article, we will see this demo in practice, and also we will dig deeper into some of the technicalities of the demo, like how to start one silo, how to create a cluster of silos, how to initialize the database for Microsoft Orleans and what tables are needed for having a cluster of Microsoft Orleans up and running.
In the end, a very useful mention is that Microsoft Orleans is a pretty similar platform to Hazelcast IMDG, which I’ve introduced some time ago in this article.
Don’t forget to follow me on Twitter!
