Before we can create a container, we need to define the policy. A policy defines which storage nodes on NeoFS you are happy to store your data on.
The approach to do this is to use the simple policy language provided. A policy is made up of 4 components
Filters are used to decide which nodes meet the requirements to store an object. Think of it as 'filtering' out nodes based on some condition. All nodes have attributes that can be used to filter their capabilities and whether they meet the requirements for the policy.
A filter can check for instance if a node is situated in Europe, and the data would be stored on SSDs (solid state drives).
A filter contains
- The name of the attribute to filter
- The value to compare the attribute with
- The way in which to make the comparison (less than, greater then, equal to etc)
|NE||not equal to|
|GE||greater than or equal to|
|LE||less than or equal to|
A filter could therefore be
Country = Italy
Rating > 3.5
Filters can also have names so they can be grouped into 'meanings', for instance
Country = Finland OR Country = Iceland AS ColdCountry
Now you can refer to Finland and Iceland as
ColdCountry in further/more complex filters such as
FILTER Country EQ "Finland" OR "Iceland" AS ColdCountryFILTER ColdCountry AND StorageType = SSD AND Price < 100FILTER SSD EQ true AS IsSSDFILTER @IsSSD AND Country EQ "RU" AND City EQ "St.Petersburg" AS SPBSSDFILTER 'Continent' EQ 'North America' OR Continent EQ 'South America' AS Americas
Note country codes come from the open flight database. NeoFS uses the code found here to manage these codes
Once a filter has found some nodes that adhere to the requirements, selectors can now be specified to decide which nodes end up in the container policy.
Selectors are slightly easier to understand as part of examples, but the principles of selectors are outlined here
A selector contains
- a name that can be referred to
- the name of the attribute that will be used to group nodes together. The nodes will be grouped into buckets based on the value of the attribute, or you however create 'random' groups of nodes.
- The number of nodes that should be included in a bucket
- The clause specifies how to interpret the count e.g
- SAME - choose nodes that are part of the same bucket
- DISTINCT - choose nodes that are from distinct buckets
#Select + Filter
Its easiest to understand a placement policy when considering this combination defines minimal suitable container.
FILTER specifies a subset of suitable nodes and
SELECT picks exact nodes from this subset. If network map does not have enough nodes for a minimal suitable container, an error will be produced.
Some examples of
This one is simpler, select 6 nodes from any location.
SELECT 6 FROM *
This example states the policy should select a single node that is located in St Petersburg, Russia
SELECT 1 FROM SPBFILTER Country EQ "RU" AND City EQ "St.Petersburg" AS SPB
(note this isn't strictly accurate but works for the example - read about Container Backup Factors)
Replicas define how many copies of objects should be stored within a Replica. A replica is a group of nodes where a single copy of an object is stored.
With replicas you can decide how much redundancy you would like when it comes to a container's contents and making sure that the data isn't lost
#Container backup Factor (CBF)
The container backup factor defines the maximum number of nodes that should be included within a container's set of nodes.
You could have a placement policy that specifies there should be
10 nodes within 2 countries with a Container Backup Factor of
In this case the CBF will attempt to have 20 nodes in 2 countries, but anything from 10-20 is good and better, but not a failure.
Note, although we have outlined CBF as a concept last here, it should come second, after
|When no select or filter is defined, the default will be just to "replicate 3 times selecting from anywhere, i.e |
|As above, however CBF has been set to 4, so this will select between 3 and 12 nodes to replicate data over|
|1 replica in every city|
|Replicate objects twice, selecting minimum of 6 nodes from anywhere. With CBF it will select up to 18 nodes|
When making policies, it should be defined in the order
REP, CBF, SELECT, FILTER
const placementPolicy = `REP 2 IN XCBF 2SELECT 2 FROM * AS X`
- Spec https://nspcc.ru/upload/neofs-spec-latest.pdf#a
- Tests/Examples https://github.com/nspcc-dev/neofs-sdk-go/blob/master/policy/query_test.go
- JSON format of policies (can help seeing complexity) https://github.com/nspcc-dev/neofs-sdk-go/blob/master/netmap/json_tests/filter_simple.json