Redis: Filter and sort your data in a SQL-like way using SORT
Redis is a database that stores your data in memory and is most often used for caching and sometimes also as a message broker.
Most of the time, it is therefore used as a simple key/value cache but it also provides structures to store for example data lists (sets), key/value hashes (hashes / hash sets), sorted sets (sorted sets) and many others.
In this article, we will mainly focus on the Set
and HSet
types in order to see how we could filter and sort these data, as we would do with SQL.
This is certainly not a good practice, you will in most cases have to use a relational database but it can be useful to know these mechanisms in some cases.
Type: Sets
The Sets
Redis allow you to insert a set of values under a given key. For example, in the case of a list of products, we can store in Redis the following Set
:
products |
---|
product:id:1 |
product:id:2 |
product:id:3 |
... |
The Redis commands for inserting, modifying and listing the elements of a set are, among others : SADD, SREM and SMEMBERS.
So to insert our data in the set set:products
:
> SADD set:products product:id:1 product:id:2 product:id:3
(integer) 1
> SREM set:product:id:2
(integer) 1
> SMEMBERS set:products
1) "product:id:1"
2) "product:id:3"
It is then easily possible to create sets containing only the identifiers of the data that must be inside the set, to classify products according to a category, for example.
In case the products with the identifiers 1
and 2
are associated to an electronic
category, I can add them in a dedicated set, which will be useful later:
> SADD set:products:category:electronic product:id:1 product:id:2
(integer) 1
So far, nothing very complicated. Now let's go a little further with the HSets
.
Type : HSets (Hashes)
The HSet
allows to store several fields/values in the same key. So we begin to see a relationship with the columns/values in a relational database.
The Redis commands allow to insert/modify and list the fields of an hset are, among others : HSET, HGET and HGETALL.
So let's make our product table evolve with new fields : price
and created_at
.
On the Redis side, we would have the keys with the following field/value pairs:
key | price | created_at |
---|---|---|
hset:product:id:1 | 9.99 | 2021-01-17T14:00:00Z |
hset:product:id:2 | 29.99 | 2021-01-17T15:00:02Z |
hset:product:id:3 | 49.99 | 2021-01-17T16:00:04Z |
... | ... | ... |
Note : Contrary to a classic set, we have here 3 distinct HSets for each key. For nomenclature reasons and to be able to access the keys quickly, I always prefix them with hset:
.
To create these entries in Redis:
> HSET hset:product:id:1 price 9.99 created_at 2021-01-17T14:00:00Z
(integer) 1
> HSET hset:product:id:2 price 29.99 created_at 2021-01-17T15:00:00Z
(integer) 1
> HSET hset:product:id:3 price 29.99 created_at 2021-01-17T16:00:00Z
(integer) 1
Now, to get the price
field of the product with the identifier 2
:
> HGET hset:product:id:2 price
"29.99"
To retrieve all fields, just use HGETALL
:
> HGETALL hset:product:id:2
"price"
"29.99"
"created_at"
"2021-01-17T15:00:00Z"
Well, we now have all the elements to allow us to query our data in a very similar way to what we know with SQL. To do this, we will use the SORT
command, which, contrary to what its name indicates, allows us to do much more than a simple sort.
Query the data with SORT
I invite you to go to the official documentation page of the SORT command.
This command allows to sort a set in ascending/descending alphanumeric order but also to sort the data according to a column provided in a HSet
.
We can also directly give it the name of the set corresponding to our filter (category electronic
for example). Let's then imagine the following SQL query:
> SELECT price
FROM products
WHERE category = 'electronic'
ORDER BY created_at DESC
This request allows us to obtain the price of products in the "electronic" category, ordered by descending creation date.
With Redis's SORT
function, to make this same request, you just have to do:
> SORT set:products:category:electronic
BY hset:products:id:*->created_at
GET hset:products:id:*->price
DESC
"29.99"
"9.99"
Here, the *
pattern will be replaced by each value provided in the Set set:products:category:electronic
, in our case the product identifier.
In case you also have single keys/values in the form product:id:1
with serialized data in a particular format, you can also retrieve them by passing the GET product:id:*
pattern to the SORT method.
How to make multiple filtering?
The SQL query remains quite simple in this case but it is possible to go further by making intersections or unions between several blinds. These can be done using the SINTERSTORE and SUNIONSTORE commands.
For example, if you have two sets: set:products:category:electronic
and set:products:category:computer
and there are products in common in both blinds, you can create a Set containing the elements in common in both blinds with:
> SINTERSTORE set:products:category:electronic-and-computer set:products:category:electronic set:products:category:computer
(integer) 1
You can then use this set as a classic set. It is also possible to generate this set on the fly (without storing it in Redis) with SINTER or SUNION.
Conclusion
Redis remains a very good tool for distributed caching, is certainly not destined to become your main database, but in some cases, it can be useful to know these manipulations in order to allow you to better exploit your data.
I hope this article has been useful, don't hesitate to contact me for any further information.