Redis: Filter and sort your data in a SQL-like way using SORT

Redis: How to filter and sort your data such as using SQL

Redis is a database that stores your data in memory and is most often used for caching and sometimes also as a message broker.

Most of the time, it is therefore used as a simple key/value cache but it also provides structures to store for example data lists (sets), key/value hashes (hashes / hash sets), sorted sets (sorted sets) and many others.

In this article, we will mainly focus on the Set and HSet types in order to see how we could filter and sort these data, as we would do with SQL.

This is certainly not a good practice, you will in most cases have to use a relational database but it can be useful to know these mechanisms in some cases.

Type: Sets

The Sets Redis allow you to insert a set of values under a given key. For example, in the case of a list of products, we can store in Redis the following Set:

products
product:id:1
product:id:2
product:id:3
...

The Redis commands for inserting, modifying and listing the elements of a set are, among others : SADD, SREM and SMEMBERS.

So to insert our data in the set set:products:

> SADD set:products product:id:1 product:id:2 product:id:3
(integer) 1

> SREM set:product:id:2
(integer) 1

> SMEMBERS set:products
1) "product:id:1"
2) "product:id:3"

It is then easily possible to create sets containing only the identifiers of the data that must be inside the set, to classify products according to a category, for example.

In case the products with the identifiers 1 and 2 are associated to an electronic category, I can add them in a dedicated set, which will be useful later:

> SADD set:products:category:electronic product:id:1 product:id:2
(integer) 1

So far, nothing very complicated. Now let's go a little further with the HSets.

Type : HSets (Hashes)

The HSet allows to store several fields/values in the same key. So we begin to see a relationship with the columns/values in a relational database.

The Redis commands allow to insert/modify and list the fields of an hset are, among others : HSET, HGET and HGETALL.

So let's make our product table evolve with new fields : price and created_at.

On the Redis side, we would have the keys with the following field/value pairs:

key price created_at
hset:product:id:1 9.99 2021-01-17T14:00:00Z
hset:product:id:2 29.99 2021-01-17T15:00:02Z
hset:product:id:3 49.99 2021-01-17T16:00:04Z
... ... ...

Note : Contrary to a classic set, we have here 3 distinct HSets for each key. For nomenclature reasons and to be able to access the keys quickly, I always prefix them with hset:.

To create these entries in Redis:

> HSET hset:product:id:1 price 9.99 created_at 2021-01-17T14:00:00Z
(integer) 1
> HSET hset:product:id:2 price 29.99 created_at 2021-01-17T15:00:00Z
(integer) 1
> HSET hset:product:id:3 price 29.99 created_at 2021-01-17T16:00:00Z
(integer) 1

Now, to get the price field of the product with the identifier 2:

> HGET hset:product:id:2 price
"29.99"

To retrieve all fields, just use HGETALL:

> HGETALL hset:product:id:2
"price"
"29.99"
"created_at"
"2021-01-17T15:00:00Z"

Well, we now have all the elements to allow us to query our data in a very similar way to what we know with SQL. To do this, we will use the SORT command, which, contrary to what its name indicates, allows us to do much more than a simple sort.

Query the data with SORT

I invite you to go to the official documentation page of the SORT command.

This command allows to sort a set in ascending/descending alphanumeric order but also to sort the data according to a column provided in a HSet.

We can also directly give it the name of the set corresponding to our filter (category electronic for example). Let's then imagine the following SQL query:

> SELECT price
  FROM products
  WHERE category = 'electronic'
  ORDER BY created_at DESC

This request allows us to obtain the price of products in the "electronic" category, ordered by descending creation date.

With Redis's SORT function, to make this same request, you just have to do:

> SORT set:products:category:electronic
  BY hset:products:id:*->created_at
  GET hset:products:id:*->price
  DESC

"29.99"
"9.99"

Here, the * pattern will be replaced by each value provided in the Set set:products:category:electronic, in our case the product identifier.

In case you also have single keys/values in the form product:id:1 with serialized data in a particular format, you can also retrieve them by passing the GET product:id:* pattern to the SORT method.

How to make multiple filtering?

The SQL query remains quite simple in this case but it is possible to go further by making intersections or unions between several blinds. These can be done using the SINTERSTORE and SUNIONSTORE commands.

For example, if you have two sets: set:products:category:electronic and set:products:category:computer and there are products in common in both blinds, you can create a Set containing the elements in common in both blinds with:

> SINTERSTORE set:products:category:electronic-and-computer set:products:category:electronic set:products:category:computer
(integer) 1

You can then use this set as a classic set. It is also possible to generate this set on the fly (without storing it in Redis) with SINTER or SUNION.

Conclusion

Redis remains a very good tool for distributed caching, is certainly not destined to become your main database, but in some cases, it can be useful to know these manipulations in order to allow you to better exploit your data.

I hope this article has been useful, don't hesitate to contact me for any further information.