Redis BitField Explained: Principles, Commands, and Practice

· 6 min read ·
#Redis
Read the Chinese version

Redis offers a wide range of convenient and useful data structures, which is one of the key reasons it became so popular as a caching middleware.

The most common ones are String, Hash, List, and Set — pretty much everyone who has studied Redis knows them. Beyond those, there are some less common but rather interesting structures such as BitMap, HyperLogLog, and BloomFilter, which can produce surprisingly good results in the right scenarios.

Today let’s look at a fairly special one: BitField.

Starting with BitMap

Before introducing BitField, I’d guess BitMap is the better-known structure. It uses a single binary bit — 0 or 1 — to represent a state, which enables SET, GET, and counting operations at the bit level.

A typical use case is tracking monthly check-in data with a BitMap: SET the bit corresponding to each day the user checked in, then use the BITCOUNT command to get the number of bits set to 1, which gives you the total number of check-in days.

Once you understand where BitMap fits, it’s natural to take the idea a step further:

  • What if the value I need to store has more than just the two states 0 and 1? For example, a user’s age (0-127).
  • What if I want to store several numeric values for a user? For example, level (0-15), age (0-127), and check-in days this month (0-31).

There are plenty of answers. To store a single numeric value, you could still use a BitMap and work it out with bitwise operations, or simply use a String. To store multiple numeric values, you could again handle the bit arithmetic yourself, or take the easier route and use a List or a Hash.

Beyond these obvious answers, there’s one more option: Redis’s BitField.

If BitMap gives you control at the granularity of a single bit, BitField gives you control at the granularity of a group of bits.

Commands and Core Concepts

The commonly used BitField commands look like this:

# GET
BITFIELD key GET encoding offset
# SET
BITFIELD key SET encoding offset value
# INCRBY
BITFIELD key INCRBY encoding offset increment

These subcommands can also be freely combined into a single long command, and Redis executes the whole thing atomically.

As you can see, the BITFIELD command involves two concepts that may be unfamiliar: encoding and offset.

encoding

The encoding is the numeric type used by that group of bits. There are two forms, uN and iN:

  • uN: unsigned, N bits. For example, u4 means an unsigned 4-bit value, ranging from 0000 to 1111, i.e. 0-15.
  • iN: signed, N bits. For example, i8 means a signed 8-bit value, ranging from -128 to 127.

Note that unsigned encodings go up to u63, while signed encodings go up to i64.

offset

The offset is the bit offset. Say you store variable 1 in 4 bits and variable 2 in the next 4 bits: an offset of 0 addresses variable 1, and an offset of 4 addresses variable 2.

There’s also a special form: if every value you store has the same size, you can write the offset as #N to address the Nth value directly (counting from 0). In that case the actual offset equals N × the width of the encoding.

Example 1: Packing User Info

Let’s design the user data we mentioned earlier:

FieldEncodingRangeoffset
Levelu40-150
Ageu70-1274
Check-in daysu50-3111
bits:  xxxx | xxxxxxx | xxxxx
        u4      u7       u5

With just 16 bits, we can store a complete user record.

Inserting a user record with BitField looks like this:

# Set data for user 0001: level 0, age 33, 10 check-in days
BITFIELD user:0001 SET u4 0 0 SET u7 4 33 SET u5 11 10

And incrementing user 0001’s level by 1 looks like this:

BITFIELD user:0001 INCRBY u4 0 1

Example 2: Storing Student Scores in Bulk

Here’s another example. Suppose we want to store the scores (0-100) of 100 students; u7 is more than enough for each score. Since every field has the same size, this is exactly where #N comes in handy:

bits:   xxxxxxx | xxxxxxx | xxxxxxx | ...
          #0        #1        #2

Setting the 59th student’s score (indexed from 0) to 60:

BITFIELD class:0001 SET u7 #59 60

These two examples show that when you’re dealing with a group of numeric values, BitField saves a great deal of space and gives you handy commands for operating directly on a group of bits.

Overflow Handling

BitField has another rather special mechanism: overflow handling. The relevant command looks like this:

BITFIELD key OVERFLOW WRAP|SAT|FAIL

BitField controls a group of bits under a given encoding, so what happens when a value goes out of range? You can choose between different overflow behaviors:

  • WRAP (the default): wrap around. The maximum value plus 1 becomes the minimum value, and the minimum value minus 1 becomes the maximum. This matches how integer overflow behaves in C.
  • SAT: saturate. The maximum value plus 1 stays at the maximum, and the minimum value minus 1 stays at the minimum.
  • FAIL: fail. In this case the command does not succeed, and the return value is NULL (nil).

One more thing worth noting: an OVERFLOW applies to the SET and INCRBY subcommands that follow it, up until the next OVERFLOW.

Conclusion

That covers how BitField works in Redis.

Overall, whenever you need to store and manipulate a group of numeric values with known and identical sizes, BitField is well worth considering as a way to save space.

That said, readability and maintainability matter too. If bitwise concepts still feel shaky to you and memory usage isn’t a concern, there’s nothing wrong with reaching for a Hash instead. Never use a technology for its own sake — BitField only becomes a real advantage when the scenario actually calls for it.