Introduction

This is the third and last of a series of posts on card counting in blackjack.  In Part 1, we started with the simplest reasonable “basic” playing strategy, in which decisions to stand, hit, double down, etc., are based solely on the player’s current hand total and the dealer’s up card.  (For example, always hit hard 16 against a dealer ten.)  Yet even using this fixed playing strategy, a player’s expected return can vary significantly from round to round, since the dealer deals multiple rounds from the same shoe before reshuffling.

In Part 2, we described how to estimate this varying expected return using the true count, calculated– in your head, at the table– as a linear combination of the probabilities of card ranks remaining in the current depleted shoe.  The true count dictates betting strategy, betting more on rounds estimated to have a favorable advantage for the player.

But we can also use the true count to vary playing strategy.  My objective in this post is two-fold.  First, I will describe the best possible playing strategy, and the maximum gain in expected return that can possibly be achieved by employing such a strategy.  Second, I will describe the more realistic indexed playing strategies that use a true count, and measure their efficiency, i.e., how close they come to realizing that best possible gain in expected return.

The perfect card counter

What does “perfect play” mean?  In the context of this discussion, I mean the playing strategy that maximizes the expected return for each round, assuming perfect knowledge of the distribution of card ranks remaining in each corresponding depleted shoe.  In other words, what if you could bring a laptop with you to the table?

Prior to each round, there are two interesting expected values to consider, that are essentially the endpoints of the spectrum of possible performance by a blackjack player.  First, at the low end, there is the expected value assuming that the player uses fixed, total-dependent basic strategy (as described in Part 1).  At the high end, there is the expected value assuming that the player instead plays perfectly, assuming knowledge of exactly how many cards of each rank remain in the current shoe.

The following figure shows the difference between these two.  That is, how much can the basic strategy player possibly expect to gain by varying playing strategy?  Each gray point represents one simulated round of play.  The x-coordinate of each point indicates the corresponding ; the y-coordinate indicates .  As before, the scatterplot is overlaid with a smoothed histogram to indicate the greater density of points near the origin.

Expected gain from using optimal strategy, vs. expected return from fixed, total-dependent basic strategy.

What is the overall per-round expected return for these two “endpoint” strategies?  The basic strategy player’s expected return is about -0.4239% (note that this value is less than the “full-shoe” expected return quoted in Part 1 due to the cut card effect), while perfect play yields an expected return of only -0.2333%.  In other words, even equipped with a laptop at the table, the house still has an advantage!  This is not as surprising as it sounds; since we are focusing on playing efficiency, we are assuming flat betting.  This merely emphasizes the point that, in shoe games, accurate betting strategy is more important than varying playing strategy.

The figure above essentially shows the “distance” between a basic strategy player and a perfect player.  The performance of any actual card counting system, no matter how simple or complex, will lie somewhere in between these two extremes.  If we define the playing efficiency of basic strategy to be zero, and the playing efficiency of perfect play to be one, then the efficiency of any other strategy is calculated using its per-round expected return according to

It now remains only to compute this expected return for some actual card counting strategies of interest, and evaluate their corresponding efficiencies.

(A word of caution: before anyone runs off quoting this as “the” formula for playing efficiency, note that these particular constants depend on all of the rule variations, number of decks, and penetration assumed at the outset of this discussion.)

True count indices

The latest additions to my blackjack analysis software allow exact evaluation of indexed playing strategies that vary based on the true count.  For example, the most common refinement of basic playing strategy is to hit hard 16 against a dealer ten… unless the true count is zero or greater, in which case you should stand.  A more complex example is soft 18 vs. dealer 2.  Basic strategy in this situation is to stand, but a more complex index strategy is to hit if the true count is less than -17, stand if it’s less than 1 (but at least -17), otherwise double down.

More generally, we can specify an arbitrarily complex indexed playing strategy as a list of “exceptions” to total-dependent basic strategy.  Each exception is identified by a tuple , where

• is the player’s hand total, with a negative value indicating a soft hand.
• is the dealer’s up card.
• is 1 if the player is allowed to double down on the hand, otherwise 0.
• is 1 if the player is allowed to split the pair hand, otherwise 0.
• is 1 if the player is allowed to surrender the hand, otherwise 0.

For each of these situations, the indexed playing strategy is given by a partition of the real line into half-open intervals of possible true counts, where each interval corresponds to a particular playing decision, encoded as 1=stand, 2=hit, 3=double down, 4=split, or 5=surrender.

For example, following are the so-called “Illustrious 18” index plays using the Hi-Lo true count.  Compare this machine-readable format with the original list generated by Cacarulo at bjmath.com.  Note how the playing decisions are interleaved with the true count indices indicating the endpoints of the corresponding intervals, with +1000 acting as “positive infinity.”

# Hi-Lo Illustrious 18 Revisited (Cacarulo)
# cnt up  dbl spl sur p1 tc1 p2 ... +1000
  +16 10   0   0   0  2   0  1      +1000
  +16 10   1   0   0  2   0  1      +1000
  +12  3   0   0   0  2  +2  1      +1000
  +12  3   1   0   0  2  +2  1      +1000
  +15 10   0   0   0  2  +4  1      +1000
  +15 10   1   0   0  2  +4  1      +1000
  +11  1   1   0   0  2  +1  3      +1000
  +12  2   0   0   0  2  +3  1      +1000
  +12  2   1   0   0  2  +3  1      +1000
   +9  2   1   0   0  2  +1  3      +1000
  +20  5   0   1   0  1  +5  4      +1000
  +20  5   1   1   0  1  +5  4      +1000
  +20  6   0   1   0  1  +4  4      +1000
  +20  6   1   1   0  1  +4  4      +1000
   +8  6   1   0   0  2  +2  3      +1000
  +16  9   0   0   0  2  +4  1      +1000
  +16  9   1   0   0  2  +4  1      +1000
  -19  6   1   0   0  1  +1  3      +1000
  +12  4   0   0   0  2   0  1      +1000
  +12  4   1   0   0  2   0  1      +1000
  -19  5   1   0   0  1  +1  3      +1000
  +13  2   0   0   0  2  -1  1      +1000
  +13  2   1   0   0  2  -1  1      +1000
  +10  1   1   0   0  2  +3  3      +1000
  +10  1   1   1   0  2  +3  3      +1000
   +8  5   1   0   0  2  +4  3      +1000
   +9  7   1   0   0  2  +3  3      +1000

In addition to the Hi-Lo Illustrious 18, I also generated full sets of indices for the Hi-Lo and Hi-Opt II counts, using CVIndex, part of the Casino Vérité suite of blackjack analysis software.

The new analysis capability that I am excited about– that motivated this series of posts– is the ability to quickly compute the exact expected return for any given subset of cards in a depleted shoe, using an indexed playing strategy as specified above.

Efficiency results

The following figures show the distribution of gain in expected return, similar to the figure above, for three different and progressively more complex card counting systems:

• Hi-Lo Illustrious 18 (as revised by Cacarulo)
• Hi-Lo with full indices
• Hi-Opt II with full indices

For easier comparison of improvement in performance, each figure has the same axis limits as the “baseline” figure above.

Hi-Lo Illustrious 18 (revisited)

Expected gain from using optimal strategy, vs. expected return using Hi-Lo Illustrious 18 indices.

Hi-Lo full indices

Expected gain from using optimal strategy, vs. expected return using full Hi-Lo indices.

Hi-Opt II full indices

Expected gain from using optimal strategy, vs. expected return using full Hi-Opt II indices.

Finally, we can compute the corresponding playing efficiencies:

• Hi-Lo Illustrious 18 has playing efficiency PE = 0.309.
• Hi-Lo with full indices has PE = 0.470.
• Hi-Opt II with full indices has PE = 0.639.

I think this analysis raises as many questions as it answers.  For example, these more accurate calculations of playing efficiency are lower than the approximations given by Griffin (see Chapter 4 in the reference below).  There are several possible reasons for the difference: is the approximation inherently biased, or is it simply due to different assumed number of decks, penetration, etc.?

References:

1. Griffin, Peter A., The Theory of Blackjack, 6th ed. Las Vegas: Huntington Press, 1999.

This is a detour from the last couple of posts; I will pick up the blackjack thread again later.  I was distracted recently by a desire to “unprotect” a password-protected Excel spreadsheet in order to view some hidden cell contents.  It turns out this is not a new problem, but it is an interesting exercise, with an opportunity to improve on what seems to be the “standard” solution.

OpenOffice.org (PDF, see Section 4.18.4) provides a description of the password protection algorithm, which is not very strong.  (This is not surprising, since it’s not intended to prevent disclosure.  As the help documentation points out, it’s really just a safety net for editing, “to help prevent others from changing, moving, or deleting important data.”)  When you enter a password to protect a worksheet, Excel does not actually store the password itself, only a 16-bit hash of the password.  When you subsequently enter the password to attempt to unprotect the worksheet, Excel simply compares the hash of the password you just entered with the stored hash value.

There are about 10^30 different possible passwords with a maximum of 15 characters.  But there are only 32,768 different possible hash values, so collisions should be easy to come by.  That is, we don’t have to recover the original password to unprotect the worksheet, we only have to enter a password with the same hash value.  We can brute-force attack the problem by trying passwords with all possible hash values.

[Edit: Unfortunately, it looks like Microsoft has gotten around to plugging this particular hole in the most recent versions of Excel (2016 where I have tested), now using SHA-512 instead of this home-grown hashing scheme.  As a result, this brute-force approach is no longer feasible.]

The hash function is simple to describe: compute the bit-wise exclusive-OR of (1) the constant 0xCE4B, (2) the password length, and (3) the ASCII value of each character of the password, as a 15-bit value rotated left bit-wise by its position.  That is, rotate the first character 1 bit left, the second character 2 bits left, etc.  The following Python code implements this function:

def excel_hash(password):
    """Return hash of given string, in the range [0x8000, 0xffff]."""
    h = 0
    for ch in password[::-1]:
        h = h ^ ord(ch)
        h = ((h << 1) & 0x7fff) | (h >> 14)
    return h ^ len(password) ^ 0xce4b

There is a solution that has been posted and re-posted many times, that essentially tries all 12-character passwords where each of the first 11 characters are A or B (and the 12th is any printable ASCII).  This is guaranteed to yield all 32,768 possible hash values.  It works.  So that should be the end of it… except I’m not sure why this particular enumeration of passwords was chosen, since it isn’t clear to me that it “covers” all possible hash values in any structured way.

First, it is more exhaustive than it needs to be.  That is, I thought perhaps this approach might have been the minimal result of a simple blind empirical search, trying longer and longer passwords of the form “[A|B]*.” (with the last printable character to “fill in the gaps”) until all possible hash values were covered.  But this isn’t the case, since we can make the solution 4 times faster by using just 10-character passwords– i.e., where the first 9 are A or B, and the 10th is any printable character.

Second, why use the characters A and B?  The ASCII values for A and B are 65 and 66, respectively, meaning that we are toggling two consecutive bits with each character, not just one, and those pairs of changing bits for consecutive password characters overlap in position, which doesn’t seem helpful.

My first thought was, “The dumbest thing that could possibly work is to use 15-character passwords, where each character is, say, either B or C.”  This way, each character toggles exactly one distinct bit in the hash, so that we try exactly one password for each possible hash value.  And indeed, this approach does work, and is about 6 times faster than the commonly-cited solution.

But instead of a 15-level nested for-loop, we can shorten the code somewhat by just varying three characters in the password, with each character exploring 5 bits of the hash.  The only trick is (1) keeping the characters’ ASCII values in the printable range, and (2) “rotating” those three characters so that their blocks of 5 bits do not overlap, by inserting constant password characters between them.

The result is the following code, with instructions on how to use it:

1. Open the worksheet that you want to unprotect.
2. Press Alt-F11 to open Visual Basic.
3. Press F7 to open an editor window.
4. Copy/paste the code below into the editor window.

Sub FindPassword()
    Dim c1 As Integer, c2 As Integer, c3 As Integer
    Dim passwd As String
    On Error Resume Next
    For c1 = 64 To 95
        For c2 = 64 To 95
            For c3 = 64 To 95
                passwd = Chr(c1) & "...." & Chr(c2) & "...." & Chr(c3)
                ActiveSheet.Unprotect passwd
                If ActiveSheet.ProtectContents = False Then
                    MsgBox "Unprotected using password: '" & passwd & "'"
                    Exit Sub
                End If
            Next
        Next
    Next
End Sub

5. Finally, press F5 to run.  A dialog is displayed indicating a valid password.  The worksheet is now unprotected.