The Problem with a Fair Coin

It has been a while since I have posted. Last time, I was working on Elements of Information Theory before I had to return it to the library.

I didn't get very far (and it has been nearly a month since I returned it).

My main approach with technical book is to implement solutions to the end of chapter problems in software. The first question in this book is as follows:

A fair coin is flipped until the first head occurs. Let X denote the number of flips required. Find the entropy H(X) in bits.

This is a fairly easy problem to solve (especially with the series summation provided with the question). However, the solution has a very high barrier in software.

To write a general software solution, I would need the following:
1) Representation of sums of infinite series.
2) Ability to convert a random variable description into the infinite series.
3) Ability to manipulate the infinite series until it matches a function with a known closed form solution.

Once I've gotten to that point, the use of the closed form solution isn't very useful.

I tend to get stuck chasing rabbit holes like this. I didn't make much progress after I got stuck in this tar pit.

The book is back at the library now and I'm going to head back to my vision work.