Until the mid twentieth century, the standard term for the expected utility was the moral expectation, contrasted with mathematical expectation for the expected value.
Both these concepts are mathematical expectations, but the expectation is defined with an integral for the continuous case, and a sum for the discrete case.
The formula can be derived by calculating the mathematical expectation of the "amount of information" contained in a digit from the information source.