Information Theory

a Treatise on the Theory of Information

copyright (c) 1959-1999 by

Romuald Ireneus 'Scibor-Marchocki

This is my life-long research interest.

I first became interested in Information Theory in about 1955. My first opportunity to engage in research in this field occurred in 1959. And, to the extent that my employers would support or tolerate it, I have continued ever since. Now, that I have a secure -- but very small -- income, I have resumed -- self-supported -- by constructing these pages.

Is this research original? I believe it to be. However, I never had access to obscure books, magazines, journals, dissertations, etc. There might be more hidden than one expects. To me, it has been original research. Thus, I had fun performing it. As far as I am concerned, that is good enough. Even if there is some similar material hidden somewhere, by placing it here -- all together -- and in the open, a useful purpose is served.

This research spans almost four decades. Unfortunately, until now, it has not been feasible for me to publish it. It will not be easy to reconstruct it at this late date. We will have to see how successfully I will be able to do so.

Acknowledgment

Peter Anton Kleyn. (There is an umlaut over the 'y'. Thus his last name looks like "Kleijn". Often, it is misspelled thus.) We were friends since the middle 1950's. Between 1959 and 1965, he was the Project Engineer of our research in Information Theory, at the three different employers. In that position, he provided a link to management. More important, he discussed the Information Theory with me. He helped develop it.

Jerome (Jerry) Joerger. (I hope that I spelled this name correctly). He was the plant manager at each of the same three employers. Somehow, he persuaded the companies to sponsor our research.

Alan Burkow. He helped me with the research during 1960-1962. Then, for five years, in the mid 1970's, he graciously invited me to his home, once or twice a month, to discuss Information Theory.

Without these three friends of mine, the research of Information Theory would never have been possible. Or, at the very least, it would have been smaller and poorer.

Clyde Royal Sparks. He was the Vice-President of Delphi Data. He retained me as a consultant in 1980, for about a decade and a half. We became good friends. In 1980 and 1981, he sponsored my implementation of the Linear Algebra algorithms as a library of computer programs. (The hardware and system software have become obsolete. Hence this library no longer is available.) In the process, some of the theorems became better crystallized. My Linear Algebra benefited. Often, we discussed Information Theory. Some day, I may re-implement the Linear Algebra as a new library.

Dedication

Mathematics does not exist to be useful, or to have any specific applications. If any applications exist or if something proves useful, it is a bonus; but, often of little concern to the research mathematician. I had fun performing the original research. I am posting the results here -- for the first time in public. If someone stumbles upon these pages and enjoys reading them, that is enough. I would love to receive an e-mail from such a someone. I hereby dedicate this work to any reader who may enjoy reading it.

This is an on-going research. It already consists of the equivalent of about a thousand typewritten pages. Obviously, I will not be able to post it here overnight. I estimate that it will take me two to three years to catch-up with the posting of what already is available. I will work on it as time allows and the mood strikes me. If anybody shows some interest, I may be encouraged to devote more time to this exercise. An apology: My spelling never was noteworthy. There is no spelling checker here. Too bad. :-)  [On 30-VI-2004, I ran this page through the spell-checker in Microsoft's FrontPage 2003.  It found a handful of misspellings, which it corrected for me.  Huzzah!]

This treatise is written in the form of individual Web pages, each consisting of one definition or theorem, etc. The code-name of the file identifies the topic and any equation, which may be contained in the statement. If applicable, a proof follows. The equations within the proof are in a sequential order, each depending only upon the one previous equation. They do not need any backward or forward reference. Furthermore, there never is any reference from elsewhere to the interior of any proof. Hence, these equations are not numbered. There may be an optional comment or discussion following.

The Table of Contents is structured hierarchically. It consists of links to these pages. The pages contain links to all of the pre-requisite material supporting any proof.

Table of Contents

 

Overall introduction

Formal presentation of the Treatise on the Theory of Information

Nothing here, as yet. Please be patient. I am notoriously lazy. -:)