and
[I am providing this compilation of those theorems and algorithms of Linear Algebra which I have found interesting and useful. Some of them are obscure; mayhap even original with me. This exposition also provides documentation for the library of Linear Algebra programs that I am writing (from scratch) concurrently in the C-language, which recently became accessible to me . Since I am writing these "closed-book" -- it has been more than a decade since I have seen a book on Linear Algebra --, I may be omitting various popular topics. :-)
May you, the reader, find this interesting. Enjoy!
Suggestions, comments, criticisms, corrections are welcome. Specifically, are there any theorems or algorithms that you would like to see included?]
The shareware library of Linear Algebra subroutines provides for either real or complex matrices. To facilitate a sneak peek, a link is provided to each corresponding helper *.h file.
00 Structures and global preferences.
01 Matrix storage allocation & deallocation; complex & partitioned; and transpose, addition, subtraction, & multiplication.
02 Frobenious norm, specialized routines for diagonal matrices, and the check_isit family of subroutines, which include: zero, identity, diagonal, ortho-normal, skew-symmetric, symmetric, inverse, equal, or complex?
03 Matrix inversion: LDU with optional pivoting, Gauss-Jordan with optional pivoting, and twice to improve precision of the inverse of an ill-conditioned matrix.
04 Specialized multiplication routines. The bi-linear transformation of a matrix.
05 Eigenvalue, eigenvector, etc. for a symmetric matrix. That is, factorization of a symmetric matrix S into an ortho-normal matrix O and a diagonal matrix E of eigenvalues: S = O E Otr. (The “Otr” is so written; because, that is the best orthography available lacking the glyph of a super-scripted dagger -- per the usual convention. This expression means the transpose of the matrix O.) At present, only the Jacobi algorithm is implemented.
06 Gramm-Schmidt ortho-normalization algorithm. Raleigh, Gaussian, Poisson, and other pseudo-random.
07 Factorization of a non-symmetric (that is, general) matrix A into a bra < diagonal D and a ket > matrix: A = < D >. Matrix inversion therefrom. (P. A. M. Dirac named this diagonal matrix "spectrum". However, I employ the name "diagonal"; because, the first letter S already has been assigned as an abbreviation for "symmetric".)
08 Factor analysis and multi-dimensional linear-regression. Factorization of a channel into an exterior bra, exterior diagonal, interior bra, interior diagonal, the transpose of the interior bra, exterior diagonal, and the transpose of the exterior bra. The small-style matrices: left bra, left diagonal, middle bra, middle diagonal, middle ket, right diagonal, and right ket also are provided. Channel rate, angle, and effective correlation coefficient. Cascaded channels.
09 I/O routines for writing and reading of an event data-set and of matrices. Also computation of the variance and average of an event data-set.
Sample programs (matrix_m03.cpp and matrix_m04.cpp) for the testing and employment of this library.
For pedagogical reasons, we attempt to keep related topics together, at the price, however, of three forward references, which we
indicate clearly. Hopefully, we have not committed a circular argument, thereby.
Links to the individual chapters of this online textbook follow: <Those topics marked with an asterisk are included in a
typical textbook.>
Ch 0. the Natural to Complex Numbers is a prerequisite; hence, explicit hyperlinks to it will be scarce.
Ch 1. the Division Ring Operations *
determinant of a matrix *
Gramm-Schmidt ortho-normalization algorithm *
Ch 2. Symmetric *
Ch 3, algorithms for the factorization of a symmetric matrix
multiplication family of algorithms
algorithm Jacobi
field of symmetric matrices which share a common orth-normal factor
Ch 4. Real
factorization of a matrix into a symmetric matrix and an orth-normal matrix
null space and various uniqueness theorems *
the determinant and the trace of a matrix each is invariant under a generalized similarity transformation *
Ch 5. algorithms for inversion of a real matrix
The LDU algorithm *
The QR algorithm *
The Gauss-Jordan algorithm *
Inversion of an ill-conditioned matrix
basis *
Ch 6. Complex
Ch 9. interpolation, extrapolation, and Fourier series
Vandermonde matrix and its determinant; elementary symmetric functions
the axis of rotation of an ortho-normal or unitary matrix
Linear Algebra aspect of the three channels in cascade problem
Ch 10. degrees of freedom, including a discussion of the uniqueness of the ket.
Ap B. data formats
Linear Algebra has its beginnings in China. They already were solving simultaneous linear algebraic equations before written history. We provide a brief listing of some typical applications.
The solution of a set of simultaneous linear algebraic equations.
factorization of a matrix A into A = < D >
factorization of a symmetric matrix S into S = O E Otr (As stated previously, Otr designates the transpose of the matrix O.)
two-dimensional:
The bending of a beam depends upon the second-order moment-matrix of its cross-section.
three-dimensional:
The dynamics of a rigid body (in a vacuum) depends upon the second-order moment-matrix of its density.
The dynamic balancing of an automotive wheel (including its tire).
The rotation of an asteroid or comet.
multi-dimensional:
Factor analysis in Statistics.
The concept of intelligence -- as in intelligence-quotient -- is the first dimension. The design and validation of intelligence tests.
The Myers-Briggs personality concept and the design of the tests. The first four factors (that is, dimensions) have been named introversion-extroversion, intuitiveness-sensing, thinking-feeling, and perceiving-judging. Here is a link to an interesting contemporary utilization of this concept.
factorization of a general real matrix A into A = < D >
Will be very valuable in the Banach space.
factorization of a Hermitian matrix H into H = U D Utr (The theorems/algorithms are described in factorization of a symmetric matrix and complex):
factorization of a general complex matrix A into A = < D >
None as yet; because, it is only with this posting of the theorems and algorithms that such a factorization has become possible.
Will be very valuable in Quantum Mechanics and in the Hilbert space.
the Jordan factorization of a general complex matrix A. The Jordan factorization of a general real (or complex) matrix A into A = M J Minv, where M is a non-degenerate matrix and J is a Jordan normal form matrix.
The form (A - x B) -- where each of the pair of matrices A and B is a positive-definite symmetric matrix -- requires the Jordan factorization of the quotient matrix (Binv A).
Classical Mechanics -- both Statics and Dynamics -- end Electricity encounter this form, whose eigen-values and corresponding eigen-vectors are desired.
I might be willing to collaborate on developing interesting applications.
[I began my study/research of Linear Algebra in the early Summer of 1960. I did not realize that I was obtaining original results. Around 1975, I purchased the Texas Instruments TI-49 programmable calculator, on which I investigated fragments of some of my algorithms. In April 1980, I was retained to write a library of Linear Algebra routines, to be written in a proprietary dialect of Fortran IV, to run on the Data General Mainframe computer. Around 1983, for the third time, I implemented my algorithms, this time, written in the APL (= A Programming Language), on an already obsolete desktop IBM computer.
In early 1981, the Mathematical Dictionary was published. For the first time, I could compare my results with the common knowledge. I was further advanced. However, even then, there was no opportunity for publishing. Hence, I never wrote anything down.
In 1999, I wrote a sequence of technical letters, on the subject of Linear Algebra, to my very good friend, Deb. These letters were based upon my notoriously fallible memory. In 2000, I posted a copy of these letters on my Web site. Even in this rough form, the information therein has been useful to some individuals who have written to me.
Finally, beginning in mid December 2003, an opportunity presented itself for me to begin writing my textbook on Linear Algebra, which I posted (right here -- you are reading it now) on my Web site in mid June 2004. This time, I worked out the steps systematically. Hopefully, the result is more reliable. I also wrote and posted what is my fourth implementation of the algorithms, this time written in the C language.
There are several reasons why I am retaining the original letters: Nostalgia, historical, and to honor Deb. The presentation is different and more detailed. Also, some results have not been incorporated here as yet.
However, in these letters there are some known errors which I leave to the readers to discover for themselves.]
Acknowledgement: My former student and very good friend, Mr. Chaumont "Joe" Devin is proof-reading this exposition. His valuable suggestions are most appreciated.
Copyright (c) 2003, 4, 6 by R.I. ‘Scibor-Marchocki. Last revised Thursday 16-th December 2004. mailto:webmaster@rism.com Broken hyperlink corrected Friday 18-th September 2005. Chapter numbering introduced Wednesday 21-st September 2005. minor addition Saturday 04-th February, 2006 another on Monday 06-th February 2006. Last revised Wednesday 01-st March 2006.