Scientific computing, math & more

Posts 1 - 10 of 38

Results for: Eigenvalues

Experiments with Variable Format Half Precision

For the past month I have been working with the variable format 16-bit floating point arithmetic that I describedin this post. It has been frustrating work. I have found that the limited precision and limited range of half precision make it barely usable for the kind of experiments with matrix computation that I like to do. In this post I will describe a few of these experiments....read more >>

Posts 1 - 10 of 38