Authors: Ondřej Čertík, Brian Beckman
In this blog post I am announcing fastGPT, fast GPT-2 inference written in Fortran. In it, I show
Fortran has speed at least as good as default PyTorch on Apple M1 Max.
Fortran code has statically typed arrays, making maintenance of the code easier than with Python
It seems that the bottleneck algorithm in GPT-2 inference is matrix-matrix multiplication. For physicists like us, matrix-matrix multiplication is very familiar, unlike other aspects of AI and ML. Finding this familiar ground inspired us to approach GPT-2 like any other numerical computing problem.
In December I left Los Alamos National Laboratory (LANL) to join GSI Technology as a compiler developer.
I have spent about 8.5 years at LANL. Physics and computing have always been my passions, so I got a Master in theoretical physics in Czechia, and then Ph.D. in chemical physics in the U.S. LANL is a great place for computational physics. I met and worked with many amazing people. It has been intellectually very satisfying to work on the projects that I was able to help with and things to learn. LANL has famous beginnings during the Manhattan Project, and it has continuously evolved since then. For somebody who was born behind the Iron Curtain, it’s been an honor to work there. If you like physics or scientific computing, I can only recommend applying there.
We have finished the first chapter on a long journey of resurrecting Fortran: forming an initial core community of developers and users with enough momentum around the new Fortran website and projects at fortran-lang.org. There is still have a long way to go to, but this was a necessary (although not sufficient) achievement and I feel this was a once in a life time event. The timing was right, we were ready, and we also got lucky. In this blog post I have tried to write up the details about the pivotal moments from my perspective. I have asked Milan Curcic who has been a co-founder of this effort to do the same (please see his blog post), more on that below. Brad Richardson has also written about his perspective.
I decided to run for WG5 Convenor (International Fortran Committee Chair). I have written up a Platform document that explains why I am running, the issues as I see them, and how I propose to work with the Committee to address them (see also my CV). I have developed this platform with many Fortran community members, both inside and outside of the Fortran Committee. Furthermore, I have also posted the platform as a GitHub repository and I encourage anyone to open issues and Pull Requests (PRs) to discuss and further refine the platform.
My webpage is generated using Hugo and it uses the hugo-resume template.
I just enabled a blog section here, so that I can post blog posts here from now on. My old blog is still hosted at https://ondrejcertik.blogspot.com/, but I do not update it anymore.