Resurrecting Fortran
March 12, 2021We have finished the first chapter on a long journey of resurrecting Fortran: forming an initial core community of developers and users with enough momentum around the new Fortran website and projects at fortran-lang.org. There is still have a long way to go to, but this was a necessary (although not sufficient) achievement and I feel this was a once in a life time event. The timing was right, we were ready, and we also got lucky. In this blog post I have tried to write up the details about the pivotal moments from my perspective. I have asked Milan Curcic who has been a co-founder of this effort to do the same (please see his blog post), more on that below. Brad Richardson has also written about his perspective.
This has been the most unusual experience for me, nothing like what I anticipated, but two years ago things seemed to fall into place and set up just right. I had to give it everything I had in order to give it a proper chance. I took the bull by the horns.
This blog post is long, but you might find it useful to get ideas how to form other such communities.
Background
I have been using Fortran on and off for a long time, but only started using it as my main language around 2010 thanks to John Pask, my advisor at LLNL at a time, who introduced me to modern Fortran and I saw that it was a superior choice for high performance numerical code than Python - my main choice until that time. I immediately realized there is not much information about modern Fortran online, so I started the fortran90.org website in 2012 to collect information about the recommended modern ways to use Fortran with the hope that it could become the website for Fortran. Over the years it received contributions from about 11 people and it is moderately popular, but overall it failed to create a well agreed upon website and a community for Fortran. fortran-utils, was another idea, which got 5 contributors and 142 stars at GitHub over the years. Not bad, but no way close to creating a thriving community. I thought perhaps there just wasn’t enough interest in Fortran, just a few enthusiasts here and there, not enough to build a community.
LFortran
Fast forward to late 2017. I got a thought: how could Fortran be interactive as Python? I thought a lot of the issues with Fortran could be fixed with a better, interactive compiler. I got to work. This eventually became LFortran, but for the time, I worked on it privately.
After a year of work, I decided to join the Fortran Standards Committee, which I didn’t know much about at the time. I knew it existed, and I wanted to know if it is still active, and if we can join forces somehow. The first meeting I was able to attend was in February 2019. I showed LFortran to the committee and there was a lot of interest. That prompted me to get it to a point where it could be announced publicly. This event occurred in April 2019 (which was exciting for me), you can read the announcement blog post. It generated tremendous interest (over 3000 people read it in the first few days). It became obvious that people wanted to hear more about why we thought Fortran was a great choice. We explained that in a follow up blog post.
This interest in Fortran and LFortran was unusual. The response to LFortran lead me to believe I was creating something useful. Almost anybody I talked to, even if they were not fans of Fortran initially, were interested after hearing about the project. I showed individuals how LFortran works, and how it is interactive and works in a Jupyter notebook just like Python or Julia. I could feel that opinions about Fortran are slowly changing. I was hoping that we could build a community around LFortran to resurrect Fortran. I have communicated with dozens and dozens of people, online and offline, publicly and privately. But so far people were still waiting, keeping an eye on LFortran, but not ready to join the effort yet.
Incubator Repository
My next Fortran Standards Committee meeting was in October 2019. Zach Jibben joined me as another
representative for LANL. The meeting started on Monday.
On Tuesday night I was in my hotel room looking from a window at Las Vegas at
night, and I got the idea that we should track the committee proposals. First I
thought to just open issues at LFortran’s GitLab repository. I called Zach and asked
his opinion on the idea. We decided to dedicate a repository to it, so I created a
fortran_proposals repository at LFortran’s GitHub organization (I chose GitHub
to get more participation, as it has more users than GitLab). We started adding
issues for ideas that would be worth discussing to include in Fortran.
In order to draw more attention to the idea, I posted to ask on Twitter “I am at the committee meeting, if anyone has any wishes that I am happy to represent it”. There were several excellent suggestions, such as the very first one from Michael Zingale that eventually became the first issue #1. It was at this time that Milan got more involved.
The next day at the committee meeting, my main job was to sell the idea to the other committee members, so they could embrace the repository. Gary Klimowicz has been tremendously helpful convicing others. The first thing he said we needed was to move it to its own organization, so we moved it to https://github.com/j3-fortran/fortran_proposals. With Gary’s help we have convinced several members of the U.S. Standards Committee to participate. The first week was critical, here are all the people that contributed to issues:
After two weeks the most active 12 people:
And by the end of February 2020 the most active 12 people:
This succession of stats show how the community was growing in the initial stages.
I have spent tremendous effort in this timeframe to nurture this interest and to get people involved. I replied to every new contributor, and directed the discussion to be the most productive. I haven’t done much work on LFortran in this timeframe, but I felt this was the opportunity to build an active online Fortran community so I gave it my best.
Stdlib
Milan suggested to start stdlib on November 28, 2019. We got a “blessing” for this effort from Steve Lionel. I asked Milan if he would be interested in leading this effort, he accepted. Things were moving fast!
Milan organized the community around stdlib. We started a GitHub organization
fortran-lang. I spent a lot of effort providing initial code and discussions
in stdlib also. Here are the first 12 contributors in the first month and a
half:
My approach with these large initial contributions is to provide enough initial momentum so that the snowball eventually starts rolling on its own, whether or not I contribute. And we succeeded, for example in October and November 2020 my own contributions are negligible and others have contributed an order of magnitude more (the first 12 people):
It works!
Fortran Package Manager
As soon as we started developing LFortran, it became obvious that we will need
a package ecosystem, so I created an
issue for it on May 22,
2019 with what I believe the goals should be. On October 27, 2019 when we had
our J3 Incubator Repository, I created an
issue there with
the same goals. There it generated a lot more discussion. On December 23, 2019
Dominik Gronkiewicz opened a new
issue at stdlib to try to
create an experimental packaging ecosystem. It got even more interest. More
crucially, Milan
wrote
“I’m in”, which was enough momentum that we created a new
repository for the
Fortran Package Manager (fpm).
I initially created a prototype in Rust, but I got stuck on a weird issue on macOS where a CI test would randomly fail. I spent dozens and dozens of hours on that, asked at the Rust Discourse, the Rust community has been very helpful. Eventually we figured out a workaround, but it took a lot of time and effort. I did not have to spend the time on this, but I am the kind of guy who expects things to work robustly on all platforms at all times and I cannot sleep well knowing that a Pull Request CI tests can randomly fail at any time. As a consequence of this, we did not get very far. But it confirmed my suspicion that Cargo is the best package manager I know of, the state-of-the-art. And that we want to implement something similar for Fortran.
January 2020 turned into February and there was a Standards Committee Meeting.
One of the highlights for me was that Brad Richardson agreed to join us to lead
the fpm effort. He already had a prototype in Haskell, so we made a gentleman’s
agreement that we will switch over to his code to finish the prototype, and
then we will re-evaluate.
Milan’s and Brad’s posts have more details what happened next.
By now, I consider fpm the most successful and exciting project that came out
of our Fortran-lang efforts so far. The production version is written in
Fortran, it can build itself, and there is a growing ecosystem of packages. It
is modeled by Cargo, the Rust package manager. We have a solid foundation to
build upon in the coming years.
Fortran Website
On Jan 17 2020 I suggested to create a website for Fortran. We got it up and running in April 2020 at fortran-lang.org, a domain that Milan has bought some time ago, waiting for the right time to use it. By October 25, the webpage was second in all search engines, except Google where it was 9th. By December 2, it was 4th in Google: Wikipedia now uses our webpage as the Fortran page and our logo. I really like the logo, it was Jacob Williams’s and Milan’s idea, see Milan’s blog post for more details.
We have achieved something incredible: in less than a year we managed to get the Fortran website to top of all major search engines.
Fortran Standards Committee
In April 2020 I announced a run for WG5 Convenor (International Fortran Committee Chair), you can read my platform and motivation in the announcement post. I did not get the position, Steve Lionel was selected as the chair. Later I ran for a J3 chair position, also did not get it, Brian Friesen was selected for it. I maintain excellent, professional relationships with both, as well as the rest of the committee. The committee as well as Steve and Brian all want to see Fortran improve. I wish them luck and I’ve helped them and the committee, as my time and energy allows.
As a highlight of the February 2021 meeting, we got Milan to officially join the committee also, thanks to Gary Klimowicz.
FortranCon 2020
Another lucky coincidence was that Tiziano Mueller and team organized the very first FortranCon 2020. We helped announce at fortran-lang.org also.
The organizers have done a great job. The conference attracted a very enthusiastic group of people. It reminded me the early SciPy conferences. The community was small, and everybody understood we had to help each other. FortranCon felt just like that. Even though it was remote only, the atmosphere was excellent. A lot of good discussions happened.
Community
There have been many people instrumental to the success of the above efforts. You can find their names when you browse the community statistics at the bottom of: https://fortran-lang.org/community/.
The first and foremost that I want to thank to is Milan. I believe my first
text message to Milan was on January 5 2020, setting up time to brainstorm
fpm. We have exchanged hundreds of messages since then and dozens of phone
calls. If it was not for him, the efforts above such as stdlib, fpm,
website, etc. would simply not happen. He took the initiative to lead many of
these efforts and provided me with the motivation to continue. He had many
excellent, complementary ideas, such as starting stdlib, the Fortran monthly
calls, having a Discourse in addition to a mailinglist (the Discourse proved a
lot more popular), having a dedicated Twitter
account for Fortran, and many other. The
contributor statistics I showed above are based on comments only, so they do
not tell the full story.
I want to thank Jeremie Vandenplas, Brad Richardson, Laurence Kedward, Peter Klausler, Gary Klimowicz, Ivan Pribec, urbanjost, FortranFan, Zaak Beekman, Sebastian Ehlert, William Clodius, Arjen Markus, Bálint Aradi, Steve Lionel, Jacob Williams, Dominik Gronkiewicz, Michael Hirsch, Marshall Ward, Chris MacMackin, septcolor, Neil Carlson and the rest of over 100 people who have contributed. It has been a pleasure to work with you all on reviving Fortran.
Conclusions
We have finished the first chapter on our long journy of resurrecting Fortran. If I stop contributing today, I don’t believe these efforts will die. From this perspective I have succeeded. As documented above, it was highly non-trivial and a tremendous amount of work to bootstrap this initial Fortran community just to get to this point.
We still have a long way to go, but the focus of our work has shifted. Now our
job is to grow and leverage the community to make fast enough progress on the
core efforts such as fpm, stdlib and other tooling. We have the platform,
infrastructure and the community to do it.
I personally want to focus more on LFortran to make it succeed. Things look very promising, just in the last month we got 4 people contributing code to the compiler. I feel it is the last missing piece of the puzzle. Together with the new Flang, GFortran, commercial Fortran compilers (https://fortran-lang.org/compilers/) and collaboration with the standards committee I believe we will eventually succeed in making Fortran a viable language for high performance numerical computing going forward.
Discussions: