Which compilers are available for CI with GitHub-hosted runners? All but `nagfor`

1

[Update: This post was linked to Hack News and received some attention.]

Testing has been playing a central role in the development of PRIMA. The tests are automated by GitHub Actions. The concept of CI (Continuous Integration) and GitHub Actions have been not only life-changing but also eye-opening to me. They enable me to test my code with intensity and extensiveness that are unimaginable otherwise.

GitHub Actions with GitHub-hosted runners are particularly useful. With them, you can test your code in a fresh environment. In addition, you do not need to worry about messing up your computer due to the tests. Most importantly, you have access to a virtually unlimited amount of computers in the cloud rather than being limited by physical computers available in your office.

Which Fortran compilers are available on GitHub Actions with GitHub-hosted runners? In my experience, all major compilers on the market except for nagfor from NAG. Here, even discontinued compilers such as g95 and Oracle sunf95 are included, but IBM Open XL Fortran compiler and Cray Fortran compiler are excluded, as they work only on vendor-specific platforms.

nagfor cannot be used with GitHub-hosted runners due to the special way it manages the license. Each license can only be used on a computer with a specific “Kusari ID”. You can deactivate the license on a computer and move it to another one, but you need to send an email to NAG to tell them the new ID and ask for a new license key — a procedure not automatable.

It is not particularly ideal to be such an exception in a world where everything is moving to the cloud, and even MATLAB is available on GitHub-hosted runners, not to mention other compilers such as Intel ifx, NVIDIA nvfortran, and AOCC flang. I do believe it is something urgent to deal with — even though it is not my business. I once enquired NAG support for the possibility of supporting GitHub-hosted runners in the future, and the response was, unfortunately, —

“our developers do not view this item as important where our product(s) is concerned.”

So, no hope at all. What a pity for such an excellent compiler!

P.S.:

How to make compilers available on GitHub Actions with GitHub-hosted runners? In addition to the fantastic fortran-lang/setup-fortran provided by @awvwgk and other contributors, I use the following scripts, which are maintained at GitHub - equipez/github_actions_scripts. Note that my scripts are homemade for personal use, and they install only the latest available version of the compilers, whereas you can control the version with fortran-lang/setup-fortran. The scripts are not composed for use on local machines, as they may make unwanted changes to your system.

See my workflow for a concrete example of using these scripts on GitHub-hosted runners.

10 Likes
2

I tried @zaikunzhang’s “Classic Flang on Linux” but I got this output from
bash installflang on my Ubuntu system:

The runner is not hosted by GitHub. Skip installation of p7zip and ncurses.
Cloning into 'flang'...
remote: Enumerating objects: 21, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 21 (delta 0), reused 0 (delta 0), pack-reused 18
Receiving objects: 100% (21/21), 704.16 MiB | 7.05 MiB/s, done.
Updating files: 100% (12/12), done.
installflang: line 39: 7za: command not found
installflang: line 45: : No such file or directory
installflang: line 46: : No such file or directory
The flang installed is:
installflang: line 52: flang: command not found
The path to flang is:
john@johns-laptop:~$

What should I have done?

3

Hi @Harper , the scripts are not to be used on local machines. They are for GitHub-hosted runners of GitHub Actions.

The error is because your machine does not have p7zip installed.

4

Just a quick remark: Downloading several GBytes of the Intel compiler each time you trigger a GitHub action (typically on each push and PR commit) is a terrible waste of resources. You could use for example the setup-oneapi action to download it only once and store it in the cache of your GitHub project. You can a have a look on the ci-workflow of the Fortuno unit testing framework for an example.

And yes, I absolutely agree: It is a pity, that the NAG-compiler does not provide any support for being used in public CI-workflows (as on GitHub/GitLab). It is my favorite compiler for code development, and many Fortran projects would benefit a lot from that. On the other hand, one can set-up web-hooks to run it on the local hardware triggered by GitHub/GitLab events. (At least in theory, I did not test this myself, yet. I am also not 100%-sure, whether that would be compliant with the licensing terms of the NAG compiler…)

8 Likes
5

This is of course doable, and I am doing it on self-hosted runners. I do not see any problem with the license, as the compiler is running on machines that have valid licenses.

However, as I mentioned, it would be much better if we could use the compiler on GitHub-hosted runners, which has so many advantages. I am willing to pay for being able to do so, and I am willing to pay much more than a normal license. Unfortunately, NAG simply has no interest in making it happen — or NAG simply has no concept about the role that CI plays in modern software development, which is so strange and so sad.

1 Like
6

I do find this slightly strange as well, but ultimately I am guessing it’s a strategic decision. If they can’t make enough revenue from selling licenses for GA usage, it doesn’t make sense to invest their resources towards that, which is understandable.

A while back, I wanted to use NAG in Modern Fortran’s for vscode CI to ensure that the linter support for nagfor was tested, but I couldn’t find a way to do it so I just gave up.

1 Like
7

I would say the management is extremely shortsighted if they regard licensing CI runners in the cloud as a threat to their revenue rather than an opportunity to enter an enormous market where they do not have any share for the time being.

BTW, NAG is the only company I know today that you need to

  • send an email to enquire the price
  • wait for two days to receive a quotation
  • send another email to place an order
  • wait for another two days to receive a confirmation
  • send another email to inform them the “Kusari ID” of your machine
  • wait for another two days to receive the license key
  • optionally, wait for another week or more to receive the receipt of your payment, if you are buying the license using a research grant like me and want the money reimbursed.

I have purchased nagfor licenses three times in the past few years, and it was always like this. It is a very impressive procedure in the second decade of the 21st century. The management of NAG must be extraordinarily confident to maintain the business in this way and believe it will last for another ? years.

Anyway, this is none of my business, but I just feel it is a great pit as nagfor is really a great compiler.

8

That’s true, the procedures are sometimes cumbersome. On the other hand, in favor of NAG, it is the only compiler I know

  • where the reported compiler bugs are usually fixed within 2-3 weeks and you (and others) can then immediately download the fixed build. (But you have to report the bug via mail :wink:)

I wish, other compiler vendors were similarly fast and responsive…

7 Likes
9

As a sideeffect of the fact that nagfor is not available on CI runners, new features of nagfor are not well tested. For example, see

Note that half-precision real has been supported at least since nagfor 7.0 released in 2021, namely three years ago. But I guess no one had ever used it in a real project before I did it with PRIMA. Otherwise, the compiler would not still contain bugs like evaluating -1.0_REAL16 * 1.0_REAL16 to NaN or 0 after supporting REAL16 for three years.

Why no one has ever tested it before? Because people are limited by the physical machines available in their offices/labs, so that they have no incentive to do so until they need half precision is truly needed in their production.

However, if the compiler were available on CI runners in the cloud, many people would have adapted a few lines in their workflow files to test half precision, just for fun or curiosity. Then the ridiculous bugs shown above would have no chance to survive three years.

1 Like
10

This is true. I have had the same experiences, and I must admit that the responsiveness is impressive. Just last week, I reported four bugs to NAG concerning half precision, and they were all fixed within a week, for which I am thankful.

I would also like to mention that, a compiler vendor that did even better was Absoft. A bug report sent to Absoft would usually receive a response on the same day and get fixed two days later.

Being a coincidence or not, it adopted a similar (but quicker) procedure for selling licenses, and it did not support cloud CI runners either.

Unfortunately, the company ceased operation two years ago.

11

My new attempt to get nagfor available for CI with GitHub-hosted runners:

https://github.com/numericalalgorithmsgroup/NAG-Cloud-HPC/issues/21

Update 20240502: the repository has been removed by NAG as they responded. (“That repository is not currently active and will be removed to avoid confusion.”)

1 Like
12

You mention Cray and IBM.

Two other Fortran vendors are NEC and Fujitsu.

Ian Chivers

13

I do not have direct experience with the NAG compiler. I am curious about what makes it so unique. Can you share any insights?

14

It detects and reports many programmer errors that other compilers do not detect.

3 Likes
15

Fully agree. For example, nagfor is quite capable of catching unused, uninitialized, or undefined variables that cannot be detected by others.

nagfor is also one of my favorite. If it were available for CI on GitHub-hosted runners, even with a VERY expensive license, I would rate it as the best.

1 Like
16

NAG does a good job at implementing very quickly the features included in new Fortran Standards. They are also quite responsive and quick to address compiler bugs. Their diagnostics are also very solid, so some folks tent to use nagfor as a way to test it code is standard-compliant.

2 Likes
17

I am one of such folks. I normally check my code with nagfor first after making changes, and then with other compilers available (gfortran, ifx, nvfortran, etc).

1 Like
18

I normally check my code with at least 2 compilers, including nagfor when I had access to it some years ago. As a result I have sent many bug reports to various compiler vendors (including some to NAG. I still treasure this reply I once got from them: “What an interesting bug to find on a Monday morning.”) Sometime in the f77 era I had to send bug reports via my university’s systems people, and was surprised when one of them asked “Why do you keep finding bugs in compilers?” I replied “Does nobody else in this university check their work?” :slight_smile:

3 Likes
19

I do similar checking with 10 compilers, and consequently, I have spotted more than thirty compiler bugs in the past four years. They were submitted to, confirmed by, and sometimes fixed by vendors. In addition, I maintain a catalog of them at GitHub:

In this process, I find that NAG is the second most responsive compiler vendor to bug reports. The first is/was Absoft, which unfortunately ceased operation a few years ago.

At the very beginning, I was astonished to find bugs in compilers. Then it became my daily life. Many people are surprised when I claim that I have spotted quite a few bugs in reputable Fortran compilers (and even in MATLAB). I tell them that they will find the same if they test code as intensively as me.

20

I am regularly finding compiler bugs, although not as frequently as you are. But because of industrial reasons we are using old and unsupported versions of compilers (ifort 18 is still in production for us, and will still be in the next few years), so the best I can do is to find workarounds in the code :slight_smile: .