Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIR: Nim intermediate representation #22777

Merged
merged 39 commits into from Oct 11, 2023
Merged

NIR: Nim intermediate representation #22777

merged 39 commits into from Oct 11, 2023

Conversation

Araq
Copy link
Member

@Araq Araq commented Sep 30, 2023

Theoretical Benefits / Plans:

  • Typed assembler-like language.
  • Allows for a CPS transformation.
  • Can replace the existing C backend by a new C backend.
  • Can replace the VM.
  • Can do more effective "not nil" checking and static array bounds checking.
  • Can be used instead of the DFA.
  • Easily translatable to LLVM.
  • Reasonably easy to produce native code from.
  • Tiny memory consumption. No pointers, no cry.

In very early stages of development.

Todo:

  • Map Nim types to IR types.
  • Map Nim AST to IR instructions:
    • Map bitsets to bitops.
    • Implement string cases.
    • Implement range and index checks.
    • Implement default(T) builtin.
    • Implement multi string concat.
  • Write some analysis passes.
  • Write a backend.
  • Integrate into the compilation pipeline.

@Araq Araq changed the title NIR: An immediate representation for Nim. WIP NIR: Nim immediate representation for Nim Sep 30, 2023
@Araq Araq changed the title NIR: Nim immediate representation for Nim NIR: Nim immediate representation Sep 30, 2023
@Araq Araq changed the title NIR: Nim immediate representation NIR: Nim intermediate representation Sep 30, 2023
@Araq
Copy link
Member Author

Araq commented Sep 30, 2023

Lol, I cannot type "intermediate representation", better only use IR.

@termermc
Copy link

termermc commented Sep 30, 2023

If it opens the gateway for CPS support, then I'm on board

@runvnc
Copy link
Contributor

runvnc commented Oct 1, 2023

Does web assembly factor into this at all? I'm sorry if this doesn't seem relevant, but it feels like web assembly runtimes will continue to ramp up in popular use until they rival the deployment level of Java. (And I know that web assembly is not really like assembly).

@Araq
Copy link
Member Author

Araq commented Oct 1, 2023

Does web assembly factor into this at all?

The IR can be translated to web assembly, yes. I have no plans to do that but in general writing a backend against the IR should be easier than previously.

@xigoi
Copy link

xigoi commented Oct 1, 2023

Is it going to help with incremental compilation?

@Araq
Copy link
Member Author

Araq commented Oct 1, 2023

It was designed with IC in mind but IC also needs more frontend patches or a new frontend altogether.

First I replace the backend passes, then later (which is really much later) the frontend. In the end not much of the old compiler remains and we have IC.

@juancarlospaco
Copy link
Collaborator

juancarlospaco commented Oct 1, 2023

This can allow for a better Nimsuggest ? 🤔

@Araq
Copy link
Member Author

Araq commented Oct 1, 2023

No, nimsuggest is frontend business, this here is for the backend.

@SirOlaf
Copy link
Contributor

SirOlaf commented Oct 1, 2023

How is this going to behave with multiple files, will it combine everything into one large list of instructions or is there intended to be some sort of import construct?
I assume it's the latter because you mention IC so as an extension to this, how do you expect it to affect projects like nimscripter and the now very dusty HCR feature of Nim?

@inv2004
Copy link
Contributor

inv2004 commented Oct 1, 2023

Just my opinion, but if it moves Nim from c-backend - it is a bad from my POV, because intermediate C helps a lot pretty often when you dig into perf (and want to match c-lang perf). Debug Ffi bindings and etc. c-backend is a huge plus of Nim

@Araq
Copy link
Member Author

Araq commented Oct 1, 2023

how do you expect it to affect projects like nimscripter and the now very dusty HCR feature of Nim?

The IR is easy to interpret and if you do that the data types between compiled and interpreted code are mostly compatible so things should become easier for nimscripter.

Yes, the IR has the notion of an "import" from different modules. If you seek to interpret the code, you need to have the imported modules in memory. Loading an IR file into memory is cheap as it is already in a linear format, there is little to do.

@Araq
Copy link
Member Author

Araq commented Oct 1, 2023

Just my opinion, but if it moves Nim from c-backend - it is a bad from my POV, because intermediate C helps a lot pretty often when you dig into perf (and want to match c-lang perf). Debug Ffi bindings and etc. c-backend is a huge plus of Nim

There are no plans to abandon C as compilation target, but instead of the AST the NIR is translated to C.

@inv2004
Copy link
Contributor

inv2004 commented Oct 1, 2023

Translation to llvm means that Nim goes away from c-backend. LLVM is also not positive trend in my opinion.

@jmgomez
Copy link
Collaborator

jmgomez commented Oct 1, 2023

@inv2004 and what this means:

Can replace the existing C backend by a new C backend.

It seems to me like the idea here is to allow for more supported backends rather than less.

@Vindaar
Copy link
Contributor

Vindaar commented Oct 1, 2023

Translation to llvm means that Nim goes away from c-backend. LLVM is also not positive trend in my opinion.

Pretty sure the "Easily translatable to LLVM" is not meant to replace the standard C / C++ backends. It's just mentioning that this should make work like nlvm much simpler.

@inv2004
Copy link
Contributor

inv2004 commented Oct 1, 2023

@Vindaar all I wanted to say - that all the things like asm, wasm, llvm and etc are interesting to play with, but it is 3rd priority to have it, but I was afraid that c-backend have a chance to loose its 1st priority. (Everything with llvm does not have the advantage Nim has)

Also, I do not know, but are there any perf-advantages nlvm gives at the moment? The same is about wasm - are there any visible benefits to have it directly or from llvm compared to have it from generated c-code?

If the IR makes the primary c-backend easier from logical endpoint - perfect

@Araq
Copy link
Member Author

Araq commented Oct 11, 2023

Further commits in follow-up PRs.

@Araq Araq merged commit 816589b into devel Oct 11, 2023
22 checks passed
@Araq Araq deleted the araq-nir branch October 11, 2023 15:44
@github-actions
Copy link
Contributor

Thanks for your hard work on this PR!
The lines below are statistics of the Nim compiler built from 816589b

Hint: mm: orc; opt: speed; options: -d:release
174040 lines; 12.808s; 770.027MiB peakmem

@SirOlaf
Copy link
Contributor

SirOlaf commented Oct 11, 2023

Seems to break koch tools with Nim/compiler/nir/ast2ir.nim(2083, 5) Error: undeclared identifier: 'setFrameMsg' for nim_dbg build step.

Araq pushed a commit that referenced this pull request Oct 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet