I see there is a method allocArrayBuffer in the runtime, but looking through every test in /tests and /newtests, I couldn't find some usage of code like:

std::byte buffer[4096];

I tried the below on a guess, but this was not valid rune:

myArray = [u32; 4096]

Heya,

When I first read the Rune readme, there were a couple of things I thought could be improved:

• I didn't understand where methods like appendMotheredHuman etc come from. We "magically" have access to something we didn't define and it feels strange. It gets described later on, but it would have been nice for a comment to say "Hey, don't worry! I'll explain why this magic-method exists in a little bit!"
• DataDraw doesn't get mentioned, and I had to piece the connection together myself. In my opinion, a HUGE amount of appeal comes from the fact that DD is integrated into Rune, and it's very interesting + explains much of the behavior and design decisions/architecture + performance.

Also, many people will want to jump directly to specific things ("how do I build this?", "show me a code example?") without reading the rest of it, so I tried to add enough context to each section that it would make sense if you went straight there and skipped the rest.

Lastly, I added some organizational structure and a TOC so folks can navigate the page easier.

Hope this is helpful, I'd like to highlight Rune in the best way possible for potential users as I think it has some neat ideas 🙂

Ahh, one last thing -- I chose the languages for the syntax highlighting for each code block based on which one looked the best/could highlight the majority of the tokens:

Go:

// Check the MAC (message authentication code) for a message.  A MAC is derived
func checkMac(macSecret: secret(string), message: string, mac: string) -> bool {
    computedMac = computeMac(macSecret, message)
    return mac == computedMac
}

Rust (TypeScript comes close too, but not quite as good):

class Human(self: Human, name: string, mother: Human? = null(self), father: Human? = null(self)) {
  self.name = name
  if !isnull(mother) {
    mother.appendMotheredHuman(self)
  }
}

I followed the "Compiling the Rune compiler" from the readme and it all finally compiled and didn't complain😅 Then I tried to run the tests and saw

/tmp/nullindirection-c4ad80.o: In function `hashValues':
/projects/rune/builtin/hashed.rn:17: undefined reference to `llvm.fshr.i64'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
./runl: line 6: ./errortests/nullindirection: No such file or directory
....
Passed: 39
Failed: 185

I tried to compile a hello world program I got the same error Also I found that by using -b flag program was able to compile

• rune: Current main
• os: Ubuntu 18.04.5 LTS bionic on Windows 10 x86_64
• clang: version 6.0.0-1ubuntu2 (tags/RELEASE_600/final) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin

The test failed when runtests.sh is executed.

I fixed it by referring to here. https://github.com/google/rune/blob/main/doc/rune4python.md#recursion

The following PR was withdrawn because it had two commit logs. https://github.com/google/rune/pull/16

Following the installation instructions in the README, during the "building rune"-step, I ran into a missing prerequisite. The make command in

$ git clone https://github.com/google/rune.git
$ git clone https://github.com/pornin/CTTK.git
$ cp CTTK/inc/cttk.h CTTK
$ cd rune
$ make

resulted in

clang -Wall -g -std=c11 -Wno-varargs -I../../CTTK -DRN_DEBUG -c array.c bigint.c io.c random.c
make[1]: clang: No such file or directory

This was off course easy fixable by installing clang. But since especially clang-14 is specifically stated as prerequisite I wondered if this list is incomplete and clang should be added to the instructions, specifically expanding

$ sudo apt-get install bison flex libgmp-dev clang-14

to

$ sudo apt-get install bison flex libgmp-dev clang clang-14

or if this was a problem on my site. Happy to give further information or to extend the README.

errortests/trunc.rn fails with a var (and also a constant)

variable should add a runtime truncation check:

%2 = trunc i64 %1 to i16, !dbg !208
; missing %1 vs %2 check

constant may throw directly at compile-time:

println "%x16" % <u16>0xdeadbeef =>
%1 = trunc i64 3735928559 to i16, !dbg !205

diff --git llvm/genllvm.c llvm/genllvm.c
index a9fff61..245cb0d 100644
--- llvm/genllvm.c
+++ llvm/genllvm.c
@@ -1490,6 +1490,8 @@ static llElement resizeSmallInteger(llElement element, uint32 newWidth, bool isS
   char *operation;
   if (newWidth < oldWidth) {
     operation = "trunc";
+    //TODO if constant throw error if truncation looses info.
+    //     else if not constant add code to check at run-time.
   } else {
     deDatatypeType type = deDatatypeGetType(oldDatatype);
     if (type == DE_TYPE_INT) {

See also: https://cwe.mitre.org/data/definitions/197.html

Just a bunch of tiny adjustments (the commit history is hopefully descriptive enough!), with some examples stemming from recent additions to the Python language added.

(Note: I intend to add/update the docs for this feature, after the other PR I have for docs is either merged or rejected)

With this Dockerfile, it's possible to build the compiler by just running:

$ docker build -t rune-compiler:latest .

You can then compile rune files like this:

$ echo 'println "Hello, World!"' > hello.rn
$ docker run -v $(pwd):/tmp rune-compiler rune -g /tmp/hello.rn

$ ls | grep hello
hello
hello.ll
hello.rn

$ ./hello
Hello, World!

It's also possible to use this as a hermetic/reproducible environment to build the compiler/core libs, and then copy them into your host machine:

$ docker run -i -v $(pwd)/rune:/tmp rune-compiler:latest sh << COMMANDS
  cp -r /usr/local/bin /tmp
  cp -r /usr/local/lib/rune /tmp/lib
COMMANDS

$ tree rune -L 2
rune
├── bin
│   ├── datadraw
│   ├── dataview
│   └── rune
└── lib
    ├── builtin
    ├── io
    ├── libcttk.a
    ├── librune.a
    ├── math
    └── runtime

$ ./rune --help
Usage: rune [options] file
    -b        - Don't load bulitin Rune files.
    -g        - Include debug information for gdb.  Implies -l.
    -l <llvmfile> - Write LLVM IR to <llvmfile>.
    -n        - No clang.  Don't compile the resulting .ll output.
    -O        - Optimized build.  Passes -O3 to clang.
    -p <dir>  - Use <dir> as the root directory for packages.
    -t        - Execute unit tests for all modules.
    -U        - Unsafe mode.  Don't generate bounds checking, overflow
                detection, and destroyed object access detection.
    -x        - Invert the return code: 0 if we fail, and 1 if we pass.
    -X        - Use the new event driven binder.

While trying to compile the rune, I was surprised to see that a lot junk of files were picked up by git I think it makes sense to ignore these files, especially the compiled binary files

There was a minor typo in the Readme.md in two places I fixed. The PR https://github.com/google/rune/pull/4 also solves one of the places but not the other.

Minor updates to the README:

• Bold and shorten the warning about using Rune in production.
• Minor editorial, sentence-level changes in the description of the language
• Update list of dependencies to enumerate all six major dependencies. Reorder the list so that the statement about installing "the first four" makes more sense.
• Use code fencing syntax instead of tabs, mark the language as shell (sh).

This looks a hell lot like an already existing language, called Nim

Python like syntax with C/C++ interop, reference counting and so on.

Did you take any inspiration? https://nim-lang.org/

And what would you say, are Rune's benefits, compared to it?

In the class example in the README, the Human class adds self to mother and father. However it does not add mother and father anywhere into self, which is what I would expect with "DoublyLinked." What does it mean?

Using some online tools like https://www.bottlecaps.de/rr/ui and https://www.bottlecaps.de/convert/ and a bit of manual fixes we can have a nice navigable railroad diagram.

Copy and paste the EBNF shown bellow on https://www.bottlecaps.de/rr/ui on the tab Edit Grammar the click on the tab View Diagram to see/download a navigable railroad diagram.

/* converted on Sun Nov 27, 2022, 10:23 (UTC+01) by bison-to-w3c v0.62 which is Copyright (c) 2011-2022 by Gunther Rademacher <[email protected]> */

goal     ::= initialize optNewlines runeFile
initialize
         ::=
runeFile ::= statements
statements
         ::= statement*
statement
         ::= appendCode
           | assertStatement
           | assignmentStatement
           | callStatement
           | class
           | debugStatement
           | enum
           | externFunction
           | finalFunction
           | foreachStatement
           | forStatement
           | function
           | generateStatement
           | generatorStatement
           | ifStatement
           | import
           | prependCode
           | printlnStatement
           | printStatement
           | refStatement
           | relationStatement
           | returnStatement
           | struct
           | switchStatement
           | throwStatement
           | unitTestStatement
           | unrefStatement
           | whileStatement
           | yield
import   ::= ( ( KWIMPORT | KWIMPORTLIB | KWIMPORTRPC ) pathExpressionWithAlias | KWUSE IDENT ) newlines
class    ::= ( classHeader | exportClassHeader ) '(' oneOrMoreParameters ')' block
classHeader
         ::= KWCLASS IDENT optWidth
optWidth ::= ( ':' UINTTYPE )?
exportClassHeader
         ::= ( KWEXPORT | KWEXPORTLIB | KWRPC ) KWCLASS IDENT optWidth
struct   ::= structHeader '{' newlines ( structMember newlines )* '}' newlines
           | exportStructHeader block
structHeader
         ::= KWSTRUCT IDENT
structMember
         ::= optConst IDENT optTypeExpression optInitializer
optConst ::= KWCONST?
optInitializer
         ::= ( '=' expression )?
exportStructHeader
         ::= ( KWEXPORT | KWEXPORTLIB ) KWSTRUCT IDENT
appendCode
         ::= appendCodeHeader block
appendCodeHeader
         ::= KWAPPENDCODE pathExpression?
prependCode
         ::= prependCodeHeader block
prependCodeHeader
         ::= KWPREPENDCODE pathExpression?
block    ::= '{' newlines statements '}' optNewlines
function ::= ( functionHeader | exportFunctionHeader | rpcHeader ) '(' parameters ')' optFuncTypeExpression block
functionHeader
         ::= ( KWFUNC | KWITERATOR ) IDENT
           | KWOPERATOR operator
operator ::= '+'
           | '-'
           | '*'
           | '/'
           | '%'
           | KWAND
           | KWOR
           | KWXOR
           | '&'
           | '|'
           | '@'
           | '^'
           | KWSHL
           | KWSHR
           | KWROTL
           | KWROTR
           | KWADDTRUNC
           | KWSUBTRUNC
           | KWMULTRUNC
           | '~'
           | '<'
           | KWLE
           | '>'
           | KWGE
           | KWEQUAL
           | KWNOTEQUAL
           | '!'
           | '[' ']'
           | KWIN
exportFunctionHeader
         ::= ( KWEXPORT ( KWFUNC | KWITERATOR ) | KWEXPORTLIB KWFUNC ) IDENT
parameters
         ::= oneOrMoreParameters?
oneOrMoreParameters
         ::= parameter ( ',' optNewlines parameter )*
parameter
         ::= optVar ( IDENT | '<' IDENT '>' ) optTypeExpression
           | initializedParameter
optVar   ::= KWVAR?
initializedParameter
         ::= optVar ( IDENT | '<' IDENT '>' ) optTypeExpression '=' expression
externFunction
         ::= ( KWEXTERN STRING functionHeader | rpcHeader ) '(' parameters ')' optFuncTypeExpression newlines
rpcHeader
         ::= KWRPC IDENT
ifStatement
         ::= ifPart elseIfPart* optElsePart
ifPart   ::= ifStatementHeader expression block
ifStatementHeader
         ::= KWIF
elseIfPart
         ::= elseIfStatementHeader expression block
elseIfStatementHeader
         ::= KWELSE KWIF
optElsePart
         ::= elsePart?
elsePart ::= elseStatementHeader block
elseStatementHeader
         ::= KWELSE
switchStatement
         ::= switchStatementHeader expression switchBlock
switchStatementHeader
         ::= KWSWITCH
switchBlock
         ::= '{' newlines switchCase* optDefaultCase '}' optNewlines
switchCase
         ::= KWCASE expression ( ',' optNewlines expression )* block
optDefaultCase
         ::= ( defaultCaseHeader block )?
defaultCaseHeader
         ::= KWDEFAULT
whileStatement
         ::= optDoStatement whileStatementHeader expression ( newlines | block )
whileStatementHeader
         ::= KWWHILE
optDoStatement
         ::= doStatement?
doStatement
         ::= doStatementHeader block
doStatementHeader
         ::= KWDO
forStatement
         ::= forStatementHeader nonConstAssignmentExpression ',' optNewlines expression ',' optNewlines nonConstAssignmentExpression block
forStatementHeader
         ::= KWFOR
assignmentStatement
         ::= assignmentExpression newlines
assignmentExpression
         ::= KWCONST? nonConstAssignmentExpression
nonConstAssignmentExpression
         ::= accessExpression optTypeExpression assignmentOp expression
assignmentOp
         ::= '='
           | KWADDEQUALS
           | KWSUBEQUALS
           | KWMULEQUALS
           | KWDIVEQUALS
           | KWMODEQUALS
           | KWBITANDEQUALS
           | KWBITOREQUALS
           | KWBITXOREQUALS
           | KWANDEQUALS
           | KWOREQUALS
           | KWXOREQUALS
           | KWEXPEQUALS
           | KWSHLEQUALS
           | KWSHREQUALS
           | KWROTLEQUALS
           | KWROTREQUALS
           | KWADDTRUNCEQUALS
           | KWSUBTRUNCEQUALS
           | KWMULTRUNCEQUALS
optTypeExpression
         ::= ( ':' expression )?
optFuncTypeExpression
         ::= ( KWARROW expression )?
accessExpression
         ::= tokenExpression ( '(' callParameterList ')' | '.' IDENT | '[' expression ( ':' expression )? ']' )*
callStatement
         ::= accessExpression '(' callParameterList ')' newlines
callParameterList
         ::= ( callParameter ( ',' optNewlines callParameter )* optComma )?
callParameter
         ::= ( IDENT '=' )? expression
optComma ::= ','?
printStatement
         ::= KWPRINT expressionList newlines
printlnStatement
         ::= KWPRINTLN expressionList newlines
throwStatement
         ::= KWTHROW expressionList newlines
assertStatement
         ::= KWASSERT expressionList newlines
returnStatement
         ::= KWRETURN expression? newlines
generatorStatement
         ::= generatorHeader '(' parameters ')' block
generatorHeader
         ::= KWGENERATOR IDENT
generateStatement
         ::= KWGENERATE pathExpression '(' expressionList ')' newlines
relationStatement
         ::= KWRELATION pathExpression pathExpression optLabel pathExpression optLabel optCascade optExpressionList newlines
optLabel ::= ( ':' STRING )?
optCascade
         ::= KWCASCADE?
optExpressionList
         ::= ( '(' expressionList ')' )?
yield    ::= KWYIELD expression newlines
unitTestStatement
         ::= unitTestHeader block
unitTestHeader
         ::= KWUNITTEST IDENT
debugStatement
         ::= debugHeader block
debugHeader
         ::= KWDEBUG
enum     ::= enumHeader '{' newlines entry+ '}' newlines
enumHeader
         ::= KWENUM IDENT
entry    ::= IDENT ( '=' INTEGER )? newlines
foreachStatement
         ::= forStatementHeader IDENT KWIN expression block
finalFunction
         ::= finalHeader '(' parameter ')' block
finalHeader
         ::= KWFINAL
refStatement
         ::= KWREF expression newlines
unrefStatement
         ::= KWUNREF expression newlines
expressionList
         ::= oneOrMoreExpressions?
oneOrMoreExpressions
         ::= expression ( ',' optNewlines expression )*
expression
         ::= dotDotDotExpression
dotDotDotExpression
         ::= selectExpression ( KWDOTDOTDOT selectExpression )?
selectExpression
         ::= orExpression ( '?' orExpression ':' orExpression )?
orExpression
         ::= xorExpression ( KWOR xorExpression )*
xorExpression
         ::= andExpression ( KWXOR andExpression )*
andExpression
         ::= inExpression ( KWAND inExpression )*
inExpression
         ::= modExpression ( KWIN modExpression )?
modExpression
         ::= relationExpression ( KWMOD bitorExpression )?
relationExpression
         ::= bitorExpression ( ( '<' | KWLE | '>' | KWGE | KWEQUAL | KWNOTEQUAL ) bitorExpression )?
bitorExpression
         ::= bitxorExpression ( '|' bitxorExpression )*
bitxorExpression
         ::= bitandExpression ( '@' bitandExpression )*
bitandExpression
         ::= shiftExpression ( '&' shiftExpression )*
shiftExpression
         ::= addExpression ( ( KWSHL | KWSHR | KWROTL | KWROTR ) addExpression )?
addExpression
         ::= KWSUBTRUNC? mulExpression ( ( '+' | '-' | KWADDTRUNC | KWSUBTRUNC ) mulExpression )*
mulExpression
         ::= prefixExpression ( ( '*' | '/' | '%' | KWMULTRUNC ) prefixExpression )*
prefixExpression
         ::= postfixExpression ( '^' postfixExpression )*
           | ( '!' | '~' | '-' | ( '<' | KWCASTTRUNC ) prefixExpression '>' ) prefixExpression
postfixExpression
         ::= accessExpression
           | '&' pathExpression '(' expressionList ')'
tokenExpression
         ::= IDENT
           | STRING
           | INTEGER
           | FLOAT
           | RANDUINT
           | BOOL
           | '[' oneOrMoreExpressions ']'
           | ( KWSECRET | KWREVEAL | KWARRAYOF | KWTYPEOF | KWUNSIGNED | KWSIGNED | KWWIDTHOF | KWISNULL )? '(' expression ')'
           | tupleExpression
           | KWNULL
           | typeLiteral
typeLiteral
         ::= UINTTYPE
           | INTTYPE
           | KWSTRING
           | KWBOOL
           | KWF32
           | KWF64
pathExpression
         ::= IDENT ( '.' IDENT )*
pathExpressionWithAlias
         ::= pathExpression ( KWAS IDENT )?
tupleExpression
         ::= '(' expression ( ( ',' optNewlines expression )+ optComma | ',' ) ')'
optNewlines
         ::= '\n'*
newlines ::= ( '\n' | ';' )+

//
// Tokens
//

// <INITIAL>\("[^"]+"\)\s+{.+?return \([^;]+\); }  -> \2 ::= \1

KWAPPENDCODE ::= "appendcode"
KWARRAYOF ::= "arrayof"
KWAS ::= "as"
KWASSERT ::= "assert"
KWBOOL ::= "bool"
KWCASCADE ::= "cascade"
KWCASE ::= "case"
KWCLASS ::= "class"
KWCONST ::= "const"
KWDEBUG ::= "debug"
KWDEFAULT ::= "default"
KWDO ::= "do"
KWELSE ::= "else"
KWENUM ::= "enum"
KWEXPORT ::= "export"
KWEXPORTLIB ::= "exportlib"
KWRPC ::= "rpc"
KWEXTERN ::= "extern"
BOOL ::= "false"
KWFINAL ::= "final"
KWFOR ::= "for"
KWFUNC ::= "func"
KWGENERATE ::= "generate"
KWGENERATOR ::= "generator"
KWIF ::= "if"
KWIMPORT ::= "import"
KWIMPORTLIB ::= "importlib"
KWIMPORTRPC ::= "importrpc"
KWIN ::= "in"
KWISNULL ::= "isnull"
KWITERATOR ::= "iterator"
KWMOD ::= "mod"
KWNULL ::= "null"
KWOPERATOR ::= "operator"
KWPREPENDCODE ::= "prependcode"
KWPRINT ::= "print"
KWPRINTLN ::= "println"
KWREF ::= "ref"
KWRELATION ::= "relation"
KWRETURN ::= "return"
KWREVEAL ::= "reveal"
KWSECRET ::= "secret"
KWSIGNED ::= "signed"
KWSTRING ::= "string"
KWSTRUCT ::= ("struct"|"message")
KWSWITCH ::= "switch"
KWTHROW ::= "throw"
BOOL ::= "true"
KWTYPEOF ::= "typeof"
KWUNITTEST ::= "unittest"
KWUNREF ::= "unref"
KWUNSIGNED ::= "unsigned"
KWUSE ::= "use"
KWVAR ::= "var"
KWWHILE ::= "while"
KWWIDTHOF ::= "widthof"
KWYIELD ::= "yield"

KWADDEQUALS ::= "+="
KWADDTRUNCEQUALS ::= "!+="
KWADDTRUNC ::= "!+"
KWANDEQUALS ::= "&&="
KWBITANDEQUALS ::= "&="
KWAND ::= "&&"
KWARROW ::= "->"
KWCASTTRUNC ::= "!<"
KWDIVEQUALS ::= "/="
KWDOTDOTDOT ::= "..."
KWEQUAL ::= "=="
KWEXPEQUALS ::= "^="
KWGE ::= ">="
KWLE ::= "<="
KWMODEQUALS ::= "%="
KWMULEQUALS ::= "*="
KWMULTRUNCEQUALS ::= "!*="
KWMULTRUNC ::= "!*"
KWNOTEQUAL ::= "!="
KWBITOREQUALS ::= "|="
KWOREQUALS ::= "||="
KWOR ::= "||"
KWROTLEQUALS ::= "<<<="
KWROTL ::= "<<<"
KWROTREQUALS ::= ">>>="
KWROTR ::= ">>>"
KWSHLEQUALS ::= "<<="
KWSHL ::= "<<"
KWSHREQUALS ::= ">>="
KWSHR ::= ">>"
KWSUBEQUALS ::= "-="
KWSUBTRUNCEQUALS ::= "!-="
KWSUBTRUNC ::= "!-"
KWBITXOREQUALS ::= "@="
KWXOREQUALS ::= "@@="
KWXOR ::= "@@"
KWF32 ::= "f32"
KWF64 ::= "f64"