I've found a couple of issues with transpiling functions and dealing with 'this'.

Example 1:

interface Foo {
    bar(s: string): void;
}
const o: Foo = {
    bar: (s: string) => print(s)
}
o.bar("foo");

local o = {bar = function(s) return print(s) end};
o:bar("foo"); --prints "table: 00709330"

Here, the colon is mistakenly used because the interface declares the function as a method, but the implementation creates it as an arrow function. This is a tough issue to solve since there's no way to know for sure which operator to use at the call site.

Example 2:

class C {
    private prop = "foo";
    public outer() {
        const o = {
            prop: "bar",
            innerFunc: function() { print(this.prop); },
            innerArrow: () => print(this.prop)
        };
        o.innerFunc(); //Should print "bar"
        o.innerArrow(); //Should print "foo"
    }
}
let c = new C();
c.outer();

C = C or {}
C.__index = C
function C.new(construct, ...)
    local self = setmetatable({}, C)
    self.prop = "foo"
    if construct and C.constructor then C.constructor(self, ...) end
    return self
end
function C.constructor(self)
end
function C.outer(self)
    local o = {prop = "bar",innerFunc = function()
        print(self.prop);
    end
,innerArrow = function() return print(self.prop) end};
    o.innerFunc(); --Incorrectly prints "foo"!
    o.innerArrow();
end
local c = C.new(true);
c:outer();

Here the inner function incorrectly captures self from the outer method.

I suspect there's probably a lot of other similar edge cases around. They all boil down to inconsistencies with how JS handles 'this' binding.

As I understand it, in JS, all functions have a 'this', which is whatever was on the left side of the dot when called. The 2 exceptions are arrow functions and stand-alone functions (ones with no left-side to bind). Arrow functions capture the 'this' from their outer scope and stand-alone functions have the global scope (window) as their 'this'.

I believe the solution is to re-work how functions are transpiled overall to mirror that behavior:

• All functions should be given an initial 'self' parameter
• Arrow functions should name this first parameter with a dummy name ('_') so they capture the self of their outer scope
• All functions (except stand-alone) should be called with colon syntax
• Stand alone functions should (maybe?) pass _G as their first parameter

With this setup, all lua functions would have a self that mirrors 'this' from the equivalent JS. Plus, there would be no need to decide whether to use dot or colon as all calls would use colon.

The one case this doesn't handle is functions from external sources that need to be called with dot syntax. A decorator would probably be the way to handle this. I know that was shot down before (see #108), but I think it's worth reconsidering.

Thoughts?

Transpile the ternary operator using and/or and boxing/unboxing. This reduces the cost of the ternary operator from a library call and the creation of two closures to a table allocation and lookup.

Example

declare function f(): number | undefined;
const a = true ? 0 : f();
const b = true ? f() : 0;

Before

-- Lua Library Imports
function __TS__Ternary(condition,cb1,cb2)
    if condition then
        return cb1()
    else
        return cb2()
    end
end

local a = __TS__Ternary(true, function() return 0 end, function() return f() end);
local b = __TS__Ternary(true, function() return f() end, function() return 0 end);

After LuaJIT:

local a = ((true) and (0) or (f()));
local b = ((true) and {f()} or {0})[1];

Other:

local a = ((true) and (0) or (f()));
local b = ((true) and function() return f(); end or function() return 0; end)();

This implements the support for TS's continue statement. Normally, tstl would throw an exception whenever the continue statement is used, saying that Lua does not support the statement.

However, while Lua indeed does not support the continue statement, it is still perfectly possible to implement the functionality of a continue statement by changing how loops get transpiled.

Here's a simple example:

The following TS

for (let i = 1; i <= 100; i++) {
  if (i < 25) {
    continue;
  }
  
  print(i);
}

Gets transpiled to

local i = 1
while(i<=100) do
    local ____continue = false
    repeat
        if i<25 then
            ____continue = true
            break
        end
        print(i)
        ____continue = true
    until true
    i=i+1
    if not ____continue then break end
end

As can be seen from the example above, the functionality of a continue statement gets achieved via a clever usage of the repeat until true block, which acts a simple do end block, but allows for a possibility of stopping the execution of the block via a break statement. The ____continue local allows for differentiation between a break statement for the repeat until true and the parent loop. If a break statement was used without setting the ____continue local to true, then it will have an effect of a normal break (break from repeat until true and break from the parent loop). However, if a break statement was used after setting the ____continue local to true, then it will act with accordance to how the effect of the continue statement should be, which is: stop executing the current cycle, and jump to the next one (break repeat until true, but do not break the parent loop). The ____continue = true gets automatically added at the end of the body of the parent loop.

So, the continue statement simply gets transpiled to:

____continue = true
break

With this implementation, you can nest multiple loops inside of each other, and the ____continue locals of the nested loops will not interfere with each other, thanks to how scoping works in Lua.

The following loops were used for testing:

declare function print(...messages: any[]): void;

for (let i = 1; i <= 100; i++) {
  if (i < 25) {
    continue;
  }
  print(i);

  for (let x = 1; x <= 10; x++) {
    if (x < 5) {
      continue;
    }
    print(x)
  }
}

let d = 10
while (d > 0) {
  d--;
  if (d > 5) {
    let o = 4;
    while (o > 0) {
      o--;
      if (o > 2) {
        continue;
      }
      print(o);
    }
    continue;
  }
  if (d == 2) {
    continue;
  }
  print(d);
}

let e = 10
do {
  e--;
  if (e > 5) {
    let o = 4;
    do {
      o--;
      if (o > 2) {
        continue;
      }
      print(o);
    } while (o > 0)
    continue;
  }
  if (e == 2) {
    continue;
  }
  print(e);
} while (e > 0)

for (let i of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]) {
  if (i < 3) {
    continue;
  }

  for (let i of [5, 6, 7, 8]) {
    if (i !== 6) {
      continue
    }
    print(i)
  }

  if (i > 8) {
    continue;
  }

  print(i);
}

for (let i in {
  a: 1,
  b: 2,
  c: 3,
  d: 4
}) {
 if (i == 'a') {
   continue;
 }

 if (i == 'b') {
   continue;
 }

 print(i);
}

The transpiled code:

-- Generated by TypescriptToLua v0.2.1
-- https://github.com/Perryvw/TypescriptToLua
require("typescript_lualib")
local i = 1
while(i<=100) do
    local ____continue = false
    repeat
        if i<25 then
            ____continue = true
            break
        end
        print(i)
        local x = 1
        while(x<=10) do
            local ____continue = false
            repeat
                if x<5 then
                    ____continue = true
                    break
                end
                print(x)
                ____continue = true
            until true
            x=x+1
            if not ____continue then break end
        end
        ____continue = true
    until true
    i=i+1
    if not ____continue then break end
end
local d = 10

while d>0 do
    local ____continue = false
    repeat
        d=d-1
        if d>5 then
            local o = 4

            while o>0 do
                local ____continue = false
                repeat
                    o=o-1
                    if o>2 then
                        ____continue = true
                        break
                    end
                    print(o)
                    ____continue = true
                until true
                if not ____continue then break end
            end
            ____continue = true
            break
        end
        if d==2 then
            ____continue = true
            break
        end
        print(d)
        ____continue = true
    until true
    if not ____continue then break end
end
local e = 10

repeat
    local ____continue = false
    repeat
        e=e-1
        if e>5 then
            local o = 4

            repeat
                local ____continue = false
                repeat
                    o=o-1
                    if o>2 then
                        ____continue = true
                        break
                    end
                    print(o)
                    ____continue = true
                until true
                if not ____continue then break end
            until not (o>0)
            ____continue = true
            break
        end
        if e==2 then
            ____continue = true
            break
        end
        print(e)
        ____continue = true
    until true
    if not ____continue then break end
until not (e>0)
for _, i in ipairs({1,2,3,4,5,6,7,8,9,10}) do
    local ____continue = false
    repeat
        if i<3 then
            ____continue = true
            break
        end
        for _, i in ipairs({5,6,7,8}) do
            local ____continue = false
            repeat
                if i~=6 then
                    ____continue = true
                    break
                end
                print(i)
                ____continue = true
            until true
            if not ____continue then break end
        end
        if i>8 then
            ____continue = true
            break
        end
        print(i)
        ____continue = true
    until true
    if not ____continue then break end
end
for i, _ in pairs({a = 1,b = 2,c = 3,d = 4}) do
    local ____continue = false
    repeat
        if i=="a" then
            ____continue = true
            break
        end
        if i=="b" then
            ____continue = true
            break
        end
        print(i)
        ____continue = true
    until true
    if not ____continue then break end
end

The outcomes match.

I don't really know if there's a certain way that the pull requests should be made, or if there's some kind of standard for the workflow. If I've done something wrong with the presentation, then please don't hesitate to correct me. :)

Some Lua APIs provide objects with methods that must be invoked using dot syntax (thing.method(...)) as opposed to colon syntax (thing:method(...)). Having a custom decorator for this would be very useful.

Edit: This behavior already exists for namespaces, it just needs to be expanded for object-oriented use. I propose a new custom decorator, !DotMethod, applicable to declarations of classes, interfaces, type literals, and methods of any of these.

Example usages:

/** !DotMethod */
declare class SomethingExternal {
	doThing(): SomethingReturned;
}

/** !DotMethod */
interface SomethingReturned {
	doOtherThing(): void;
}

type SomethingElse = {
	/** !DotMethod **/
	dotMethod(): void,
	regularMethod(): void
}

let count = 0;
count++;

let count01 = count++;
let count02 = count += 1;
let count03 = count = count + 1;

let counter01 = `Next count: ${++count}`;
let counter02 = `Next count: ${count = count + 1}`;

Transpiles to

local count = 0
count=count+1

-- unexpected symbol near '='
local count01 = count=count+1
-- unexpected symbol near '='
local count02 = count = count+1
-- unexpected symbol near '='
local count03 = count = (count+1)

-- ')' expected near '='
local counter01 = "Next count: "..tostring(count=count+1)..""
-- ')' expected near '='
local counter02 = "Next count: "..tostring((count = (count+1)))..""

This is an issue because lue doesn't support assignment in an expression.

Proposal: Wrap the assignments in an anonymous function call when the context requires it:

local count = 0
count=count+1
-- Left becomes its own statement and operand becomes the return value
local count01 = (function() count=count+1; return count end)()
local count02 = (function() count=count+1; return count end)()
local count03 = (function() count = (count+1); return count end)()

local counter01 = "Next count: "..tostring((function() count=count+1; return count end)())..""
-- Extra parenthesis around the assignment causes and error, changed:
--     (count = (count+1)); -> count = (count+1);
local counter02 = "Next count: "..tostring((function() count = (count+1); return count end)())..""

Resolves #194 (or rather brings support for Lua 5.1 and universal).

Implemented using repeat...until true, if, and some conditional or chaining. Relies on switch fallthrough amalgamation / unrolling for default case fallthrough.

switch (0 as number) {
    case 0:
    case 1:
    case 2:
        out.push("0,1,2");
    default:
        out.push("default");
    case 3:
        out.push("3");
}

repeat
    local ____switch3 = 0
    local ____cond3 = ((____switch3 == 0) or (____switch3 == 1)) or (____switch3 == 2)
    if ____cond3 then
        __TS__ArrayPush(out, \\"0,1,2\\")
    end
    if ____cond3 then
        __TS__ArrayPush(out, \\"default\\")
    end
    ____cond3 = ____cond3 or (____switch3 == 3)
    if ____cond3 then
        __TS__ArrayPush(out, \\"3\\")
    end
    if not ____cond3 then
        __TS__ArrayPush(out, \\"default\\")
        __TS__ArrayPush(out, \\"3\\")
    end
until true

Update:

The final approach was to or together the conditions into a variable and check that condition in the if statement to ensure that we do not add any additional comparisons that would trigger undesired side effects.

In order to handle the default case, we must still amalgamate the fallthrough cases that a default would fall into, and provide that as the final or clause, by checking that no previous condition had passed.

I am making my best attempt to ensure that any redundant or useless cases are removed, and empty cases coalesced.

Unforeseen bonus, the generated code is a bit easier to reason about!

Orignal:

The approach I took is to gather the fall-through bodies based on the defined order in the switch statement and amalgamate them until I hit a break statement that would break the control flow of the switch case. I then move the resulting default block to a new else statement. It is crucial to handle the code generation for the default case before moving it into the else block, as the rules of fall through would dictate that the case directly after the default would execute.

Took lots of testing until it clicked in my head that this method is indeed conformant with how a switch statement executes. switchWithBracketsBreakInInternalLoop was the real kicker, and made me realize handling break statements in their proper scope was missing. By skipping the child tree of any loop or switch the code will not pick up unnecessary breaks of switch control flow.

Performance would be the only open question, although I would not expect this method to be any slower than jumping with goto, as it is really just an unrolled switch statement.

Also send me your code clarity, clean-up, cognitive load lightening, and readability suggestions, please!

function foo() {
    try {
        return "foo";
    } finally {
        return "bar";
    }
}
print(foo());

function foo()
    xpcall(function()
        return "foo"
end,
function(e)
end)
    return "bar"
end
print(foo()); --Prints "bar"

The return statement only returns out of the function passed to xpcall. I'm not totally sure what a good way to deal with this would be.

Closes #654

This PR fully replaces our IIFEs with preceding statements. To do this, it uses a stack based system akin to what is detailed out here: roblox-ts/roblox-ts#537. Statements can be added before the current expression being transformed using context.addPrecedingStatements(). If nodes higher up want to catch and manipulate these statements, they can do so by pushing and popping a statement collection with context.pushPrecedingStatements() and context.popPrecedingStatements().

Significant refactoring was required in a few places to pull this off. Call expression handling has been modified a lot, but should be understandable. Dealing with assignments was particularly tricky, though, as there's a complex web of cases (especially destructuring assignments). At some point I imagine a full rewrite of assignment handling might be needed to keep it understandable and maintainable, but I'm not even sure what that would look like right now 😅

Evaluation Order

The biggest challenge with this system is maintaining evaluation order in statements with multiple expressions. Consider this example:

function foo(a: number, b: number) {
    let i = 0;
    let o = {
        method(...args: unknown[]) { /* ... */ }
    };

    function bar(): number {
        // Could modify a, b, i, o, or o.method!
    }

    o.method(a, b, i += bar());
}

Once i += bar() is lifted into a preceding statement, any expressions before it in the statement could be affected, and thus need to be cached in temps ahead of time to maintain evaluation order.

Unfortunately, without deeper inspection of the preceding statements being generated, we have to aggressively do this caching, even in cases which don't seem to need it upon visual inspection. For example:

declare function foo(x: number): void;
let i = 0;
foo(i++);

Transpiles to:

local i = 0
local ____foo_2 = foo
local ____G_1 = _G
local ____i_0 = i
i = ____i_0 + 1
____foo_2(____G_1, ____i_0)

Both foo and even _G have to be cached in temps because we can't guarantee that the preceding statements generated by ++i won't modify them. It's possible we could add some specific optimizations for cases like this to reduce the temps, but it would be tricky and a bit messy. Also, thanks to operator overloading, it is possible ++i could have actually modified those values.

const variables won't be cached like this however. Those looking to reduce temps should prefer const over let whenever possible.

Sparse Array Lib

To help with situations which could generate extreme numbers of temps, this PR also adds some new lib functions for manipulating sparse arrays. Consider this:

foo(a, b, c, d, e++, f, g);

Instead of caching all of those variables in temps, we can push them into an array while preserving evaluation order and unpack them into the call's arguments. But, since any of these could have nil values, we use a custom array that tracks its true size.

local ____foo_2 = foo
local ____array_1 = __TS__SparseArrayNew(_G, a, b, c, d)
local ____e_0 = e
e = ____e_0 + 1
__TS__SparseArrayPush(____array_1, ____e_0, f, g)
____foo_2(
    __TS__SparseArraySpread(____array_1)
)

This technique can also be used to handle spread expressions in the middle of arguments, so it also fixes #1142 as a bonus.

Right now, argument lists and array expressions which would generate 3 or more temps when transformed will revert to this behavior. That number can be adjusted, if needed.

Other Notes

• Some helpers for generating useful temp names has been added to make the output lua a bit more readable
• Some things like optional chaining could be reworked to take advantage of this, but I'll save those for separate PRs
• This also fixes #1127

Edit - Update

The issue does not have to do with TSTL modules. See my final reply below.

Original

• I use TSTL to make mods for a game.
• Each mod uses a "common" TSTL module for helper functions. (For reference, it is here.)
• However, when I load two mods that both use the same "common" TSTL module, foo instanceof Map stops working properly, returning false when it should be returning true (INSIDE the common module, not in the base mod).
• If I enable mod 1 and disable mod 2, everything works as expected.
• If I enable mod 2 and disable mod 1, everything works as expected.

This is similar to an issue that Perry fixed 22 days ago: https://github.com/TypeScriptToLua/TypeScriptToLua/issues/1061

I know this is pretty vague, so I tried to make a minimal reproducible test case. I sunk two hours into mocking game functionality and made some progress, but I am not able to reproduce the issue via just emulating the game loading two mods with dofile(main1.lua) and dofile(main2.lua). The game works via callbacks, so maybe I am not emulating them well enough.

It is pretty miserable trying to continue this repro attempt, but this is a really critical bug, as I want to be able to use TSTL for all my mods, not just one of them. Can anyone point me in the right direction of how to troubleshoot this and/or provide more information?

Hi I'm having a problem installing the transpiler via npm.

What I did:

sudo npm install -g typescript-to-lua

What I expected to happen:

Successful installation

What happened instead

npm ERR! path /usr/lib/node_modules/typescript-to-lua/dist/index.js
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall chmod
npm ERR! enoent ENOENT: no such file or directory, chmod '/usr/lib/node_modules/typescript-to-lua/dist/index.js'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/radoslaw/.npm/_logs/2018-09-18T13_01_23_621Z-debug.log

Tail of aforementioned log:

1255 silly build [email protected]
1256 info linkStuff [email protected]
1257 silly linkStuff [email protected] has /usr/lib/node_modules as its parent node_modules
1258 silly linkStuff [email protected] is part of a global install
1259 silly linkStuff [email protected] is installed into a global node_modules
1260 silly linkStuff [email protected] is installed into the top-level global node_modules
1261 verbose linkBins [ { tstl: './dist/index.js' }, '/usr/bin', true ]
1262 timing action:build Completed in 12ms
1263 verbose unlock done using /home/radoslaw/.npm/_locks/staging-a072192f34a17023.lock for /usr/lib/node_modules/.staging
1264 timing stage:rollbackFailedOptional Completed in 34ms
1265 timing stage:runTopLevelLifecycles Completed in 1716ms
1266 verbose stack Error: ENOENT: no such file or directory, chmod '/usr/lib/node_modules/typescript-to-lua/dist/index.js'
1267 verbose cwd /home/radoslaw/Documents/nakama/lua-logic-ts
1268 verbose Linux 4.15.0-33-generic
1269 verbose argv "/usr/bin/node" "/usr/bin/npm" "install" "-g" "typescript-to-lua"
1270 verbose node v10.10.0
1271 verbose npm  v6.4.1
1272 error path /usr/lib/node_modules/typescript-to-lua/dist/index.js
1273 error code ENOENT
1274 error errno -2
1275 error syscall chmod
1276 error enoent ENOENT: no such file or directory, chmod '/usr/lib/node_modules/typescript-to-lua/dist/index.js'
1277 error enoent This is related to npm not being able to find a file.
1278 verbose exit [ -2, true ]

Installing it locally doesn't help - it fails with the same error (up to location of node_modules in my project)

What is my environment like

Ubuntu 16.04, npm 6.4.1, node 10.10.0

What I've tried doing besides that

1. Installing locally (mentioned)
2. Installing tsc, node-ts globally and retrying installation
3. Cloning the repo and building from source, which went OK. Unfortunately I don't know what to do next to get a binary or a script, that will act as a transpiler.

Currently I am using namespaces to represent Lua globals with functions inside of them- that is okay, but this is not okay:

/** @noSelf */
declare namespace fs {
	...

	/** Deletes a file or directory. */
	function delete(path: string): void // <---- delete is a reserved word that cannot be used here

	...
}

This is a problem - the function is named delete but TypeScript does not have the capability to represent this!

Merging is not allowed:

And even without type checking, the transpilation only recognizes the namespace declaration above so it doesn't even work correctly:

My only option is to include the function definition inside the namespace, but since function identifiers cannot be strings there is no way to name the function delete, so there needs to be a way to communicate to TypeScriptToLua that the function on the Lua side is named that way.

The issue? There isn't yet such a feature.

I propose an @luaName annotation, which will allow you to specify a Lua name for the function. This would allow me to name the function something different like _delete and then in the transpilation step it is turned back to delete.

Example:

/** @luaName bar */
declare function foo(x: string): void

foo('Hi!')

would transpile to

bar('Hi!')

Likewise,

/** @noSelf */
declare namespace fs {
	/** Deletes a file or directory.
	 * @luaName delete */
	function _delete(path: string): void
}

fs._delete('some path')

would transpile to

fs.delete('some path')

Say we have 2 files:

test1.ts:

import { print } from "test2";

print("Hello, world!");

test2.ts

export declare function print(this: void, ...args: any[]): void;

They will transpile into the following: test1.lua

--[[ Generated with https://github.com/TypeScriptToLua/TypeScriptToLua ]]
local ____exports = {}
local ____test2 = require("test2")
local ____print = ____test2.print
____print("Hello, world!")
return ____exports

test2.lua

--[[ Generated with https://github.com/TypeScriptToLua/TypeScriptToLua ]]
local ____exports = {}
return ____exports

In the lua code, you can see that test1.lua is trying to get a function print from test2.lua yet test2.lua doesn't actually export anything. Is this normal? Is that how TypeScript normally works or is there something wrong here? If so, I would absolutely appreciate it if it could be fixed.

defineProperty affects previously instantiated classes by overriding a property

interface Test {
  a: number;
}

class Test {
  constructor(a: number) {
    Object.defineProperty(this, "a", {
      value: a,
    });
  }
}

const t1 = new Test(1);
print(t1.a); // 1

const t2 = new Test(2);
print(t2.a);  // 2

print(t1.a); // 2

Hi,

I looking for a way to run tstl --watch, and to be able to trigger an action (notify another app that the bundle have changed) each time the code is re-transpiled (after the target bundle is updated).

Is there an API for this?

(I guess I can write an external watch for the target bundle)

As stated in Discord:

// these should all create the same result
const [ a1, a2, a3, a4 ] = [...$vararg] as string[];
const [ b1, b2, b3, b4 ] = [...$vararg] as LuaMultiReturn<[string, string, any[], object]>;

// this creates the expected result
const [ c1, c2, c3, c4 ] = [...$vararg];

see Playground 🙌

This function would run after Lua files have been written to disk (in order to allow plugins to perform post-build tasks such as optimizing/minifying/obfuscating/bytecode-compilation/etc).