Another Lua-based language implementation

[Kingdaro]

I've decided to put this on hold, but the repository is here for anyone still interested.

So I wanted to see if I could address a few of Lua's shortcomings using a different language implementation of my own, and LunaScript (possibly temporary title) is the result of that.

LunaScript on GitHub

What LS brings to the table:

• #1 Rule: everything is an expression, and every language construct has a return value. This adds a lot of flexibility as shown in the examples.
• Assignment operators, +=, -=, etc.
• Automatic variable localization
• Require helpers
• More convenient, less awkward looping constructs
• Keeps the spirit of Lua as a simple, easy to use, easy to read and well-structured language.

The Luna compiler is still a work in progress, as is the language draft, so there's no compiler yet. Just a bunch of badly written incomplete parser code. I'd show more examples, but most of them would just look like regular Lua, which is the goal.

If you're interested, opinions on anything that should be included or excluded from the language are appreciated. Sidenote, I'm not experienced in either language design nor implementation, so a forewarning that half the time, I probably won't know what I'm talking about.

[Kingdaro]

And another post for an issue about the result of looped expressions, like for, while, and repeat. Opinions on this would be great as well. https://github.com/Kingdaro/lunascript/issues/1

[davisdude]

Hey, I just thought of something that could potentially be a problem: variable scope
How does it handle things like this:

Code: Select all

tab = { 1, 2, 3, 'foo' }

function checkThing( x, y, z )
    local remove = false
    for i = 1, #tab do
        remove = remove or x == tab[i]
    end
    if remove then table.remove( tab, 1 )
end

[Kingdaro]

Like this:

Code: Select all

local tab
tab = { 1, 2, 3, 'foo' }

function checkThing( x, y, z )
    local remove = false
    for i = 1, #tab do
        remove = remove or x == tab[i]
    end
    if remove then table.remove( tab, 1 )
end

Here's a more complex example:

Code: Select all

--> (lua) local a, b
a = 5
b = 10

do
  --> (lua) local b
  a = 15
  local b = 20
end

print(a, b) --> 15, 10

Any variable that's assigned that's not found in the scope above will be localized within its block. At some point, I'll probably have writeups on scope, return values of expressions and other stuff. A reference, pretty much.

[pgimeno]

Is your aim to retain compatibility with Lua, at least for the not very advanced features? I see problems with that.

[airstruck]

I see problems with that.

The thing that really jumps out at me is the idea of moving back to pre-5.0-style table iteration. Before 5.0, "for k, v in t" did work as described here (except with key first and value second). In 5.0, that syntax still worked but was undocumented. A table with a __call metamethod was still treated like a table. Post-5.0, for..in only takes a callable iterator, so the behavior of a callable table is not ambiguous (it's treated like a function). What happens in this language/dialect when a callable table is the subject of a for..in loop?

I'd also question the usefulness of some of the other proposals. The idea of all statements and blocks being treated as expressions that resolve to some value is interesting, but the benefits don't seem to justify the added complexity (at least, not from looking at the examples).

For example, this:

Code: Select all

letters = for char in "hello":gmatch('.') do
    assign char
end

Is it really a huge improvement over this?

Code: Select all

local letters = {}
for char in ('hello'):gmatch('.') do
    letters[#letters + 1] = char
end

Sure, it's a few less keystrokes, but is it really worth the trouble of inventing a new language to save those keystrokes? If this is a common pattern, it should pretty easy to write a little helper function:

Code: Select all

function tabulate (f)
    local t, v = {}, f()
    repeat t[#t + 1] = v; v = f() until v == nil
    return t
end

...

local letters = tabulate(('hello'):gmatch('.'))

I feel the same way about the proposed "import," I get that it's saving a few keystrokes but it just doesn't seem worth the hassle of inventing a new language or adding new syntax to Lua. Also, the idea that import 'lib.inspect' would implicity create a new local variable named "inspect" feels like a step in the wrong direction. I want my variables to be defined explicitly, not implicitly. What happens if I do this?

Code: Select all

local classModule = I_AM_BARTBES and 'lib.slither' or 'lib.30log'
import classModule
-- I just declared a variable, but I'm not sure what it is.

By the same token, the implicit local scoping rules seem to carry a higher cognitive overhead than Lua's rules. If I see something like somevar = {}, I have to look at every outer scope (in the worst case) to figure out whether that variable is local to the scope where the assignment was made, or local to an outer scope. If I didn't find it in an outer scope, it could still refer to a global variable, so now I have to look through the entire project. With Lua's syntax, I know immediately if something is local, and if it doesn't have the local keyword, I know it's from an outer scope (including the global scope). I strongly prefer explicit scoping of variables over implicit (I'm not a fan of implicit globals in Lua either, but it's easily resolved by locking the global table).

I'd probably find the list comprehension stuff to be useful, but there's no way I'd move from pure Lua to a transpiled language just for that. The added build step, and the headache of dealing with debug, source maps and so on make it pretty hard to justify. I'd rather just use small helper functions to do the same thing.

Anyway, it's interesting to think about this kind of stuff, thanks for sharing the ideas. Sorry if this comment sounded overly negative, but I assume you posted this because you wanted feedback, so I figured I'd share my thoughts.

[Kingdaro]

Is your aim to retain compatibility with Lua, at least for the not very advanced features? I see problems with that.

Not really looking for compatibility here. Just wanting it to look like Lua, since I like the way Lua looks, but not all of how it functions.

I listed compatibility on the README, but I'll just go ahead and throw that out the window right now, because that seems pretty much impossible if I want to move forward at all without adding a trillion keywords.

I did want feedback, and I appreciate it. I should mention that this is more of a hobby than anything else, since I've just had a growing interest in language implementations as of recent, which made me wonder the question of: if I could change one or more things about the language I like the most, which would those be? That's mostly irrelevant though, but some background, so I don't try to make it seem like I'm trying to break any new ground in the Lua world, or anything like that. I'd honestly be surprised if this made it anywhere past the translation stage.

Another thing I want to point out, is that it's less about "saving keystrokes" more than it is just making it look a little easier to read on the eyes, and to add some functionality that just seems annoyingly missing in some areas. Even then, saving keystrokes on small things makes a big difference over time.

The assign proposal is just an option of many, as I'm still mostly trying to figure out, if going forward with 'everything is an expression', what kind of sensible value a loop should return, and the general consensus is a table. The implementation details of that are the tricky part, so your point isn't invalid. I've thought about just tossing all loops out the window if I somehow could, and just using one statement for iteration, somehow. Maybe. But, again, one of many options. Have yet to arrive at what seems most fitting for Luna.

The import syntax is just there because it, in my opinion, just looks and reads a hell of a lot nicer than a typical require. The rules on what it does would be documented explicitly, and you'd know what was being assigned. Optionally, I could have a statement similar to JavaScript's import, but I didn't want to look to anything too verbose, and I wanted something that could still be made sense out of.

I share your opinions on locals, though. 100%, really. I've always favored explicit scope declarations. What I haven't favored is having to cross-declare functions or other variables at the top of a scope for any other function to be able to access them.

Code: Select all

local function foo()
  bar = bar + 1
end

local bar = 0
foo()

This is a simple example that's probably better fixed by passing bar as a parameter, but this problem crops up a lot in Lua.

In that regard, moving to auto-scoped variables was maybe not the best idea, then. I can still have explicit locals, but instead just have them all automatically localized at the top still. That seems alright.

All that aside, the biggest challenge is definitely justifying the move from just pure Lua to a transcompiler, for problems that can be solved through other simple means in Lua alone. Still, that's where the improvement of the language itself comes in. Optionally, it could just become its own thing? Who knows. ¯\_(ツ)_/¯

[pgimeno]

Not really looking for compatibility here. Just wanting it to look like Lua, since I like the way Lua looks, but not all of how it functions.

I listed compatibility on the README, but I'll just go ahead and throw that out the window right now, because that seems pretty much impossible if I want to move forward at all without adding a trillion keywords.

One major feature of Lua that almost every Lua programmer uses, and that heavily influences how it looks, is that there's no need to have a separator between statements. There's one exception to that rule, because the syntax is ambiguous as-is and a statement separator breaks the ambiguity. I think that introducing "assignment is an expression" risks introducing further ambiguities.

If you know BNF, I'd advise you to try to develop your grammar in a format that a parser generator like Bison accepts, regardless of whether you implement it with that tool or on your own, because these tools inform you of any ambiguities in the language. That will let you have a formal expression of the grammar, like the one in the Lua manual: http://www.lua.org/manual/5.1/manual.html#8 (though that one isn't complete because as it says, it doesn't list operator precedence).

[Kingdaro]

One major feature of Lua that almost every Lua programmer uses, and that heavily influences how it looks, is that there's no need to have a separator between statements. There's one exception to that rule, because the syntax is ambiguous as-is and a statement separator breaks the ambiguity. I think that introducing "assignment is an expression" risks introducing further ambiguities.

I'm aware of this. I've done the best I can to avoid ambiguities in the current syntax definition of the language.

If you know BNF, I'd advise you to try to develop your grammar in a format that a parser generator like Bison accepts, regardless of whether you implement it with that tool or on your own, because these tools inform you of any ambiguities in the language. That will let you have a formal expression of the grammar, like the one in the Lua manual: http://www.lua.org/manual/5.1/manual.html#8 (though that one isn't complete because as it says, it doesn't list operator precedence).

I actually have defined the grammar of the language in BNF, in the syntax.md. But thanks anyway, heh. I was going to link that file at some point in the readme, as to maybe make things a little clearer.

[pgimeno]

I missed that, sorry. Anyway, my point was about having it automatically checked, by making it a valid input file for e.g. Bison.