Z. Transcompile to Go language (exploratory stage)

For information on transcompiling to Go language, please refer to the following article.

../go

The information here is old, but I'll keep it for reference.

Considering transcompiling from LuneScript to Go language.

This is where you record what you are discussing.

aim

The goals of transcompiling from LuneScript to Go language are as follows.

Speeding up LuneScript
- LuneScript supports the code completion function, but it becomes heavy and inconvenient for large-scale code.
- Improve the usability of LuneScript's code completion feature by speeding it up.
Improved efficiency of LuneScript version upgrade work by speeding up
- LuneScript's self-hosted build/test time currently takes less than 2 minutes.
- It takes 2 minutes for each change, so I try to speed it up with Go.
- We were working on transcompiling to C language, but the implementation of GC in C language did not proceed as expected, so Go, which supports GC at the language level, will be supported first.

value

Types in LuneScript	Go types	type in Lns Random for Go
nil, null	interface{}	nil
int	int	LnsInt (alias)
real	float64	LnsReal(alias)
str	string	LnsStr (alias)
bool	bool	LnsBool(alias)
List	unique structure	LnsList
Array	arrangement	LnsArray
Map	map	LnsMap
Set	map (put item in key)	LnsSet
class	Structure
interface	interface
form	func	LnsForm
enum	int/float64/string	LnsEnum
stem	interface{}	LnsStem
nilable	interface{}

Handling int/real

LuneScript int/real is defined using go's type alias as follows:

type LnsInt = int
type LnsReal = float64

Handling nilables

LuneScript's nilable cannot be treated as the original value as is. However, you can still check for equality. This is the same as go and LuneScript.

Dealing with boolean values

LuneScript is false for nil and false and true otherwise. Since go requires judgment by true/false, prepare a condition conversion function for LuneScript.

Handling and/or

LuneScript's and/or is not a logical operation, it controls the evaluation of expressions, and the evaluation result is not bool. go evaluates to bool.

Handling of generics

Since go does not have generics, all types of generics are handled with interface{}.

Dealing with Sets

Since go doesn't have Set, we use Map instead.

return multiple values

Multi-value returns in go and LuneScript behave differently.

When hoge() is a multi-value return function that returns x and y, the behavior is as follows.

code	LuneScript expansion result	go expansion result
(hoge())	x	x, y
hoge(), val	x, val	x, y, val

To realize the above difference, go prepares the following conversion functions.

// 多値返却の先頭 int を返す
func carInt( multi ...interface{} ) int {
    if len( multi ) == 0 {
        panic( "nothing" )
    }
    return multi[0].(int)
}
// 多値返却の先頭 int! を返す
func carIntN( multi ...interface{} ) interface{} {
    if len( multi ) == 0 {
        return nil
    }
    if multi[0] == nil {
        return nil
    }
    return multi[0].(int)
}

classes and inheritance

Go has structs and receivers, but no inheritance.

LuneScript has inheritance, so we need to implement inheritance in Go.

Shows how to support the following LuneScript classes in Go.

// @lnsFront: ok
interface IF {
  pub fn sub1():int;
}
class Parent {
   let val1:int;
   pub fn sub1():int {
      return self.val1;
   }
}
class Sub extend Parent {
   let val2:int;
   pub override fn sub1():int {
      return self.val2;
   }
   pub fn sub2():int {
      return self.val2;
   }
}
class SubSub extend Sub {
   let val3:int;
   pub override fn sub1():int {
      return self.val3;
   }
   pub fn sub3():int {
      return self.val3;
   }
}

Equivalent code in Go

Show equivalent code in Go

package main

import "fmt"

type ParentMtd interface {
    sub1 () int
}
type Parent struct {
    val1 int
    FP ParentMtd
}
type ParentDownCast interface {
    ToParent() *Parent
}

func (obj *Parent ) ToParent() *Parent {
    return obj
}


func (self *Parent) sub1() int {
    return self.val1
}

func NewParent(val1 int) *Parent {
    parent := Parent{ val1, nil }
    parent.FP = &parent
    return &parent
}

type SubMtd interface {
    ParentMtd
    sub2 () int
}

type Sub struct {
    Parent
    val2 int
    FP SubMtd
}
type SubDownCast interface {
    ToSub() *Sub
}

func (obj *Sub ) ToSub() *Sub {
    return obj
}


func (self *Sub) sub1() int {
    return self.val2
}
func (self *Sub) sub2() int {
    return self.val2
}

func newSub(val1,val2 int) *Sub {
    sub := Sub{ Parent{ val1, nil }, val2, nil }
    sub.Parent.FP = &sub
    sub.FP = &sub
    return &sub
}


type SubSubMtd interface {
    SubMtd
    sub3 () int
}

type SubSub struct {
    Sub
    val3 int
    FP SubSubMtd
}
type SubSubDownCast interface {
    ToSubSub() *SubSub
}

func (obj *SubSub ) ToSubSub() *SubSub {
    return obj
}
func (obj *SubSub ) ToSub() *Sub {
    return &obj.Sub
}

func (self *SubSub) sub1() int {
    return self.val3
}
func (self *SubSub) sub2() int {
    return self.Sub.sub2()
}
func (self *SubSub) sub3() int {
    return self.val3
}


func newSubSub(val1,val2,val3 int) *SubSub {
    subsub := SubSub{ Sub{ Parent{ val1, nil }, val2, nil }, val3, nil }
    subsub.Parent.FP = &subsub
    subsub.Sub.FP = &subsub
    subsub.FP = &subsub
    return &subsub
}

func testParent( obj *Parent ) {
    fmt.Println( obj.FP.sub1() )
}

func testSub( mess string, obj *Sub ) {
    fmt.Println( mess, obj.FP.sub1(), obj.FP.sub2() )
}

func testCast( obj *Parent ) {
    cast, ok := obj.FP.(SubDownCast)
    if ok {
        testSub( "cast", cast.ToSub() )
    } else {
        fmt.Println( "cast NG" )
    }
    
}

func Lns_init() {
    subsub := newSubSub( 1, 2, 3 )
    fmt.Println( subsub.val1, subsub.val2, subsub.val3 )
    fmt.Println( subsub.FP.sub1(), subsub.FP.sub2(), subsub.FP.sub3() )
    testSub( "subsub.Sub", &subsub.Sub )
    testParent( &subsub.Parent )
    testCast( &subsub.Parent )

    sub := newSub( 1, 2 )
    testSub( "sub", sub )
    testParent( &sub.Parent )
    testCast( &sub.Parent )

    testCast( NewParent( 1 ) )
}

Inheritance implementation method

Parent class

First, we will explain the Parent class.

// @lnsFront: ok
class Parent {
   let val1:int;
   pub fn sub1():int {
      return self.val1;
   }
}

data structure

Define the following structure and interface to represent the Parent class.

type ParentMtd interface {
    sub1 () int
}
type Parent struct {
    val1 int
    FP ParentMtd
}
type ParentDownCast interface {
    ToParent() *Parent
}
func (obj *Parent ) ToParent() *Parent {
    return obj
}

The ParentMtd interface is responsible for
- Define a method in the Parent class
- Express the morphism of the Parent class
A Parent struct has a member and a ParentMtd
ParentDownCast is defined per class for downcasting

method

Define the following receiver function to represent the method of the Parent class.

func (self *Parent) sub1() int {
    return self.val1
}

constructor

Define the following as a constructor of Parent class.

func NewParent(val1 int) *Parent {
    super := &Parent{ val1, nil }
    super.FP = super
    return super
}

This constructor does the following:

member initialization
FP settings

How to use the Parent class

Parent is used like this:

parent := NewParent( 1 )
print( parent.FP.sub1() )

When calling a method, always call it through the FP interface.

Sub class

Describe the Sub class.

// @lnsFront: skip
class Sub extend Parent {
   let val2:int;
   pub override fn sub1():int {
      return self.val2;
   }
   pub fn sub2():int {
      return self.val2;
   }
}

data structure

Define the following structure and interface to represent the Sub class.

type SubMtd interface {
    ParentMtd
    sub2 () int
}
type Sub struct {
    Parent
    val2 int
    FP SubMtd
}
type SubDownCast interface {
    ToSub() *Sub
}
func (obj *Sub ) ToSub() *Sub {
    return obj
}
func (obj *Sub ) ToParent() *Parent {
    return &obj.Parent
}

The SubMtd interface declares the methods defined in Sub.
- Do not include Parent methods
The Sub structure declares the data of the Parent structure and the members defined in Sub.

method

Define the following receiver function to represent the method of the Sub class.

func (self *Sub) sub1() int {
    return self.val2
}
func (self *Sub) sub2() int {
    return self.val2
}

constructor

Define the following as a constructor of Sub class.

func newSub(val1,val2 int) *Sub {
    sub := &Sub{ Parent{ val1, nil }, val2, nil }
    sub.Parent.FP = sub
    sub.FP = sub
    return sub
}

This constructor does the following:

member initialization
FP settings
- FP of super is also set here
- Set the FP of this super to &sub instead of &super to achieve polymorphism

SubSub class

Describe the SubSub class.

// @lnsFront: skip
class SubSub extend Sub {
   let val3:int;
   pub override fn sub1():int {
      return self.val3;
   }
   pub fn sub3():int {
      return self.val3;
   }
}

data structure

Define the following structure and interface to represent the SubSub class.

type SubSubMtd interface {
    SubMtd
    sub3 () int
}

type SubSub struct {
    Sub
    val3 int
    FP SubSubMtd
}
type SubSubDownCast interface {
    ToSubSub() *SubSub
}
func (obj *SubSub ) ToSubSub() *SubSub {
    return obj
}
func (obj *SubSub ) ToSub() *Sub {
    return &obj.Sub
}
func (obj *SubSub ) ToParent() *Parent {
    return &obj.Parent
}

The SubSubMtd interface declares the methods defined by SubSub.
- Do not include Sub methods
The SubSub structure declares the data of the Sub structure and the members defined by SubSub.

method

Define the following receiver function to represent the methods of the SubSub class.

func (self *SubSub) sub1() int {
    return self.val3
}
func (self *SubSub) sub2() int {
    return self.Sub.sub2()
}
func (self *SubSub) sub3() int {
    return self.val3
}

Method definition without overriding

It should be noted that the sub2() method calls self.Sub.sub2().

The SubSub class does not override the sub2 method. In other words, the sub2 method of SubSub uses the method of the Sub class. So we are calling the Sub.sub2 method.

constructor

Define the following as a constructor of SubSub class.

func newSubSub(val1,val2,val3 int) *SubSub {
    subsub := &SubSub{ Sub{ Parent{ val1, nil }, val2, nil }, val3, nil }
    subsub.Parent.FP = subsub
    subsub.Sub.FP = subsub
    subsub.FP = subsub
    return subsub
}

This constructor does the following:

member initialization
FP settings
- Also set the FP of Parent and Sub here
- Implement polymorphism by setting &subsub to FP of Parent and Sub

IF interface

// @lnsFront: ok
interface IF {
  pub fn sub1():int;
}

data structure

LuneScript's interface uses Go's interface as it is.

interface IF {
  pub fn sub1():int;
}

method call

To call a method of the Parent class, do the following:

func test(parent *Parent) int {
  print( parent.FP.sub1() )
  print( parent.sub1() )
}

Difference between parent.FP.sub1() and parent.sub1()

There are two patterns for method calls:

parent.FP.sub1()
- Method calls that support polymorphism
parent.sub1()
- Method call defined in Parent class
  - No support for polymorphism

overhead

Method calls that support polymorphism have a large overhead.
Method calls that support polymorphism should be limited to cases where polymorphism is necessary.
Whether or not polymorphism is required is currently not defined in LuneScript.
- It is necessary to introduce final declarations for classes and methods so that polymorphism can be clearly stated.

up-cast / down-cast

up-cast is achieved by accessing the embedded pointer
- up-casting to an interface uses the interface type held by the object
A down-cast implements an interface with a type assertion.
- Define a DownCast interface for each class, cast to that interface, and then execute the cast function to the target class

var ifObj IF = obj.FP // インタフェースをセットする
parent := &obj.Parent // アップキャスト
(parent.FP.(SubDownCast)).ToSub() // obj を Sub にダウンキャストする

Summary of Classes

Declare an interface that defines the methods of the class
- Embed the interface of the method defined in the Super class

type TestMtd interface {
    SuperMtd
    method() int
}

Declare a struct to hold the members of the class and the above interface
- Inheritance embeds the inheriting type

type Test struct {
    Super
    val int
    FP TestMtd
}

Define interface for downcast

type TestDownCast interface {
    ToTest() *Test
}

Define a method for downcasting
- This method declares everything for the Super class

func (obj *SubSub) ToSub() *Sub {
    return &obj.Sub
}

Declare receivers that define the behavior of the methods of the class
- Declare the receiver including the methods of the Super class
- A non-overridden function calls the method of the struct that defines it

func (self *Test) method() int {
    return self.super.method()
}

Initialize members and interface FP in the constructor
- interface initializes including the interface FP of the Super class
Method calls are called via interface FP
- Method calls without polymorphism call the struct's methods directly

obj.FP.method() // ポリモーフィズム有効
obj.method()    // ポリモーフィズム無効

up-casting is accomplished by accessing the member's Super class pointer
- up-casting to an interface uses the interface type held by the object
down-cast implements an interface with a type assertion and an interface for downcasting.

var ifObj IF = obj.FP // インタフェースをセットする
super := &obj.super // アップキャスト
(parent.FP.(SubDownCast)).ToSub() // obj を Sub にダウンキャストする

The interface uses Go's interface as it is
- Use interface FP when up-casting from a class object to an interface

symbol name

Symbol names in LuneScript and go differ significantly in the following ways.

namespace
- LuneScript is in the same file (module)
- go is in the same directory (package)
Public/private control method
- LuneScript is controlled by pub/pro etc.
- go controls the case of the first letter of a symbol

This difference causes the following problems:

When defining a symbol sym with the same name in different files FileA.lns and FileB.lns in LuneScript, when converting this to go, if you define a symbol sym with the same name in FileA.go and FileB.go with the same configuration , the symbol sym will result in a duplicate definition error.
Symbols defined publicly in lowercase in LuneScript become private in go.
- Symbols defined privately in uppercase in LuneScript are public in go.

To work around this problem, handle symbol names as follows:

Add the file name to the beginning of the symbol of the function or class that is subject to public/private control. Add G (G for GLOBAL) if it is public, and l (l for local) if it is private.

In other words, if you convert the following LuneScript source to go,

// @lnsFront: ok
fn func() {
}
pub class Class {
   let val1:int;
   pub let val2:int;
}

The relationship between LuneScript and go symbols is as follows.

public/private	lns	go
private	func	lfile_func
Release	Class	Gfile_Class
private	val1	lval1
Release	val2	Gval2

Arguments and local variables are basically converted as they are because there is no difference in scope between LuneScript and go.

Lua VM

In the current LuneScript, Lua VM is used during Macro expansion. There are two ways to use the Lua VM with Go:

Use gopher-lua, a port of Lua to Go
use liblua

Using gopher-lua makes dealing with the Lua VM easier, but it has the following limitations:

Lua VM version becomes Lua5.1
Slow compared to liblua
- According to the information on the official Wiki (<https://github.com/yuin/gopher-lua/wiki/Benchmarks>), the execution time of fib(35) is next.
  
  lua5.1.4
  
  1.71sec
  
  Gopherlua
  
  5.40sec

From the above, the LuneScript transcompiler uses liblua.

cgo

Use cgo to use liblua from Go.

cgo is a package for calling C language libraries from Go.

The C code written in the comment before import "C" is parsed and expanded into a C package so that it can be accessed by Go, like so:

// #include <stdlib.h>
// #cgo CFLAGS: -I/usr/include/lua
// #cgo LDFLAGS: -ldl -lm -llua
// #include <lauxlib.h>
// #include <lualib.h>
import "C"

import "unsafe"

// lua のコードを実行する
func lua_runScript( script string ) {
    var vm * C.lua_State = C.luaL_newstate()
    if vm == nil {
        return
    }
    defer C.lua_close( vm )
    
    C.luaL_openlibs( vm )

    block := C.CString( script )
    defer C.free( unsafe.Pointer( block ) )
    
    C.luaL_loadstring( vm, block )
    C.lua_pcallk( vm, 0, C.LUA_MULTRET, 0, 0, nil )
}

func main() {
   lua_runScript( "print( 'hello world' )" )
}

Since cgo does not support #define macro functions, functions with macro definitions such as the following must be expanded and processed by yourself.

#define luaL_dostring(L, s) \
	(luaL_loadstring(L, s) || lua_pcall(L, 0, LUA_MULTRET, 0))