tech

[English] [Japanese]

80. Transcompiling to Go Language

LuneScript can be transcompiled to Go as well as Lua as a transcompiled language.

With this feature, LuneScript's self-hosting code is transcompiled into go language code and built with go to speed up LuneScript (approximately 16 times).

See below for acceleration. https://ifritjp.github.io/blog2/public/posts/2021/2021-06-28-lunescript-build-time-2000/

However, since the support is limited to the functions required for LuneScript self-hosting, some functions are not supported.

However, LuneScript's self-hosting itself uses most of LuneScript's features, so it can handle normal usage without any problems.

How to transcompile to go language

To transcompile to go language, by specifying -langGo at LuneScript command line option save or SAVE, output the transcompile result to go to a file with .go extension.

Working with Lua

When transcompiling to the go language, you need to use special types when working with Lua.

See below for details.

../lua

Configuration after transcompiling to Go language

LuneScript is a transcompiler to Lua, and has the advantage of being able to use Lua code directly from LuneScript.

This feature is inherited even after transpiling to Go language.

For example, the following code will work even after converting to Go language.

// @lnsFront: ok
let step = 30;

// Lua のフィボナッチ関数
let code = ```
local function fib ( n )
   if n < 2 then return n end
   return fib(n - 2) + fib(n - 1)
end
return fib
```;

// Lua コードのロード
form fibFunc( num:int ):int;
fn loadLua( txt:str ) : Luaval<fibFunc> {
   let fib;
   __luago {
      let loaded = unwrap _load( txt ## );
      fib = unwrap loaded(##);
   }
   return fib@@fibFunc;
}

// プロファイル用マクロ
macro _profile( exec:__exp ) {
   {}
   {
      let prev = os.clock();
      print( "%s = %d: step = %d: time = %g"
             (,,,,exec, ,,exec, step, os.clock() - prev ) );
   }
}

// Lua のコードをロードして実行
let fib_lua = loadLua( code );
__luago {
   _profile( fib_lua( step ) );
}

// LuneScript のフィボナッチ関数
fn fib_lns( num:int ) : int {
   if num < 2 {
      return num;
   }
   return fib_lns( num - 2 ) + fib_lns( num - 1 );
}
// Go で定義した fib_lns 関数を実行
_profile( fib_lns( step ) );

This code does Fibonacci calculations with lua and LuneScript.

The first code variable defines the lua code, the loadLua() function loads it, and the loaded Lua function is fib_lua. And the fib_lns() function in the second half is a LuneScript function that performs Fibonacci calculations.

Transcompiled to Lua

Transpiling the above code to Lua and running it will output:

fib_lua( step ) = 24157817: step = 37: time = 5.69
fib_lns( step ) = 24157817: step = 37: time = 5.38

This shows the calculation result and calculation time of fib_lua() and fib_lns() respectively.

Looking at this, you can see that the execution time of function fib_lua() loaded by Lua and function fib_lns() of Lns takes almost the same time.

This is a natural result. This is because when LuneScript is converted to Lua, fib_lns() written in LuneScript and fib_lua() written in Lua are almost the same Lua code.

When the above code is transcompiled to Lua, the code looks like this:

--miniGo.lns
local _moduleObj = {}
local __mod__ = '@miniGo'
local _lune = require( "lune.base.runtime2" )
if not _lune2 then
   _lune2 = _lune
end
local step = 37
local code = [==[
local function fib ( n )
   if n < 2 then return n end
   return fib(n - 2) + fib(n - 1)
end
return fib
]==]
local function loadLua( txt )
   local loaded = _lune.unwrap( _lune.loadstring52( txt ))
   local fib = _lune.unwrap( loaded(  ))
   return fib
end
local fib_lua = loadLua( code )
do
   local prev = os.clock(  )
   print( string.format( "%s = %d: step = %d: time = %g", "fib_lua( step )", fib_lua( step ), step, os.clock(  ) - prev) )
end
local function fib_lns( num )
   if num < 2 then
      return num
   end
   return fib_lns( num - 2 ) + fib_lns( num - 1 )
end
do
   local prev = os.clock(  )
   print( string.format( "%s = %d: step = %d: time = %g", "fib_lns( step )", fib_lns( step ), step, os.clock(  ) - prev) )
end
return _moduleObj

Here's an excerpt from the notable fib_lns() function definition:

local function fib_lns( num )
   if num < 2 then
      return num
   end
   return fib_lns( num - 2 ) + fib_lns( num - 1 )
end

As you can see, fib_lns() and fib_lub() have equivalent code.

So it's no surprise that fib_lns() and fib_lua() take about the same amount of time.

When transcompiled to go

On the other hand, when transpiling to go , the execution result is:

fib_lua( step ) = 24157817: step = 37: time = 6.07
fib_lns( step ) = 24157817: step = 37: time = 0.34

You can see that fib_lns() takes about 1/18 less time than fib_lua().

Here is the result of transcompiling to go:

// This code is transcompiled by LuneScript.
package lnsc
import . "lnsc/lune/base/runtime_go"
var init_miniGo bool
var miniGo__mod__ string
var miniGo_step LnsInt
var miniGo_code string
var miniGo_fib_lua *Lns_luaValue
type miniGo_fibFunc_1003_ func (arg1 LnsInt) LnsInt
// 14: decl @miniGo.loadLua
func miniGo_loadLua_1009_(txt string) *Lns_luaValue {
    var loaded *Lns_luaValue
    loaded = Lns_unwrap( Lns_car(Lns_getVM().Load(txt, nil))).(*Lns_luaValue)
    var fib LnsAny
    fib = Lns_unwrap( Lns_car(Lns_getVM().RunLoadedfunc(loaded,Lns_2DDD([]LnsAny{}))[0]))
    return fib.(*Lns_luaValue)
}

// 36: decl @miniGo.fib_lns
func miniGo_fib_lns_1025_(num LnsInt) LnsInt {
    if num < 2{
        return num
    }
    return miniGo_fib_lns_1025_(num - 2) + miniGo_fib_lns_1025_(num - 1)
}

func Lns_miniGo_init() {
    if init_miniGo { return }
    init_miniGo = true
    miniGo__mod__ = "@miniGo"
    Lns_InitMod()
    miniGo_step = 37
    miniGo_code = "local function fib ( n )   if n < 2 then return n end   return fib(n - 2) + fib(n - 1)endreturn fib"
    miniGo_fib_lua = miniGo_loadLua_1009_(miniGo_code)
    {
        var prev LnsReal
        prev = Lns_getVM().OS_clock()
        Lns_print([]LnsAny{Lns_getVM().String_format("%s = %d: step = %d: time = %g", []LnsAny{"fib_lua( step )", Lns_getVM().RunLoadedfunc(miniGo_fib_lua,Lns_2DDD(miniGo_step))[0].(LnsInt), miniGo_step, Lns_getVM().OS_clock() - prev})})
    }
    
    {
        var prev LnsReal
        prev = Lns_getVM().OS_clock()
        Lns_print([]LnsAny{Lns_getVM().String_format("%s = %d: step = %d: time = %g", []LnsAny{"fib_lns( step )", miniGo_fib_lns_1025_(miniGo_step), miniGo_step, Lns_getVM().OS_clock() - prev})})
    }
    
}
func init() {
    init_miniGo = false
}

Since it is very difficult to read, the definition part of the fib_lns() function is extracted as follows.

func miniGo_fib_lns_1025_(num LnsInt) LnsInt {
    if num < 2{
        return num
    }
    return miniGo_fib_lns_1025_(num - 2) + miniGo_fib_lns_1025_(num - 1)
}

Although the function name is long, you can see that the LuneScript code is converted to go as it is. When executing a function, just call the function normally as follows.

 miniGo_fib_lns_1025_(miniGo_step)

On the other hand, Lua's fib_lua() function is loaded by miniGo_fib_lua = miniGo_loadLua_1009_(miniGo_code) with a function for executing Lua code, and when executing it, the function is called as follows.

Lns_getVM().RunLoadedfunc(miniGo_fib_lua,Lns_2DDD(miniGo_step))

As you can see, fib_lns() and fib_lua() are completely different.

build

The following steps are required to transcompile and build to go.

  • Generate go.mod
  • Register the LuneScript runtime with go.mod
  • Generate main.go
  • Generate .go from .lns
  • Update go.mod
  • go build

If you have updated .lns, repeat the process starting with "Generate .go from .lns".

I will explain each step.

Generate go.mod

Execute the following command in the directory where lune.js is located.

$ go mod init test # <--- test は環境に合せて指定してください

Register the LuneScript runtime with go.mod

Register the LuneScript runtime in go.mod with the following command.

$ go get github.com/ifritJP/LuneScript/src@latest

You'll run go mod tidy in the following steps, but be sure to run go get first.

Generate main.go

go requires the entry function main().

The following command generates main.go which defines the main() function of go.

$ lnsc hoge.lns mkmain main.go

where hoge.lns specifies the main module which defines __main in lns. main.go specifies the output path.

A module that defines a __main function is required when transpiling to go.

The main.go generated by this command contains code to initialize the lns runtime.

Generate .go from .lns

Generate .go with the following command.

$ lnsc hoge.lns save -langGo

If the main module contains the __main function, add the –main option.

$ lnsc hoge.lns save -langGo --main hoge

where hoge in –main hoge is the import path of the main module.

For example, if your main module is foo/bar/hoge.lns,

$ lnsc foo/bar/hoge.lns save -langGo --main foo.bar.hoge

becomes.

A __main function is required when transcompiling LuneScript to go.

Update go.mod

Update go.mod with the following command.

$ go mod tidy

go build

After converting all lns files and generating main.go, run go build.

$ go build

If the above gives an error, try the following.

$ go build -tags gopherlua

This will build the transcompiled module in go.

Need to link Lua library after transcompiling to go

追記

Lua ライブラリのリンクを回避する方法を用意しました。

../lua_runtime

As mentioned above, LuneScript works closely with Lua. And even after transcompiling to go, it still supports that interaction.

In order for this interaction to work, the code transcompiled to go needs the lua VM to do the lua work.

To be more precise, in addition to this interaction with Lua, LuneScript also has operations that require the Lua VM. Specifically, Lua VM is used for processing macro expansion and processing some built-in functions.

That lua VM assumes the official lua-5.3.4 and links liblua5.3.so .

The go language has the advantage of being able to generate a single, environment-independent executable, but unfortunately transcompiling LuneScript to go requires linking to liblua5.3.so .

Note that not only liblua5.3.so is required at runtime, but also include files for lua-5.3.4 are required at build time.