tech

[English] [Japanese]

24. Macro

This time, I will explain about LuneScript macros.

macro

LuneScript supports macros.

Maybe it's easier to say "metaprogramming" than "macro" these days.

To explain "what is a macro", I think it will be easier to understand if you explain the difference from a function.

A function is a collection of operations. On the other hand, macros can define multiple function definitions themselves collectively. Of course, what can be defined as a macro is not only function definitions, but almost all processes can be defined.

I think Lisp is the most major programming language for macros, but LuneScript macros are not as advanced as Lisp. However, they are not as restrictive as C language macros.

Macro basics

Macros are expanded at compile time, not at run time.

It's difficult to write a macro if you don't keep this in mind, so be sure to keep this in mind when writing a macro.

Simple macro example

Here's a simple macro example:

// @lnsFront: ok
macro _Hello() {
   print( "hello world" );
}
_Hello(); // hello world

This is the macro _Hello that prints hello world.

Macro definitions use the macro keyword.

This example is a macro that is no different from using a function definition, so there is no point in defining it as a macro.

However, it is a good example to convey that "macros can be defined in the same way as function definitions".

In programming languages with macros, macro definitions often require special processing unlike general function definitions.

As a result, the hurdles of the mind rise, saying, "Macro seems to be something difficult."

However, in LuneScript, like the hello world sample above, you can define macros in the same way as defining functions in general.

However, you cannot define a meaningful macro by writing a macro like the example above.

Below we'll show you how to define meaningful macros.

macro example

Here's an example of a slightly more practical macro:

This macro has the following specifications:

  • A macro _Test that defines a function that returns an int value
  • The int value returned by the function is given as an argument to the macro
  • The name of the defined function is determined by the value of int
  • Specifically, the function name that returns 1 is func1

Here is the code for the specific macro:

// @lnsFront: ok
macro _Test( val:int ) {
   {
      let name = "func%d"(val);
   }
   fn ,,,name(): int {
      return ,,val;
   }
}

_Test( 1 );
_Test( 10 );

print( func1(), func10() ); // 1  10

When expanding a macro, it's pretty much the same as a function call.

In this case, _Test( 1 ), _Test( 10 ) are macro expansions. _Test( 1 ), _Test( 10 ) expands to

// @lnsFront: ok
// Test( 1 )
fn func1():int {
   return 1;
}
// Test( 10 )
fn func10():int {
   return 10;
}

This defines functions func1() and func10() , so print( func1(), func10() ) outputs 1 10 .

This macro is described below.

how to write a macro

The macro definition syntax is:

// @lnsFront: skip
macro name( arg ) {
   {
      macro-statement
   }
   expand-statement
}

It starts with the keyword macro, followed by the macro name name and the arguments arg. Macro name name must start with _. Conversely, non-macro symbol names must not start with _.

Macro arguments support the following types:

  • int
  • real
  • str
  • bool
  • stat
  • List, Map, Set above
  • sym
  • __exp
  • __block

sym, stat, __exp, and __block are described below.

Next comes the macro-statement block, followed by the expand-statement .

To understand how to define a macro, it is easier to understand expand-statement first, so expand-statement will be explained before explaining the macro-statement block.

argument

The following argument types are available for macros only.

  • sym
  • stat
  • __exp
  • __block
sym

sym is a type that can store symbols.

Symbols can be used as functions, variables, members, classes, and all symbols.

stat

stat is a type that can store statements.

__exp

__exp is a type that can store all expressions.

Any expression can be specified, for example 1 + 1 or func() . However, it must be an expression that can be evaluated without error at the time the macro is called.

__block

__block is a type that can store the block statement {}.

Like __exp, it must be a block that can be evaluated without error at the time the macro is called.

expand-statement

expand-statement writes the code after macro expansion.

In the _Test macro example, the next part is the expand-statement.

// @lnsFront: skip
   fn ,,,name(): int {
      return ,,val;
   }

This will expand the function definition.

Macro-only operators are available in this expand-statement. ,, is that operator.

Available operators include:

  • ,,,,
  • ,,,
  • ,,

,,,, is an operator that converts the symbol obtained by evaluating the immediately following variable to a string.

,,, is an operator that converts the string obtained by evaluating the immediately following variable into a symbol.

,, is an operator that expands the variable that immediately follows it.

So, in the example above, ,,,name converts the string in the name variable to a symbol, ,,val expands the val variable, and the _Test( 1 ) macro expands to:

// @lnsFront: ok
fn func1():int {
   return 1;
}

Any code can be written in expand-statement as long as it satisfies the following restrictions.

expand-statement must be a statement.

In other words, expand-statement can write any code unless it doesn't form a statement, such as an expression or part of an incomplete token.

You can also write multiple statements in expand-statement.

macro-statement

In the macro-statement block, define the variables used by expand-statement. Variables used in expand-statement must be declared in the topmost scope of the macro-statement block.

In the _Test macro example, the macro-statement is:

// @lnsFront: skip
   {
      let name = "func%d"(val);
   }

Here we are defining the variable name. Sets "func%d" (val) as the initial value of name.

Inside the macro-statement, you can use all the features of LuneScript. Specifically, you can also define functions within macro-statement.

For example, the _Test macro could also be written like this:

// @lnsFront: ok
macro _Test( val:int ) {
   {
      fn funcname(): str {
         return "func%d"(val);
      }
      let name = funcname();
   }
   fn ,,,name(): int {
      return ,,val;
   }
}

In this example, the macro-statement declares the funcname() function and assigns its result to the name variable.

The functions that can be used in macro-statement are only LuneScript standard functions. Even if the function is defined in the same source, if the function is defined outside the macro, it cannot be used from the macro-statement.

A macro-statement can use macro-only operators just like an expand-statement.

Specifically, the following operators are available:

  • ,,,,
  • ,,,
  • ,,
  • `{}
  • ~~

``,,,,'' ``,,,'' ``,,'' is almost the same as expand-statement. The difference with expand-statement is that while expand-statement targets the variable that follows it, macro-statement targets the expression that follows it.

`{} can use the statement written in `{} as it is.

For example, the _Test macro above can also be written using `{} as:

// @lnsFront: ok
macro _Test( val:int ) {
   {
      let defstat = `{
         fn ,,,"func%d"(val)~~():int {
            return ,,val;
         }
      };
   }
   ,,defstat;
}

_Test( 1 );
_Test( 10 );

print( func1(), func10() ); // 1  10

Here, `{} is used to store the function definition itself in the variable defstat, and defstat is expanded with expand-statement.

Extracting the initialization part of this defstat looks like this:

// @lnsFront: skip
      let defstat = `{
         fn ,,,"func%d"(val)~~():int {
            return ,,val;
         }
      };

Here you can see the use of ~~ .

~ is used to delimit operator expressions such as ,,, . The above uses ~ after "func%d"(val). This indicates that the expression to which the ,,, operator applies is up to "func%d"(val), after which the () is part of the macro-expanding statement.

If ~~ is not specified, it means that () is attached to the string generated by "func%d"(val), resulting in a syntax error.

Here is an example listing for `{}:

// @lnsFront: ok
macro _Test( val:int ) {
   {
      let mut statList:List<stat> = [];
      for count = 1, val {
         statList.insert(
            `{          
               fn ,,,"func%d"(count)~~():int {
                  return ,,count;
               }
            } );
      }
   }
   ,,statList;
}

_Test( 5 );

print( func1(), func2(), func3(), func4(), func5() ); // 1 2 3 4 5

In this example, multiple function definitions (func1 to func5) are performed by storing multiple function definitions in the list statList of `{} and expanding them.

Note that the macro-statement block is optional. If you omit the macro-statement block, omit the entire {} as follows.

// @lnsFront: skip
macro name( arg ) {
   expand-statement
}

Functions available in macro-statement

The following functions are available in macro-statement.

  • fn _lnsLoad( name:str, code:str ): stem;

This function loads the LuneScript code specified by code and returns that module.

macro expansion

The way macros are expanded is the same as for function calls.

public macro

Macros can be exposed to external modules.

By declaring pub as follows, the macro can be used at the import destination.

// @lnsFront: ok
pub macro _Hello() {
   print( "hello world" );
}

A little practical macro example

Here is an example of a slightly more practical macro.

In order to handle JSON used in parameters and responses of REST API provided by Google etc. with LuneScript, it is convenient to classify each JSON format of REST API. In such a case, manually defining a class that handles various types of JSON format data is inefficient and causes bugs.

So let's create a macro that loads the sample JSON format and defines a class that can store that JSON format.

For this example, load the following JSON file,

{
    "val1": "abc",
    "val2": 0
}

A macro that defines the following classes for handling the above JSON.

// @lnsFront: ok
class Hoge {
  pri let val1:str {pub};
  pri let val2:int {pub};
}

Here is a concrete example of a macro:

// @lnsFront: skip
macro _MkClass( name:str, path:str ) {
   {
      let mut memStatList:List<stat> = [];
      if! let mut fileObj = io.open( path ) {
         if! let txt = fileObj.read( "*a" ) {
            let defMap = "pub let val = %s;" (txt);
            let mod = _lnsLoad( "json", defMap );
            if! let jsonval = mod.val {
               fn getType( val:stem ): str {
                  switch type( val ) {
                     case "number" {
                        return "int";
                     }
                     case "string" {
                        return "str";
                     }
                  }
                  return "stem";
               }
               forsort val, key in jsonval@@Map<str,stem> {
                  memStatList.insert( `{
                     pri let ,,,key : ,,,getType( val )~~ {pub};
                  } );
               }
            }
         }
      }
   }
   class ,,,name {
      ,,memStatList;
   }
}
_MkClass( "Hoge", "hoge.js" );

let hoge = new Hoge( "ABC", 100 );
print( hoge.$val1, hoge.$val2 );

This macro loads JSON from a file and declares a class to store the JSON structure.

Specify the class name in the first argument of the macro.

This macro does the following:

  • Open the specified file and read the JSON string defined in that file.
  • Generate LuneScript code from JSON string txt with "pub let val = %s;" (txt);
  • Use _lnsLoad() to load the generated LuneScript code
  • Extract json val from loaded module and enumerate JSON elements with forsort
  • Generate `{} declaring a member holding the enumerated elements and add it to memStatList
  • Declare a class with name and memStatList.

In this example, the members are treated as int and str type data for simplicity. It does not support lists etc.

Common map between macros

Macros are actions that are performed at compile time. Also, each macro execution is independent. When executing two macros A and B, it is not possible to change the control of macro B depending on the execution result of macro A.

However, this can be inconvenient. Therefore, common map between macros is used to share data within macros.

*This is an experimental feature.

From within the macro-statement of the macro, the special variable __var is available.

The special variable __var has the following restrictions:

  • public macros cannot use __var
  • Macros that access __var must be used from the same namespace that defines the macro.
  • If __var is accessed from a different namespace, the contents of that __var are undefined.

The type of this variable is:

let mut __var:Map<str,stem>

This variable is created at the start of compilation for each module, and all macros access the same variable.

For example:

// @lnsFront: ok
   macro _test0( name:str, val:int ) {
      {
         __var[ name ] = val;
      }
   }
   macro _test1() {
      {
         let val;
         if! let work = __var[ "hoge" ] {
            val = work@@int;
         }
         else {
            val = 10;
         }
      }
      print( "%s" (,,val) );
   }
   _test0( "hogea", 1 );
   _test1(); // 10
   _test0( "hoge", 1 );
   _test1(); // 1

In this example, the _test0() macro holds the int data in __var[ "hoge" ] and the _test1() macro changes the processing depending on the stored value of __var[ "hoge" ].

summary

LuneScript can define macros in the same way as functions.

Also, by using macros, you can define various processes.

Next time, I will explain how to build a project developed using LuneScript.