tech

[English] [Japanese]

A. Tag jumping with lnstags (source code tagging system)

LuneScript supports tag jumps (source code tag system).

What is a tag jump (source code tagging system)?

A tag jump provides access to the following information in the source code.

  • Definition position of symbol (class, function, variable etc..)
  • Reference position of symbol (class, function, variable etc..)

example

An example is shown below.

// @lnsFront: ok
class Foo {
   pub fn func() {
   }
}
class Bar {
   pub fn func() {
   }
}
let foo = new Foo();
foo.func();
let bar = new Bar();
bar.func();

For example, given the mini.lns file above, to list where the func method of class Foo is defined, run the following command:

$ lnstags inq def @mini.Foo.func
@mini.Foo.func      2 mini.lns            pub fn func() {

To list the reference location of the func method of the Bar class, execute the following command.

$ lnstags inq ref @mini.Bar.func
@mini.Bar.func     12 mini.lns         bar.func();

See below for more information on how to use the lnstags command.

Features of lnstags

lnstags provides the following functions in addition to general tag jump functions.

  • A list of places where the value of the variable is set

    • It distinguishes whether it is setting a value or referring to it, and can list only the locations where a value is set.
  • Considering Inheritance Relationships

    • When calling an overridden method, the actual method called differs due to polymorphism.
    • lnstags lists all methods that may be called.

Usage

lnstags is published at the following URL.

<https://github.com/ifritJP/lnstags>

Build method

There are two build methods:

  • use go install
  • close the repository and make

go install is usually fine.

go install

Install lnstags in $GOPATH/bin by running:

go install -tags gopherlua github.com/ifritJP/lnstags@latest

Build with go.

$ git clone --depth 1 https://github.com/ifritJP/lnstags
$ cd lnstags
$ make build ONLY_GO=y

The above will generate lnstags/lnstags.

Analysis of source code

Source code analysis is executed on the project directory of the source code to be analyzed.

$ cd proj  // lune.js を置いてあるディレクトリに移動する
$ lnstags init
$ lnstags build test/main.lns

lnstags build should be run with either:

  • Specify a main .lns file for the project
$ lnstags build main.lns
  • Specify it as lnstags build @- and specify the path of the .lns file in stdin line by line.
$ find . -iname '*.lns' | lnstags build @-

lnstags will parse all imported modules from the specified lns file.

As a result, in many cases, just specifying the main .lns file is fine.

The latter case is used when all modules cannot be traced from one lns file.

Note that lnstags currently does not support incremental updates.

So if you use @- for build you need to specify all .lns files.

The source code analysis results are registered in the lnstags.sqlite3 file.

DB update

If the source code is changed after being analyzed by lctags build, inconsistency will occur between the information registered in the DB and the actual source code information.

To update the DB, do one of the following:

  • Run lctags build again.
  • Run lctags update.
$ lctags update

lctags update

lctags update updates the information based on the lns source file list registered in the DB.

Unlike lctags build, you don't need to specify the source file to analyze.

If the number of lns files to be analyzed increases or decreases, lctags build should be used instead of lctags update.

inquiry

After analyzing the source code with lnstags build, query the symbol information.

The query has the following pattern:

  • inq
  • inq-at
  • suffix

inq

inq makes an inquiry by specifying a symbol name.

// @lnsFront: ok
class Foo {
   pub fn func() {
   }
}
class Bar {
   pub fn func() {
   }
}
let foo = new Foo();
foo.func();
let bar = new Bar();
bar.func();

For example, given the mini.lns file above, the symbolic name for the func method in the Foo class would be:

@mini.Foo.func

where @mini indicates the module name and mini refers to mini.lns. For example, for abc/def/ghi.lns, the module name will be @abc.@def.@ghi. Foo.func points to the func method of the Foo class.

And by executing the following command, the definition location of @mini.Foo.func is listed.

$ lnstags inq def @mini.Foo.func
@mini.Foo.func      2 mini.lns            pub fn func() {

The def of this command lnstags inq def specifies the definition location query.

inquiry mode

The types of inquiries are as follows.

option motion
def definition position
ref reference position
set Setting position

inq-at

If you know the fully qualified name of the symbol you want to inquire about, you can use the inq command, but finding out the fully qualified name can be difficult or cumbersome.

So inq-at queries for a symbol at a given position in the source.

When querying the definition location of foo.func() in the 5th column of the 10th line when there is the following source,

// @lnsFront: ok
class Foo {
   pub fn func() {
   }
}
class Bar {
   pub fn func() {
   }
}
let foo = new Foo();
foo.func();
let bar = new Bar();
bar.func();

Run the following command.

$ lnstags inq-at def mini.lns 10 5
@mini.Foo.func      2 mini.lns            pub fn func() {

This will query for the symbol at the specified location.

In the above case, we recognize that the fully qualified name of foo.func in mini.lns 10 5 is @mini.Foo.func and def query it.

When using inq-at, the specified lns file must be built without errors.

Also, it takes time to parse the AST of the specified lns file.

suffix

inq-at gets the fully qualified name by specifying the location of the lns file, while suffix gets the list of fully qualified names with a suffix match on the symbol name.

For example, to display a list of fully qualified names that match func at the end:

$ lnstags suffix func
@mini.Foo.func
@mini.Bar.func

When using from emacs

Load lisp/lnstags-conf.el.

The keybindings are as follows.

Key operation
M-t of the symbol at the cursor position. definition position jump to
M-r of the symbol at the cursor position. reference position jump to
M-s of the symbol at the cursor position. Setting position jump to
M-m Tag jump history
C-t jump back

For M-t, M-r, M-s, do a suffix query to get the fully qualified name, list the fully qualified names that match the symbol, and do an inq query on the selected fully qualified names.

By prefixing M-t, M-r, and M-s with C-u, you can inq-at query the cursor position.

analysis time

lnstags uses the AST analysis part of the LuneScript transcompiler to register the symbol information in the source code to be analyzed in the DB.

Therefore, the time required for DB registration is almost equivalent to the Lns file transcompiling time.

Even parsing LuneScript's self-hosted code takes less than 10 seconds.

By using the LuneScript module to perform the heaviest AST analysis of the source code tagging system, lnstags itself has a simple configuration of just over 2000 lines of code. (as of 2021)

Note that lnstags itself is also developed in LuneScript.

As mentioned above, lnstags does not support DB diffs. This is because we believe that there are currently no large-scale LuneScript projects that take that long.

If you are using LuneScript for a large project, please contact me for reference.