Gather reference dependency tree from TypeScript file

Question

Question:

Regarding compiling TypeScript code server-side, is there a way to get a list of all the reference paths either for a single .ts file - or better, the whole compilation (starting from a single .ts file)? In order, preferably.

I'd prefer to use the existing parser if possible, rather than parsing the files with new code.

Context:

Since I don't think it exactly exists, I want to write:

1. a server-side web user control that takes a .ts path and generates a cache-busting script tag pointing to
2. an HttpHandler that compiles the requested .ts file ONCE at first request and then adds CacheDependencies to all the reference dependencies paths. When a file changes, the script-generating web user control updates it's cache-busting suffix of subsequent requests.

so in Release Mode, <tsb:typescript root="app.ts" runat="server" /> yields

<script type="text/javascript" src="app.ts?32490839"></script>

where the delivered script is the on-demand, cached single-file script.

And in Debug Mode the unmodified tag instead yields:

<script type="text/javascript" src="dependency1.ts?32490839"></script>
<script type="text/javascript" src="dependency2.ts?32490839"></script>
<script type="text/javascript" src="app.ts?32490839"></script>

As far as I've looked, this mode of operation is not supported by the TypeScript Visual Studio plugin nor any of the Optimizer bundlers. The bundlers do close to what I'm asking for, but they don't cache-bust and they don't single-file compile without annoying explicit bundling of the files.

I don't mind any performance hit at the very first request while the scripts are compiled. Besides that, perhaps there's a really great reason that this setup shouldn't or can't exist. If this cannot or clearly should not be done, I'd appreciate answers in that vein as well.

I've seen other questions on StackOverflow that dance around this desire in my interpretation, but nothing so explicit as this and none with relevant answers.

Thanks!

Also, is executing tsc.exe in a different process the best way for my HttpHandler to compile at runtime or is there a slick, safe, and simple way to do this in-process?

Answer 1

In 2021 we have --explainFiles

If you want to examine your codebase more carefully (such as differentiating between type-only imports and runtime imports), you can use the Typescript API. The possibilities are infinite, but perhaps these parts below help set you in a good direction (probably has bugs):

import * as ts from "typescript";

interface FoundReference {
    typeOnly: boolean;
    relativePathReference: boolean;
    referencingPath: string;
    referencedSpecifier: string;
}

const specifierRelativeFile = /^\..*(?<!\.(less|svg|png|woff))$/;
const specifierNodeModule = /^[^\.]/;

const diveDeeper = (path: string, node: ts.Node, found: FoundReference[]) =>
    Promise.all(node.getChildren().map(n => findAllReferencesNode(path, n, found)));

const findAllReferencesNode = async (path: string, node: ts.Node, found: FoundReference[]) => {
    switch (node.kind) {
        case ts.SyntaxKind.ExportDeclaration:
            const exportDeclaration = node as ts.ExportDeclaration;

            if (exportDeclaration.moduleSpecifier) {
                const specifier = (exportDeclaration.moduleSpecifier as ts.StringLiteral).text;

                if (specifier) {
                    if (specifierRelativeFile.test(specifier)) {
                        found.push({
                            typeOnly: exportDeclaration.isTypeOnly,
                            relativePathReference: true,
                            referencingPath: path,
                            referencedSpecifier: specifier
                        });
                    } else if (specifierNodeModule.test(specifier)) {
                        found.push({
                            typeOnly: exportDeclaration.isTypeOnly,
                            relativePathReference: false,
                            referencingPath: path,
                            referencedSpecifier: specifier
                        });
                    }
                }
            }

            break;
        case ts.SyntaxKind.ImportDeclaration:
            const importDeclaration = node as ts.ImportDeclaration;
            const importClause = importDeclaration.importClause;

            const specifier = (importDeclaration.moduleSpecifier as ts.StringLiteral).text;

            if (specifier) {
                if (specifierRelativeFile.test(specifier)) {
                    found.push({
                        typeOnly: (!!importClause && !importClause.isTypeOnly),
                        relativePathReference: true,
                        referencingPath: path,
                        referencedSpecifier: specifier
                    });
                } else if (specifierNodeModule.test(specifier)) {
                    found.push({
                        typeOnly: (!!importClause && !importClause.isTypeOnly),
                        relativePathReference: false,
                        referencingPath: path,
                        referencedSpecifier: specifier
                    });
                }
            }

            break;
        case ts.SyntaxKind.CallExpression:
            const callExpression = node as ts.CallExpression;

            if ((callExpression.expression.kind === ts.SyntaxKind.ImportKeyword ||
                (callExpression.expression.kind === ts.SyntaxKind.Identifier &&
                    callExpression.expression.getText() === "require")) &&
                callExpression.arguments[0]?.kind === ts.SyntaxKind.StringLiteral) {

                const specifier = (callExpression.arguments[0] as ts.StringLiteral).text;

                if (specifierRelativeFile.test(specifier)) {
                    found.push({
                        typeOnly: false,
                        relativePathReference: true,
                        referencingPath: path,
                        referencedSpecifier: specifier
                    });
                } else if (specifierNodeModule.test(specifier)) {
                    found.push({
                        typeOnly: false,
                        relativePathReference: false,
                        referencingPath: path,
                        referencedSpecifier: specifier
                    });
                } else {
                    await diveDeeper(path, node, found);
                }
            } else {
                await diveDeeper(path, node, found);
            }

            break;
        default:
            await diveDeeper(path, node, found);

            break;
    }
}

const path = "example.ts";

const source = `
import foo from "./foo";
import * as bar from "./bar";
import { buzz } from "./fizz/buzz";

export foo from "./foo";
export * as bar from "./bar";
export { buzz } from "./fizz/buzz";

const whatever = require("whatever");

const stuff = async () => {
    require("whatever");

    const x = await import("xyz");
}
`

const rootNode = ts.createSourceFile(
    path,
    source,
    ts.ScriptTarget.Latest,
    /*setParentNodes */ true
);

const found: FoundReference[] = [];

findAllReferencesNode(path, rootNode, found)
.then(() => { 
    console.log(found); 
});

[
  {
    "typeOnly": true,
    "relativePathReference": true,
    "referencingPath": "example.ts",
    "referencedSpecifier": "./foo"
  },
  {
    "typeOnly": true,
    "relativePathReference": true,
    "referencingPath": "example.ts",
    "referencedSpecifier": "./bar"
  },
  {
    "typeOnly": true,
    "relativePathReference": true,
    "referencingPath": "example.ts",
    "referencedSpecifier": "./fizz/buzz"
  },
  {
    "typeOnly": false,
    "relativePathReference": true,
    "referencingPath": "example.ts",
    "referencedSpecifier": "./bar"
  },
  {
    "typeOnly": false,
    "relativePathReference": true,
    "referencingPath": "example.ts",
    "referencedSpecifier": "./fizz/buzz"
  },
  {
    "typeOnly": false,
    "relativePathReference": false,
    "referencingPath": "example.ts",
    "referencedSpecifier": "whatever"
  },
  {
    "typeOnly": false,
    "relativePathReference": false,
    "referencingPath": "example.ts",
    "referencedSpecifier": "whatever"
  },
  {
    "typeOnly": false,
    "relativePathReference": false,
    "referencingPath": "example.ts",
    "referencedSpecifier": "xyz"
  }
] 

Once you have the referencedSpecifier, you need some basic logic to resolve it to the next path and repeat your exploration with that next resolved file, resursively.

Answer 2

Bundles do this for JavaScript if you enable optimisations and use the bundle HTML helper too add the bundle to the page. The cache-bust URL is a short hash of the JavaScript file, so if you change any script in the bundle, the new hash breaks it out of the client cache.

My recommendation is to bundle the compiled JavaScript and have the bundler minify and cache-bust. You can get TypeScript to generate a single file with all dependencies using the out flag...

tsc --out final.js app.ts

Your bundle now only needs to include final.js - which saves on explicitly listing all files and adding new ones later.

You can still write something to intercept and compile at runtime. My preference is to have this done before runtime as one of the benefits of TypeScript is compile-time checking - how will you handle a compile error if the script is being demanded by a client. Even if you do it at runtime, I would still add the JavaScript reference to the page, not the .ts file.

Answer 3

You can use Madge, either interactively or in node.

See also: How can I see the full nodejs "require()" tree starting at a given file?