Compiler as a service: How to build Refactoring tools?

Question

Lisa Feigenbaum from Microsoft talks here about "Compiler as a service". I have read this would make it easier to build refactoring tools. How? Mono's CAAS is great but if Microsoft version is similar I don't see how this specific use case is done.

Answer 1

"Compiler as a service" means breaking down the compiler into individual pieces.

Instead of having one big monolithic black box where source code goes into one end and compiled assemblies comes out the other, you get lots of smaller (black) boxes with typed output.

So you could, for instance, feed the source code into one box, and get an abstract syntax tree (AST) out the other. This tree could then be manipulated, before it is fed into the optimizer, out of which comes some other representation of the code, which could be fed into the compiler, which then outputs executable code.

Since I don't know a lot about the exact plans for the "compiler as a service" part of a future .NET, the above is just a wild guess, but that's how I see the possibilities.

Refactoring could then operate on the AST, and I would assume there was a way to go from the AST back to the original source code, both through mapping and conversion (mapping means you could take a node in the AST and ask "which part of the source code does this node correspond to", and conversion would mean "could you please give me the source code that this AST now represents, after I have modified it".)

For instance, I would see both JetBrains and DevExpress, both making refactoring tools for Microsoft, having to evaluate their own efforts into writing code that reads and picks apart code for refactoring vs. using the one provided by the CAAS.

Answer 2

The Roslyn CTP includes a walkthrough for building s "Code Action" which is our terminology for something that can be either "quick fix", if it is linked to something wrong with the code, or a refactoring if it is offered contextually.

Also take a look at the CodeRefactoring project template that you will see in Visual Studio if you install the Roslyn CTP.

Answer 3

My company's (Semantic Designs) offers a "compiler as a service": the DMS Software Reengineering Toolkit.

DMS is generic with respect to computer (any formal) languages. Given an language descriptions (DMS has robust versions of these for C++, C#, Java, PHP, COBOL, and many other languages), DMS can parse source to ASTs, and regenerate valid source from those ASTs, including the orginal comments.

DMS provides various analyzer engines, both configurable attribute evaluations, flow analysis, iterative solvers, use-def analysis, local and global call graph construction, and global points-to anlaysis. ASTs may be modifed by procedural code (classic compiler tree hacking) or by source-to-source transformations. The transformations may map ASTs in one language to that same language ("optimization") or to other langauges ("refinement/translation"). This is very mature infrastructure with 15+ years of continuous engineering behind it.

You use DMS by selecting/defining the set of input/output languages you want, and building custom code written in compiler-oriented DSLs (compiled by DMS to make your desired tool!) to invoke the various bits of DMS machinery that accomplish your purpose.

DMS has been used to build language migration tools (see B-2 Stealth Bomber mission software migration), large-scale C++ re-architecting tools, code generation tools to run automobile factory assembly cells, and many classic software engineering tools (test coverage, profilers, clone detection, smart differencers). It is a natural foundation for refactoring tools; we are working on these :-}