Musing from C4ML 2019

30 Mar 2019

Paper : Getting to Machine Learning from a General Purpose Compiler

• ML algorithms require computation of derivative
• Embedding derivatives into computation library
• Symbolic Execution

  f(x) = x*e^x , f’(x) = e^x * d/dx(x) + x * d/dx(e^x)

• Computation library knows derivative individually without symbolic representation

• Type Composibility is Solved!
  • Basic Types -> Arrays -> matrix -> tensors
  • Basic Ops -> libray functions -> high level library
• Compiler Composibility -> Not Solved!
  • Programming Models
    • ISL
    • Task Parallel
    • SPMD
  • Optimizers
    • Type : Scalar , Tensor
    • Data Layout
    • Data access
  • Codegen - GPU ASIC …
• Concept of Dynamic Program
  • Basically interpreter style for non-x86
  • Use HLL -> IR (static Analysis) -> JIT /AOT -> dynamic analysis -> hardware
• Machine Learning compiler Requires automatic diff.
  • Given Callable, create Auto Diff. Version of Callable.
  • After Diff, Run Dead Code Elimination. (Wow!)

### AI Langauge in scala

• OptiML

### Compiling ML with XLA

### Glow: Graph Lowering Compiler Techniques for Neural Networks

### The Sparse Tensor Algebra Compiler

### Polyhedral Compilation of ML Computation Graphs

### Compiling Deep Neural Networks for ACAP Devices

### TVM: An Automated End-to-End Optimizing Compiler for Deep Learning

### MLIR Primer: A Compiler Infrastructure for the End of Moore’s Law