Automatic Differentiation
Forward/tangent and reverse/adjoint modes for efficiently computing sensitivities in numerical programs and neural networks.
- dco/c++
- Higher-order derivatives
- C++
Research
Derivative information for understanding and compressing neural networks.
My doctoral research combines automatic differentiation, interval arithmetic, and significance analysis to identify neural-network components that can be removed in a structured, explainable way.
IASA · accessible view
Which parts of a model can meaningfully affect its output?
Interval Adjoint Significance Analysis estimates how important neurons or channels can be across a range of inputs—not only at one sampled point.
For engineering audiences: IASA supplies a principled ranking signal for structured pruning. It connects derivative-based sensitivity with interval bounds, then uses that signal to remove whole neurons or channels and evaluate the resulting efficient model.
IASA · technical view
Interval tangents and adjoints as bounded significance signals.
The method combines interval evaluation with forward/tangent or reverse/adjoint automatic differentiation to propagate bounded derivative information through a neural network.
These interval sensitivities support uncertainty-aware significance estimates for MLP neurons and CNN channels. The doctoral work studies structured pruning on architectures including VGG-16 and ResNet-56, with bias augmentation and Sobolev-style gradient matching as post-pruning refinement strategies.
- Derivative modes
- Tangent / forward and adjoint / reverse
- Structures
- MLP neurons and CNN channels
- Model families
- VGG-16, ResNet-56, regression and classification networks
Research workflow
From input ranges to a smaller model.
- 01Input domain
Represent ranges or uncertainty in the network input.
- 02Interval propagation
Propagate activation intervals through the model.
- 03Tangent / adjoint
Compute interval derivative information in forward or reverse mode.
- 04Significance ranking
Estimate bounded influence for neurons or channels.
- 05Structured pruning
Remove lower-significance model components.
- 06Optional retraining
Use bias augmentation and Sobolev-style gradient matching.
Focus areas
A connected research programme.
Automatic differentiation is the computational foundation; interval analysis and structured pruning turn it into a model-efficiency method.
Interval Adjoint Significance Analysis
Interval-valued tangent and adjoint propagation for estimating how strongly neurons or channels can influence a model's behavior.
- IASA
- Interval arithmetic
- Sensitivity analysis
Structured Neural-Network Pruning
Derivative-informed ranking of neurons and CNN channels to produce smaller models with practical structured reductions.
- MLPs
- VGG-16
- ResNet-56
- PyTorch
Scientific Computing & HPC
Reproducible numerical and machine-learning experiments on Linux GPU clusters using Slurm, job automation, and model-exchange tooling.
- GPU
- Slurm
- ONNX
- Reproducibility
Research engineering
Methods backed by implementation and infrastructure.
The research requires more than model training: numerical C++ development, higher-order differentiation, GPU execution, scheduling, conversion, logging, and reproducibility.
ML & scientific computing
PyTorch · TensorFlow / Keras · Automatic differentiation · Neural-network pruning · Scientific ML · Numerical methods · Sensitivity analysis
AD & interval methods
dco/c++ · Tangent / forward mode · Adjoint / reverse mode · Higher-order derivatives · Interval arithmetic · Boost interval
HPC & engineering
Linux · Slurm · GPU computing · Multi-GPU experiments · Job arrays · Experiment automation · Git / GitHub
Model tooling
ONNX · tf2onnx · onnx-sim · Reproducibility pipelines · Experiment logging · Scientific debugging
Core publications
Research outputs.
The two works most directly representing IASA and gradient-aware pruning.
Interval Adjoint Significance Analysis for Neural Networks
Computational Science — ICCS 2020 · LNCS 12139, pp. 365–378 · Springer International Publishing
Towards Sobolev Pruning
Platform for Advanced Scientific Computing Conference (PASC24)