Sher Afghan MalikAlgorithmic Differentiation
Forward/tangent and reverse/adjoint modes for efficiently computing sensitivities in numerical programs and neural networks.
- dco/c++
- Higher-order derivatives
- C++
Research
Derivative information for understanding and compressing neural networks.
My doctoral research combines algorithmic differentiation, interval arithmetic, and significance analysis to identify neural-network components that can be removed in a structured, explainable way.
IASA · accessible view
Which parts of a model can meaningfully affect its output?
Interval Adjoint Significance Analysis estimates how important neurons or channels can be across a range of inputs—not only at one sampled point.
Algorithmic differentiation (AD), also known as automatic differentiation, provides derivative information for the analysis. For engineering audiences, IASA supplies a principled ranking signal for structured pruning: it connects derivative-based sensitivity with interval bounds, then uses that signal to remove whole neurons or channels and evaluate the resulting efficient model.
IASA · technical view
Interval tangents and adjoints as bounded significance signals.
The method combines interval evaluation with algorithmic differentiation to propagate bounded derivative information through a neural network. Forward mode is also called tangent mode, while reverse mode is also called adjoint mode.
These interval sensitivities support uncertainty-aware significance estimates for neurons and channels in deep neural networks. The doctoral work studies structured pruning for multilayer perceptrons and convolutional neural networks, with bias augmentation and Sobolev-style gradient matching as post-pruning refinement strategies.
- Derivative modes
- Forward / tangent and reverse / adjoint
- Structures
- MLP neurons and CNN channels
- Model families
- Deep neural networks for regression and classification
Research workflow
From input ranges to a smaller model.
- 01Input domain
Represent ranges or uncertainty in the network input.
- 02Interval propagation
Propagate activation intervals through the model.
- 03Tangent / adjoint
Compute interval derivative information in forward or reverse mode.
- 04Significance ranking
Estimate bounded influence for neurons or channels.
- 05Structured pruning
Remove lower-significance model components.
- 06Optional retraining
Use bias augmentation and Sobolev-style gradient matching.
Focus areas
A connected research programme.
Algorithmic differentiation is the computational foundation; interval analysis and structured pruning turn it into a model-efficiency method.
Interval Adjoint Significance Analysis
Interval-valued tangent and adjoint propagation for estimating how strongly neurons or channels can influence a model's behavior.
- IASA
- Interval arithmetic
- Sensitivity analysis
Structured Neural-Network Pruning
Derivative-informed ranking of neurons and CNN channels to produce smaller models with practical structured reductions.
- Deep neural networks
- MLPs
- CNNs
- PyTorch
Scientific Computing & HPC
Reproducible numerical and machine-learning experiments on Linux GPU clusters using Slurm, job automation, and model-exchange tooling.
- GPU
- Slurm
- ONNX
- Reproducibility
Research engineering
Research implementation.
The research is implemented through C++ and Python, dco/c++, PyTorch/TensorFlow, and reproducible GPU/HPC workflows using Linux and Slurm.
Core publications
Research outputs.
Interval Adjoint Significance Analysis for Neural Networks
Computational Science — ICCS 2020 · LNCS 12139, pp. 365–378 · Springer International Publishing
Towards Sobolev Pruning
Platform for Advanced Scientific Computing Conference (PASC24)