Projects

Design Deep Learning like Mathematical Models

Deep learning excels at learning diverse mappings, but its success relies heavily on the mathematical framework in which it operates. By incorporating principles from differential geometry and Bayesian statistics, deep learning becomes a flexible mathematical model that we can tailor to our specific needs.

Geometric Deep Learning

Deep Learning Ensuring Laws of Physics

A model of physical phenomena is considered accurate if it can reproduce the geometric properties associated with the laws of physics, rather than merely their superficial dynamics. For example, the molecule representation invariant to position and angle ensures the conservation of momentums.

Takashi Matsubara and Takaharu Yaguchi, "FINDE: Neural Differential Equations for Finding and Preserving Invariant Quantities," Proc. of The Eleventh International Conference on Learning Representations (ICLR2023), Kigali, May 2023. link arXiv
Yuhan Chen, Takashi Matsubara, and Takaharu Yaguchi, "KAM Theory Meets Statistical Learning Theory: Hamiltonian Neural Networks with Non-Zero Training Loss," Proc. of The Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI2022), Virtual, Feb., 2022. (Oral) link arXiv
Takashi Matsubara, Yuto Miyatake, and Takaharu Yaguchi, "Symplectic Adjoint Method for Exact Gradient of Neural ODE with Minimal Memory," Advances in Neural Information Processing Systems 34 (NeurIPS2021), Virtual, Dec., 2021. link arXiv
Yuhan Chen, Takashi Matsubara, and Takaharu Yaguchi, "Neural Symplectic Form: Learning Hamiltonian Equations on General Coordinate Systems," Advances in Neural Information Processing Systems 34 (NeurIPS2021), Virtual, Dec., 2021. (Spotlight) link
Takashi Matsubara, Ai Ishikawa, and Takaharu Yaguchi, "Deep Energy-Based Modeling of Discrete-Time Physics," Advances in Neural Information Processing Systems 33 (NeurIPS2020), Virtual, Dec., 2020. (Oral) link arXiv
Kohei Shimamura, Shogo Fukushima, Akihide Koura, Fuyuki Shimojo, Masaaki Misawa, Rajiv Kalia, Aiichiro Nakano, Priya Vashishta, Takashi Matsubara, and Shigenori Tanaka, "Guidelines for Creating Artificial Neural Network Empirical Interatomic Potential from First-Principles Molecular Dynamics Data under Specific Conditions and Its Application to α-Ag₂Se," Journal of Chemical Physics, vol.151, 124303, 2019. link

Topology-aware Data Generation

Deep learning models often overlook the topological structure inherent in data distributions or object shapes. Incorporating such structure enables natural and refined artificial data generation.

Takehiro Aoshima and Takashi Matsubara, "Deep Curvilinear Editing: Commutative and Nonlinear Image Manipulation for Pretrained Deep Generative Model," Proc. of The IEEE/CVF Computer Vision and Pattern Recognition Conference 2023 (CVPR2023), Jun., 2023. arXiv
Takumi Kimura, Takashi Matsubara, and Kuniaki Uehara, "Topology-Aware Flow-Based Point Cloud Generation," IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 11, pp. 7967-7982, 2022. link
Takumi Kimura, Takashi Matsubara, and Kuniaki Uehara, "ChartPointFlow for Topology-Aware 3D Point Cloud Generation," Proc. of ACM International Conference on Multimedia (ACMMM2021), Virtual, Oct., 2021. (Oral) link arXiv

Bayesian Deep Learning

Deep Generative Models

Generative models extract detailed features for reconstruction and are less susceptible to overfitting to a prominent subset of features. Also, they can incorporate prior knowledge about the causal relationship between factors and enable unbiased decision-making.

Takashi Matsubara, Koki Kusano, Tetsuo Tashiro, Ken'ya Ukai, and Kuniaki Uehara, "Deep Generative Model of Individual Variability in fMRI Images of Psychiatric Patients," IEEE Transactions on Biomedical Engineering, vol. 68, no. 2, pp. 592-605, 2021. link
Takashi Matsubara, Tetsuo Tashiro, and Kuniaki Uehara, "Structured Deep Generative Model of FMRI Signals for Mental Disorder Diagnosis," Proc. of The 21st International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI2018), Granada, Sep. 2018, pp. 258-266. (acceptance rate 372/1,068=0.348) link
Takashi Matsubara, Ryo Akita, and Kuniaki Uehara, "Stock Price Prediction by Deep Neural Generative Model of News Articles," IEICE Transactions on Information and Systems, Vol.E101-D, No.4, pp.901-908, 2018. link

Uncertainty Quantification

The real-world is not fully deterministic, and the machine learning methods is preferable to quantity the uncertainty in their decision making, allowing for human intervention if necessary. The uncertainty quantification is also useful to identify outliers, with applications such as quality control for manufactured products.

Takashi Matsubara, Kazuki Sato, Kenta Hama, Ryosuke Tachibana, and Kuniaki Uehara, "Deep Generative Model using Unregularized Score for Anomaly Detection with Heterogeneous Complexity," IEEE Transactions on Cybernetics, vol. 52, no. 6, pp. 5161-5173, 2022. link arXiv
Kazuki Sato, Satoshi Nakata, Takashi Matsubara, and Kuniaki Uehara, "Few-shot Anomaly Detection using Deep Generative Models for Grouped Data," IEICE Transactions on Information and Systems, vol.E105-D, no.2, pp.436-440, 2022. link
Kenta Hama, Takashi Matsubara, Kuniaki Uehara, and Jianfei Cai, "Exploring Uncertainty Measures for Image-Caption Embedding-and-Retrieval Task," ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 17, no. 2, article no. 46, 2021. link arXiv