Hierarchical Intrinsic and Extrinsic Causal Factors

Updated: Jul 23

Visual objects are composed of a recursive hierarchy of perceptual wholes and parts, whose properties, such as shape, reflectance, and color, constitute a hierarchy of intrinsic causal factors of object appearance. However, object appearance is the compositional consequence of both an object’s intrinsic and extrinsic causal factors, where the extrinsic causal factors are related to illumination, and imaging conditions.Therefore, this paper proposes a unified tensor model of wholes and parts, and introduces a compositional hierarchical tensor factorization that disentangles the hierarchical causal structure of object image formation, and subsumes multilinear block tensor decomposition as a special case. The resulting object representation is an interpretable combinatorial choice of wholes’ and parts’ representations that renders object recognition robust to occlusion and reduces training data requirements. We demonstrate our approach in the context of face recognition by training on an extremely reduced dataset of synthetic images, and report encouraging face verification results on two datasets – the Freiburg dataset, and the Labeled Face in the Wild (LFW) dataset consisting of real-world images, thus, substantiating the suitability of our approach for data starved domains.


References: M.A.O. Vasilescu, E. Kim,"Compositional Hierarchical Tensor Factorization: Representing Hierarchical Intrinsic and Extrinsic Causal Factors ”, In The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’19): Tensor Methods for Emerging Data Science Challenges, August 04-08, 2019, Anchorage, AK. ACM, New York, NY, US References M.A.O. Vasilescu, E. Kim, ”Compositional Hierarchical Tensor Factorization: Representing Hierarchical Intrinsic and Extrinsic Causal Factors”, 25TH ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD2019) Workshop on Tensor Methods for Emerging Data Science Challenges, August 5, 2019.


W. Si, K. Yamaguchi, M.A.O. Vasilescu, ”Face Tracking with Multilinear (Tensor) Active Appearance Models”, June 2013. ”Multilinear Projection for Face Recognition via Canonical Decomposition ”, M.A.O. Vasilescu, IEEE International Face and Gesture Conf. (FG’11), 476-483, 2011.


M.A.O. Vasilescu, ”Multilinear Projection for Face Recognition via Rank-1 Analysis ”, CVPR, IEEE Computer Society and IEEE Biometrics Council Workshop on Biometrics, June 18, 2010.


T. Ivanov, L. Mathies, M.A.O. Vasilescu,”Head pose estimation using multilinear subspace analysis for robot human awareness”, ICCV, 2nd IEEE International Workshop on Subspace Methods, September, 2009.


”TensorTextures: Multilinear Image-Based Rendering”, in CG Magic: The Landscape of Computer Graphics Technology, Noriko Kurachi (ed.), AK Peters Ltd., Publishers of Science and Technology, 2008. M.A.O. Vasilescu, D. Terzopoulos, ”Multilinear (Tensor) Image Synthesis, Analysis and Recognition”, (invited paper) IEEE Signal Processing Magazine, November 2007, 118123. Exploratory DSP Column. ”Multilinear Projection for Appearance-Based Recognition in the Tensor Framework”, M.A.O. Vasilescu, D. Terzopoulos, IEEE 11th International Conference on Computer Vision, 2007.


M.A.O. Vasilescu, D. Terzopoulos, ”Multilinear (Tensor) ICA and Dimensionality Reduction”, it Proc. 7th International Conference on Independent Component Analysis and Signal Separation (ICA07), London, UK, September, 2007, in Lecture Notes in Computer Science, 4666, Springer-Verlag, New York, 2007, 818826.


G Grindlay, MAO Vasilescu, "A Multilinear (Tensor) Framework for HRTF Analysis and Synthesis”, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Honolulu, Hawaii, April 15-20, 2007. M. Alex O. Vasilescu, ”Manifold Decomposition and Low Dimensionality Parameterization” Proceedings of the Learning Workshop, Snowbird, UT, April, 2006. M.A.O. Vasilescu,“Incremental Multilinear SVD”, in TRICAP, Crete, Greece, June 4 - June 9, 2006, extended abstract and presentation. M.A.O. Vasilescu, D. Terzopoulos,“Multilinear Independent Components Analysis and Multilinear Projection Operator for Face Recognition”, in Workshop on Tensor Decompositions and Applications, CIRM, Luminy, Marseille, France August 29 - September 2, 2005. M.A.O. Vasilescu, D. Terzopoulos, “Multilinear Independent Components Analysis”, in Proceedings of the IEEE Computer Vision and Pattern Recognition Conf. (CVPR ’05), San Diego, CA, June, 2005, vol.1, 547–553. M.A.O. Vasilescu, D. Terzopoulos, “TensorTextures: Multilinear Image-Based Rendering”, ACM Transactions on Graphics, 23(3): 336–342, 2004. (Proceedings ACM SIGGRAPH 2004 Conference, Los Angeles, CA, August, 2004.) D. Terzopoulos, Yuecheng Lee and M.A.O. Vasilescu, “Model-Based and Image-Based Methods for Facial Image Synthesis, Analysis and Recognition”, Proceedings of the Sixth International Conferences on Automatic Face and Gesture Recognition (F&G ’04), Seoul, Korea, May 2004, pg. 3-8. M.A.O. Vasilescu, D. Terzopoulos, “Multilinear Independent Components Analysis”, Learning 2004, Snowbird, UT, April, 2004. M.A.O. Vasilescu and D. Terzopoulos, “TensorTextures”, ACM SIGGRAPH 2003 Sketches and Applications, San Diego, CA, July, 2003. M.A.O. Vasilescu, D. Terzopoulos, “Multilinear Subspace Analysis for Image Ensembles”, in Proceedings of the IEEE Computer Vision and Pattern Recognition Conf. (CVPR ’03), Madison, WI, June, 2003, 93–99. M.A.O. Vasilescu, D. Terzopoulos, “Learning Multilinear Models of Images”, Learning 2003, Snowbird, UT, April, 2003. M.A.O. Vasilescu, D. Terzopoulos, “Multilinear Image Analysis for Facial Recognition”, Proceedings of the International Conference on Pattern Recognition (ICPR 2002), Quebec City, Canada, Aug, 2002, 511–514. “Human Motion Signatures for Action Recognition”, M.A.O. Vasilescu, Proceedings of International Conference on Pattern Recognition (ICPR 2002), Quebec City, Canada, Aug, 2002. M.A.O. Vasilescu, D. Terzopoulos, “Multilinear Analysis of Image Ensembles: TensorFaces”, 2002 European Conference on Computer Vision (ECCV ’02), Copenhagen, Denmark, May, 2002, pages 447–460. M.A.O. Vasilescu, ,“An Algorithm for Extracting Human Motion Signatures”,Proceedings of Computer Vision and Pattern Recognition CVPR 2001, Lihue, HI, December, 2001. M.A.O. Vasilescu, “Human Motion Signatures for Character Animations”, ACM SIGGRAPH 2001 Sketches and Applications, Los Angeles, CA, August, 2001. T. Sayed-Mahmood, M.A.O. Vasilescu, S. Sethi, “Recognition Action Events from Multiple View Points”, in Proceedings of the Workshop on Detection and Recognition of Events in Video, International Conference on Computer Vision (ICCV 2001), Vancouver, Canada, July 8, 2001. M. Vasilescu, D. Terzopoulos, “Adaptive meshes and shells: Irregular triangulation, discontinuities, and hierarchical subdivision”, in Proc. Computer Vision and Pattern Recognition Conf. (CVPR ’92), Champaign, IL, June, 1992, pages 829–832. D. Terzopoulos, M. Vasilescu, “Sampling and Reconstruction with Adaptive Meshes”, in Proc. Computer Vision and Pattern Recognition Conf. (CVPR ’91), Lahaina, HI, June, 1991, pages 70–75. M. Alex O. Vasilescu Invited Papers: M.A.O. Vasilescu, D. Terzopoulos, "A Tensor Algebraic Approach to Image Synthesis, Analysis and Recognition", (invited paper), Proc. Sixth International Conference on 3D Digital Imaging and Modeling (3DIM07), Montreal, PQ, August, 2007, 39. M.A.O. Vasilescu, Terzopoulos, "Multilinear (Tensor) Image Synthesis, Analysis and Recognition", (invited paper), EEE Signal Processing Magazine, November, 2007, 118123. D. Terzopoulos, Y. Lee, M.A.O. Vasilescu, Exploratory DSP Column. “Model-Based and Image-Based Facial Synthesis, Analysis, and Recognition”, 6th IEEE International Conference on Automatic Face and Gesture Recognition, Seoul, Korea, May, 2004, 3–8. Chapters in Books: M.A.O. Vasilescu, “TensorTextures: Multilinear Image-Based Rendering”, in CG Magic: The Landscape of Computer Graphics Technology, Noriko Kurachi (ed.), Ohmsha Publisher of Science and Engineering Books, Tokyo, 2005,