Slam Via Variable Reduction From Constraint Maps: Fill & Download for Free

CVPR 2018 had ~ 6000 attendees and almost 1000 papers! Here’s a word cloud I made from the main conference paper titles:As you can see, topics like adversarial learning, analyzing humans (esp. faces), videos, visual question answering, attention, generative modelling etc. were heavily represented.It is impossible to see all the papers, so below I will present the ones that I encountered, which was influenced by my research interests. Grouped by area:Human body pose estimationOrdinal Depth Supervision for 3D Human Pose Estimation: Instead of asking humans to annotate the 3D joint locations from an RGB image (hard!), they collect pairwise information about which joint from a pair is closer to the camera. They show that this ordinal depth information, along with joint location in RGB image, is useful for training robust 3D human pose estimators.Cross-modal Deep Variational Hand Pose Estimation: Interesting paper that uses a Variational autoencoder to learn a low-dimensional latent space from which a 3D hand pose can be decoded. I think using VAE for hand pose estimation makes a lot of sense because it has been shown that human hands live in a low-dimensional manifold compared to their degrees of freedom. They can also embed input data from multiple modalities into the same embedding space.DensePose: Dense Human Pose Estimation In The Wild: They collect a large dataset of dense correspondences between image points and UV coordinates on the surface of a standard human mesh. Using this dataset, they can train a really good CNN that maps image points to body mesh UV coordinates in the presence of occlusions and appearance changes. Not much novelty from an algorithmic point of view, but worth mentioning because of the awesome results that supervised learning can deliver when you have enough data, and they release their dataset.End-to-end Recovery of Human Shape and Pose: Nice paper on human pose estimation from RGB images, using the idea of learning the prior from data mentioned in point 3 of the Machine learning section below.3D face reconstructionSelf-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz, andExtreme 3D Face Reconstruction: Seeing Through Occlusions: The general pipeline seems to be 1) find a “base” face using a PCA-like dimensionality-reduction method, 2) use a CNN to predict the high-frequency corrections to be superimposed on the base faceCamera LocalizationInLoc: Indoor Visual Localization With Dense Matching and View Synthesis: Camera localization in large indoor environments. The pipeline is pretty standard, with some portions “deepified”: extract dense features in images -> describe images with NetVLAD -> retrieve top k images from the training set based on NetVLAD similarity between query image and train image database -> get camera pose hypotheses by densely matching points. The next step, pose hypothesis verification by view synthesis, is novel and relies on the availability of a high-quality 3D reconstruction of the environment.Semantic Visual Localization: Cross-view and across-time camera localization in large scale outdoor environments. The pipeline is similar to the paper above, except that they use a pre-trained semantic segmentation network to abstract away the variations that happen across time, and use depth maps operate in a voxel volume. Their process to generate a descriptor for voxel volumes, which relies on being able to reconstruct a full scene from a partial scene, is interesting.Learning Less is More - 6D Camera Localization via 3D Surface Regression: An interesting paper that mostly follows up from the D-SAC paper from CVPR 17. One interesting contribution is the ability to learn to predict scene coordinates without access to a 3D reconstructed volume during training. They do this by forcing the CNN to learn a mapping from image patches to 3D geometry through a 2 step process.Geometry-Aware Learning of Maps for Camera Localization: Image-based indoor and outdoor camera localization, where the focus was on using geometric constraints induced by visual odometry to improve the camera localization CNN. They show that visual odometry can be effectively used to ingest lots of unsupervised data to keep improving the CNN.Language and visionGuide Me: Interacting with Deep Networks: Really cool paper in which they use natural language input to modulate mid-level features of a CNN to improve the final object detection performance. For example: CNN does not detect dog -> NL input: missing dog in the top-left corner -> CNN detects dog.Embodied Question Answering: They attack the task of learning to navigate a small-scale artificial environment to be able to answer a question specified as language input. For example, question: “which color is the car?” -> agent navigates the environment to find the car -> answers “red”.Machine learningStructured Uncertainty Prediction Networks: They focus on generating crisp images from autoencoders and variational autoencoders without having an adversarial game driving the training. They do this by learning to predict not just the mean but also the variance for a Gaussian distribution underlying each pixel. Intuitively this means that the network is penalized less for predicting a wrong mean value for a pixel it is not very certain about. At test time, interestingly, they add a sample drawn at random from a 0-mean Gaussian with the predicted variance to the predicted mean image (which is blurry) to get a GAN-style crisp image.OLÉ: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learning: This paper proposes a new way to embed visual data in low dimensional manifold (where they live anyway). They propose a new loss function that can be plugged into any classification loss: minimize the rank of the matrix containing features from data supposed to be classified into the same category, and maximize the rank of the matrix containing features from data from various categories. A comparison with triplet loss would have been interesting.Impose natural priors on generated output: This was actually from a workshop on Monday. The classical way to predict output in an ill-posed problem is to impose priors on the output. This paper argues that this prior should not be hand-crafted, but learnt from separate, unpaired data. They do this by attaching a pre-trained discriminator to the output, which is trained on a separate dataset to predict whether its input is from the natural distribution.Fast Geometric Deep Learning with Continuous B-Spline Kernels: A novel convolution operator based on B-splines that allows CNNs to work on structured and geometric input e.g. graphs and meshes.A tractable surrogate for the optimization of the intersection-over-union measure in neural networks: A new loss function that allows optimization for the IoU metric for semantic segmentation.Detail-Preserving Pooling in Deep Networks: Very interesting idea that introduces a new pooling operator that can preserve more detail than max- or average-pooling. Can be potentially useful for tasks that need higher spatial detail even at higher levels of semantic abstraction. Some more justification, for example better boundary predictions for semantic segmentation or more accurate attention heatmaps, would have made the paper stronger.Optical flowA Lightweight Convolutional Neural Network for Optical Flow Estimation: This paper is largely similar to PWC-Net (below) I think, and IMO deserves to share more of the praise limelight that PWC-Net has enjoyed.PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume: This is probably my favorite paper from the conference, because it is very well structured and motivated and uses deep learning in the right places for the right tasks. This paper uses pyramids (P), warping (W) and local cost-volumes (C) to learn to predict optical flow with a network that is both smaller and more accurate than the state of the art. Something that the author mentioned during the talk resonated strongly with me (don’t remember the exact words): “If you know something about the real world, build it into the network, don’t learn it.”Point-cloud processingSparse Lattice Networks for Point Cloud Processing: This network presents a new approach for performing machine learning on pointcloud data, which has challenges because pointclouds are variable length sets of 3D points, where it is expensive to define local neighborhoods and perform convolutions. They advocate the use of sparse bilateral convolution layers on a high-dimensional lattice.Frustum PointNets for 3D Object Detection from RGB-D Data: Straightforward application of PointNet++ to frustums in LIDAR pointclouds. Frustums are defined from bounding-box detections in RGB images.Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation: This was from the DL for Visual SLAM workshop on Monday. Andreas Geiger presented a cool idea that can fuse features from both FCNs operating on RGB images and 3D convnets operating on voxel representations: use depth map to project the RGB features to their respective voxels. The idea is quite simple, but I think has great potential for fusing features learned from two different modalities.Crazy new ideasI mean crazy in a good impressive way :)Cross-modal biometric matching: Simple paper that learns to associate faces with voices. Given a sound clip, they learn to predict which person in the two input face images spoke the clip.Learning and Using the Arrow of Time: This paper learns to predict whether a video is moving forward or backward in time, using optical-flow images are input. They get nice visualizations by plotting the optical flow arrows in the areas of the image that inform the forward/backward prediction.How to write a good paperThis talk was presented by Jitendra Malik as a part of the Good Citizen of CVPR panel. Highlights:When writing the paper, divide it into 4 blocks: title, abstract, introduction, and rest of the paper. Spend equal time on each block.Choose the title which makes the paper memorable, several years from now.First line of introduction should motivate the whole paper. For example, a paper introducing a new dataset about turtles might start with “Turtles are highly underrepresented in the ImageNet database, despite being the most amazing animal.”Attending CVPR ’18 was a great experience, I was exposed to many new ideas!I blog about computer vision titbits at Daily Computer Vision.