Learning 3-D Scene Structure from a Single Still Image
Ashutosh Saxena, Min Sun and Andrew Y. Ng
Computer Science Department, Stanford University, Stanford, CA 94305
{asaxena,aliensun,ang}@cs.stanford.edu
Abstract
We consider the problem of estimating detailed 3-d struc-
ture from a single still image of an unstructured environment.
Our goal is to create 3-d models which are both quantita-
tively accurate as well as visually pleasing.
For each small homogeneous patch in the image, we use a
Markov Random Field (MRF) to infer a set of “plane param-
eters” that capture both the 3-d location and 3-d orienta-
tion of the patch. The MRF, trained via supervised learning,
models both image depth cues as well as the relationships
between different parts of the image. Inference in our model
is tractable, and requires only solving a convex optimiza-
tion problem. Other than assuming that the environment is
made up of a number of small planes, our model makes no
explicit assumptions about the structure of the scene; this
enables the algorithm to capture much more detailed 3-d
structure than does prior art (such as Saxena et al., 2005,
Delage et al., 2005, and Hoiem et el., 2005), and also give
a much richer experience in the 3-d flythroughs created us-
ing image-based rendering, even for scenes with significant
non-vertical structure.
Using this approach, we have created qualitatively cor-
rect 3-d models for 64.9% of 588 images downloaded from
the internet, as compared to Hoiem et al.’s performance of
33.1%. Further, our models are quantitatively more accu-
rate than either Saxena et al. or Hoiem et al.
1. Introduction
When viewing an image such as that in Fig. 1a, a human
has no difficulty understanding its 3-d structure (Fig. 1b).
However, inferring the 3-d structure remains extremely chal-
lenging for current computer vision systems—there is an in-
trinsic ambiguity between local image features and the 3-d
location of t he point, due to perspective projection.
Most work on 3-d reconstruction has focused on using
methods such as stereovision [16] or structure from mo-
tion [6], which require two (or more) images. Some methods
can estimate 3-d models from a single image, but they make
strong assumptions about the scene and work in specific set-
tings only. For example, shape from shading [18], relies on
purely photometric cues and is difficult to apply to surfaces
that do not have fairly uniform color and texture. Crimin-
isi, Reid and Zisserman [1] used known vanishing points to
Figure 1. (a) A single image. (b) A screenshot of the 3-d model
generated by our algorithm.
determine an affine structure of the image.
In recent work, Saxena, Chung and Ng (SCN) [13, 14]
presented an algorithm for predicting depth from monocular
image features. However, their depthmaps, although use-
ful for tasks such as a robot driving [ 12] or improving per-
formance of stereovision [15], were not accurate enough to
produce visually-pleasing 3-d fly-throughs. Delage, Lee and
Ng (DLN) [4, 3] and Hoiem, Efros and Hebert (HEH) [9, 7]
assumed that the environment is made of a flat ground with
vertical walls. DLN considered indoor images, while HEH
considered outdoor scenes. They classified the image into
ground and vertical (also sky in case of HEH) to produce a
simple “pop-up” type fly-through from an image. HEH fo-
cussed on creating “visually-pleasing” fly-throughs, but do
not produce quantitatively accurate results. More recently,
Hoiem et al. (2006) [8] also used geometric context to im-
prove object recognition performance.
In this paper, we focus on inferring the detailed 3-d struc-
ture that is both quantitatively accurate as well as visually
pleasing. Other than “local planarity,” we make no explicit
assumptions about the structure of the scene; this enables our
approach to generalize well, even to scenes with significant
non-vertical structure. We infer both the 3-d location and the
orientation of the small planar regions in the image using a
Markov Random Field (MRF). We will learn the relation be-
tween the image features and the l ocation/orientation of the
planes, and also the relationships between various parts of
the image using supervised learning. For comparison, we
also present a second MRF, which models only the location
of points in the image. Although quantitatively accurate, this
method is unable to give visually pleasing 3-d models. MAP
inference in our models is efficiently performed by solving
a linear program.
Using this approach, we have inferred qualitatively cor-
1
资源评论
不要麻酱
- 粉丝: 157
- 资源: 4
最新资源
- CNN-LSTM时间预测分析：单输入单输出电力负荷预测模型.pdf
- CNN-LSTM数据分类预测MATLAB代码：基于卷积神经网络-长短期记忆网络，主程序可读取本地EXCEL数据.pdf
- CNN-LSTM组合算法回归预测：精准预测的Matlab实现，程序注释清晰.pdf
- CNN-LSTM组合预测模型（多列输入、单列输出回归预测模型，2020版及以上，含基本注释代码）.pdf
- CNN-RVM多变量回归预测Matlab程序（附测试数据及图例）.pdf
- CNN-RNN网络在电力负荷预测中的应用：Matlab作图，单输入单输出模型.pdf
- CNN-SVM回归预测模型：基于MATLAB 2019及以上版本的'多输入单输出'模型，包含清晰中文注释与高质量评价指标.pdf
- CNN-SVM多特征分类预测（Matlab代码实现：直接替换数据即可使用，环境要求Matlab2018b及以上）》.pdf
- CNN-SVM结合的MISO回归预测模型：'MainCNN_SVR.m'主文件及使用说明.pdf
- CNN分类中应用随机擦除增强的MATLAB源码分享.pdf
- CNN卷积神经网络FPGA加速器实现（小型）_ 从软件到硬件的深度学习部署.pdf
- CNN卷积神经网络多变量回归预测（Matlab）：'经验证'的7输入1输出程序.pdf
- CNN卷积神经网络多输入多输出预测（Matlab）_ 10个输入特征、3个输出变量的验证程序.pdf
- CNN卷积神经网络回归预测算法（基于Matlab 2018b以上版本实现）的代码与样本.pdf
- CNN卷积神经网络用于时间序列预测的高精度Matlab程序.pdf
- CNN卷积神经网络在Matlab中实现多特征分类预测：保证在Matlab 2018b及以上环境下有效运行，可视化输出分类准确率，输入12个特征输出4类标签.pdf
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
