2. 内 容0 Space of Multimedia Computing
1 Media and Multimedia
2 Media Processing
2.1 Media Food(Value) Chain
2.2 Digital Item Declaration
2.3 Multimedia Content Representation(coding)
2.4 Digital Item Identification and Description
2.5 Content Management and Usage
3 Networked Multimedia Applications
4. 多媒体信息处理的总目标 To enable transparent and augmented use of multimedia resources across a wide range of networks and devices.
使能跨越各种不同网络和设备, 透明地、强化地使用多媒体资源。
5. 1 Media and Multimedia
6. 1.1 计算机与媒体媒体又称媒介、媒质,指的是用于表示、存储、分发、传输和展现数据(信息) 的手段、方法、工具、设备或装置。
( In general, one describes medium as a means for representation, storage, distribution, transmission and presentation of data. )
22. 多媒体(内容)处理的7要素1 数字项目说明(Digital Item Declaration)
2 内容表示(Content Representation)
3 数字项目的标识与描述(Digital Item Identification and Description)
4 内容管理与使用(Content Management and Usage)
5 知识产权管理与保护(Inte11ectual Property Management and Protection)
6 内容发送与终端及网络(Terminals and Networks)
7 事件报告(Event Reporting)。
23. 2.2 Digital Item Declaration
24. GoalEstablish a uniform and flexible abstraction and interoperable schema for defining digital items.
为定义数字项而建立一个统一、灵活的抽象表示和可互操作的模式。
25. Digital Item Declaration的需求1Digital items are open and extensible to any and all media resources types and description schemes.数字项是开放的,可扩展到所有媒体资源类型和描述模式.
Composite items can be constructed from other items, without losing the structure and properties of the sub-items. 复合项由子项组合而成,子项的结构和性质不丢失.
Multiple composite items may share individual elements.多个复合项可以共享单个元素.
26. Digital Item Declaration的需求An individual element may be referenced by multiple locations within a digital item. 一个元素可以被数字项中的多个地方引用.
To enable applications to correctly manipulate and validate digital items.能使应用程序正确操纵和验证数字项.
Identification and revision of digital items and their components must be supportable in an open and extensible manner.对数字项及其组件的标识和修改必须以开放和可扩展的方式进行.
27. Digital Item Declaration的需求2Digital items explicitly define the relationships between elements and their corresponding descriptors.数字项对其元素与相应描述符之间的关系显式地进行定义
Descriptors may be simple statements or full media components.描述符可以是简单语句也可以是整个媒体组件
Descriptors can be described by other descriptors.描述符也可以由其它描述符描述
Anchors must be declarable within a component that allow:
descriptors to be associated with a specific point or range within a media resource.
linking back from within a media resource to the anchor.
锚(anchor)必须在一个组件中予以说明,从而允许描述符与媒体资源中的一个指定点或指定范围相关联,也允许从媒体资源反向链接到锚。
28. Current Situation目前还没有数字项的标准模型或表示方法:
没有一种能适合任何场合中说明各种各样内容的通用的、灵活的、可以互操作的解决方案,
也没有一个能将任何种类媒体资源同各种类型媒体描述信息链接在一起的统一的方法。
由于数字项的概念是建立在显式地描述媒体数据与描述数据之间关系的想法之上,这就严重限制了标准模型或表示的存在。
29. 举例: MPEG-4的场景说明MPEG-4 视听场景(audiovisual scenes) 是由若干媒体对象(media objects)以层次方式组合而成的,最低层的称为初级(primitive) 媒体对象, 如:
still images (e.g. as a fixed background),
video objects (e.g. a talking person - without the background)
audio objects (e.g. the voice associated with that person);
text and graphics;
talking synthetic heads and associated text used to synthesize the speech and animate the head;
synthetic sound
初级媒体对象既可以是自然媒体,也可以是合成媒体,既可以是 2维的,也可以是3维的。
30. MPEG-4的场景说明MPEG-4视听场景的描述可以看作一棵树, 叶节点是初级媒体对象, 中间节点是复合媒体对象(compound media objects),根节点则为场景。使用这种方法, 内容提供商(作者)可以构造出相当复杂的场景, 例如:
place media objects anywhere in a given coordinate system;
apply transforms to change the geometrical or acoustical appearance of a media object;
group primitive media objects in order to form compound media objects;
apply streamed data to media objects, in order to modify their attributes (e.g. a sound, a moving texture belonging to an object; animation parameters driving a synthetic face);
change, interactively, the user’s viewing and listening points anywhere in the scene.
34. Requirements 1Data Types – it shall be possible to represent a large range of data types, both with natural and synthetic origin, as well as any combination of these data types, e.g. still pictures, frame-based as well as arbitrarily shaped video, specific and generic 3D models, graphics, text, natural and synthetic audio, natural and synthetic speech, etc.
数据类型:应能够表示各种不同的数据类型,包括自然的和合成的,也包括两者的组合。如:静止图像,其于帧的视频,任意形状的视频,专用的和通用的3D模型,图形,文本,自然声音和合成声音,自然语音和合成语音等。
35. Requirements 2Content Variety – it shall be possible to represent any type of content for all the data types considered.
内容种类:上述所有数据类型应能表示任何种类的内容。
36. Requirements 3Efficiency – it shall be possible to represent content for each of the data types above mentioned in the most bit efficient way for different target qualities; for a multimedia scene composed as a combination of various elements of the data types mentioned above, it shall be possible to selectively choose the coding efficiency/quality for each one of them.
有效性:根据不同的目标质量,上述每一种数据类型都使用最有效的数据位来表示内容。对于由不同数据类型的多种元素组成的多媒体场景,应能对每个元素选择其编码的效率/质量。
37. Requirements 4Scalability – it shall be possible to represent all the elements in the multimedia scene in a scalable way and with a fine granularity in the dimensions considered relevant, e.g. spatial, temporal, quality.
可分级性:应能以一种可分级的方法来表示多媒体场景中的所有元素,且在所考虑的相关维度如时间、空间、质量上是细粒度可分级的。
38. Requirements 5Random Access – it shall be possible to randomly access, within a limited time and with fine resolution, at all scalable layers, all elements in the multimedia scene; for a multimedia scene composed as a combination of various elements of the data types mentioned above, it shall be possible to randomly access each one of them.
随机存取:在一定时间内,以比较精细的分辨率,在所有可分级的层次中,对多媒体场景中的所有元素可以进行随机存取;对于由不同数据类型元素组成的多媒体场景,应能随机存取场景中的每一个元素。
39. Requirements 6Error Resilience – it shall be possible to selectively protect the various elements in a multimedia scene against channel errors with relevant error patterns, e.g. mobile networks, ATM networks or storage media.
出错恢复:对多媒体场景中的各种元素,应能有选择地进行保护,以免受到通道错误的影响,如移动网络、 ATM网络或存储介质错误等。
40. Requirements 7Interaction – it shall be possible to interact with the various elements in the multimedia scene, with a fine granularity, both in space and time; for a multimedia scene composed as a combination of various elements of the data types mentioned above, it shall be possible to independently interact with each one of them.
交互:无论是时间和空间,应能以细粒度方式与多媒体场景中的各种元素进行交互;对于由不同数据类型元素组成的多媒体场景,应能单独同其中的每个元素进行交互。
41. Requirements 8Synchronization – it shall be possible to synchronize all the elements in the multimedia scene as well as other relevant data.
同步:应能对多媒体场景中所有元素以及其它相关数据进行同步。
42. Requirements 9Multiplexing – it shall be possible to multiplex the coded data corresponding to the various elements in the multimedia scene as well as other relevant data.
复用:应能对多媒体场景中不同元素的编码数据以及其它相关数据进行复用。
43. Current Situation编码问题已研究了很多年,已经有了很多有名的标准,涉及许多种媒体, 满足了不同类型的需求. 例如:
still pictures: JPEG, JPEG-LS and JPEG2000
frame-based video: H.261, H.263, MPEG-1 and MPEG-2 Video
object-based pictures and video: MPEG-4 Visual.
audio: MPEG-1, MPEG-2 and MPEG-4 Audio,
speech: G.723, G.728 and MPEG-4 Audio.
synthetic visual content, VRML and MPEG-4
synthetic audio content , MIDI and MPEG-4 Audio.
尽管这些标准在满足相关需求方面已相当完整,但现有标准并不完全满足某些需求。