Project Objectives

aceMedia brings together a critical mass of experts in the area of analysis of digital content and experts in the area of knowledge representation and learning to collaborate to achieve a major step forward in supporting users in intuitively accessing, managing, communicating, and enjoying collections of content. Bringing content and knowledge together is not only reflected in the consortium but also in the central theme of the project: the ACE. An ACE combines content, metadata, and intelligence into a single entity, allowing automated, knowledge assisted content processing. In addition, aceMedia goes one step further: applications are also considered part of the intelligence layer of the ACE. Therefore, an ACE represents the content, knowledge describing the content (metadata layer), and knowledge describing the applications (intelligence layer). This combination is the key difference from the current state of the art. Currently the content and the applications are separated. For example, a JPEG file does not tell the machine how to process it; it is assumed that the JPEG decoder resides on the system that displays the picture. ACEs contain the application (or links to the application) that is essential for consumption of the content and will thus allow the user to deal with the content in the future, even as systems change over time.

For professional content producers and managers, current systems require powerful hardware and software platforms to perform their analyses on the content in real-time, and to search the huge collections of content needed for a given application. For the consumer the situation is even less mature. Generally, personal media collections have to be manually annotated which is cumbersome and time consuming. aceMedia research will lead to development of tools which enable users to conveniently manage their ever-growing content collections, e.g. by the use of automated semantic annotation, and intelligent search and retrieval.

For content communication, technologies are needed to deal with the heterogeneity of the network environment in which future users will be accessing content. Techniques such as scalable coding are potentially very valuable, as they allow content to be encoded only once, but delivered in any number of formats to suit user preferences, available network conditions, and target devices.

User-oriented development and user interaction objectives

The new aceMedia technology will be a powerful set of algorithms, tools and concepts, which will be made available for users in two novel application systems. This will be done, first, by applying state-of-the-art methods of user-oriented development and user evaluation, second, by developing the appropriate interfaces to fully exploit the new technology in a usable and convincing way, and third, by developing personalisation modules as a basis for adaptivity. While the methods of user-orientation and user evaluation are proven and established, the interface and the personalisation both are challenges where the project cannot build on existing solutions but has to devise and implement special and innovative approaches.

All development within aceMedia will be focused on user needs, and two application systems developed in aceMedia will be evaluated for user-perceived qualities (e.g. usefulness, usability, trustworthiness, and user satisfaction). User-oriented development will be a continuous, iterative process as recommended by ISO 13407. Several HCI challenges will be researched. One such challenge is system adaptivity, which will be based on context modelling, using a user model that learns from the user's interaction, and attaches knowledge about usage to ACEs. Another goal is to design user interfaces which allow users to benefit from knowledge-based processing whilst protecting them from complexity of interaction. aceMedia will develop sophisticated, user oriented, solutions for knowledge-based content retrieval and browsing. Particular HCI research will be undertaken to design the user interface for mobile devices with their specific display and processing constraints. Among the user groups to be involved will be content managers/authors as well as end users. Special efforts will be made to include users that are very often at a disadvantage in accessing content, such as female, elderly or disabled users. The final user evaluation of aceMedia will include users from several European countries, and will apply quantitative state-of-the-art methods.

Knowledge-driven content processing objectives

aceMedia will develop the content engineering technology required to support knowledge discovery for the creation of intelligent adaptable ACEs. It will target the following specific objectives: (i) development, implementation and testing of an advanced content processing toolbox to support knowledge discovery, (ii) research, development, implementation and testing of scalable media encoding techniques able to automatically accommodate changes according to available bandwidth and device capability, (iii) research, development, implementation and testing of middleware to support interoperability across different platforms (iv) development, implementation and testing of content visualization techniques for browsing and navigation.

Candidate technologies that can be adapted and used to develop the codecs needed to support the adaptability aspect of ACEs have already been identified. Content coding produced at the root or any of the main provider nodes using hierarchically embedded coding technology will enable simple decision programs at each node to dynamically and "on the fly" select from the received stream just the amount of data that can be forwarded by each one to their ongoing branches. Consequently, terminals will get the highest possible content resolution allowed by the channel and therefore the best user experience can be offered.

A good example of technology able to produce hierarchically embedded codes for the transmission of video is the Embedded Zerotree Wavelet method (EZW). Although now established in image coding, extensions to general multimedia applications require further investigation. The EZW is capable of encoding content achieving the best bitrate-distortion ratio. Furthermore, the bit stream can be precisely controlled by the bit rate performing optimally for all rates. In other words, for a given bandwidth budget, the EZW code produces the best image quality. The EZW belongs to the family of embedded coders, which are constructed to generate bit streams that satisfy some of the scalability properties needed to realize the ACE concept. Another example of scalable coders is JPEG2000. JPEG2000 is not directly based on the popular EZW schema, it uses a very flexible form of block-bitplane coding. JPEG2000 technology can be adapted to satisfy the requirements of the project, while remaining compliant with the original standard.

Codecs needed for aceMedia will be developed within an appropriate standards framework (e.g. MPEG), in which IPR may be owned by aceMedia partners. Licensing to companies outside aceMedia, within the context of the standards' bodies own IPR rules, is anticipated. Where possible, as in the case of the scalable audio codec, the IPR for codecs needed in aceMedia will be owned by partners. Where this is not the case, we will seek alternatives based on open source technologies.

To complete the technology needed to produce truly scalable multimedia, scalable audio coding and scalable streaming are required. Queen Mary University of London's fine-grain scalable audio codec based on MDCT and Embedded Zero Trees, will be evaluated. The unique feature of this codec is that, after encoding, each coded audio frame can be tailored to any size, with an accuracy of one bit. This is because its internal representation is like a chain of bits, where the bits are in order of significance to the decoder. Thus the tail of the chain may be discarded, leaving a still-decodable head. This way, entirely flexible scalability is obtained. Also, optionally, a simple FEC packet loss compensation technique can be built around the fine-grain scalable codec, which fills in for missing packets by substituting with a lower quality copy that has been delivered in a subsequent packet. Suitable arrangements within the consortium will be made for the use of this technology within the aceMedia project

Techniques for real-time media-adaptation based on analysis of compressed content (i.e. performed without decoding) will be extended to approaches capable of providing summaries of video content at any level of detail. Since current approaches to content browsing, such as key frame lists or mosaicing, have proven to be insufficient, hierarchical strategies for content-based image annotation and retrieval will be investigated using previous work in this field as a starting point. The use of MPEG-7 and MPEG-21 DIA standards will ensure development ultimately results in an open and interoperable framework.

Ontology infrastructure, knowledge-assisted content analysis, semantic reasoning and intelligent content retrieval objectives

Although description of multimedia information has recently seen significant progress, the pace of automatic extraction of such a description, and especially of its semantic part, is rather slow, due to the limitations of state of the art multimedia analysis systems. It is acknowledged that in order to achieve semantic analysis and knowledge mining from multimedia content, ontologies are essential to express the key entities and relationships describing multimedia in a formal machine-processable representation. Ontology modelling and ontology-based metadata creation currently address mainly text-based resources or simple annotation of photographs.

From a knowledge point of view, developments have taken place over some years to represent knowledge and reason with knowledge. One of the goals of explicit knowledge representation techniques is to be able to derive new knowledge by using logic and inference rules. Over the last decade, this area of expertise has gained new interest in the context of the Semantic Web. New languages such as RDF (Resource Description Framework) and OWL (Web Ontology Language) are currently defined by the World Wide Web consortium (W3C) to be able to add meaning to information on the web to allow for better search and retrieval. As a next step, inference rules and logic are to be used by intelligent applications to derive new information from existing information on the web. Ontologies define a set of meanings for a specific domain of information.

aceMedia will produce advances in ontology modelling in terms of both methodology and expressiveness in order to address the additional requirements of multimedia resources. The project will improve the state of the art by applying ontology-based discourse representation and analysis based on semantic reasoning to multimedia resources. Specifically, aceMedia will: (i) study and evaluate state of the art knowledge representation technologies, (ii) provide an ontology infrastructure for automated knowledge extraction and usage in multimedia content creation, management, exchange and consumption, and (iii) build customized ontologies for the specific domains of the aceMedia applications. Ontologies will be extended and enriched to include low-level audiovisual features, descriptors and behavioural models in order to support knowledge-aware multimedia analysis, media-aware semantic inferencing and high-level semantic reasoning, which will provide the means for automatic content annotation and generation of the ACE scalable metadata layer. Finally, innovative techniques for context modelling and analysis will be developed, assisting both the ACE creation process and its usage through user query interpretation and intelligent search, retrieval and relevance feedback. The definition, representation and analysis of context will be based on formal concept analysis (FCA) theory, in combination with neural networks and fuzzy set theory.

System development, integration and application objectives

In the aceMedia project, software components related to user interaction, knowledge-driven content processing as well as knowledge-assisted content analysis, reasoning and retrieval will be integrated to an innovative system, with respect to existing systems. On the one hand, existing systems related to the semantic web approach such as FOAF are based mainly on context knowledge. On the other hand, content-chain systems such as video asset management systems (e.g. from Bulldog, e-motion, Convera, Imagine Products, Lanterna Magica, Mediasite, Mediaware Solutions, Pictron, Virage) or the demonstrator from IST-PISTE are based on knowledge about multimedia content. In the aceMedia system, ACEs will combine content with several types of knowledge (e.g. about content, context, provider etc). Additionally, an intelligence layer will ensure a certain level of adaptability. To prove this novel concept of ACEs, two application demonstrators will be specified, developed and integrated–one in the Personal Content Services domain, and one for Commercial Content Management. A coherent and reliable outcome will be ensured by adopting a common development environment and by using software repositories with version control and software test suites and metrics. The final applications will be checked against the specified requirements to validate the building process. Both application demonstrators will undergo thorough usability testing to assess their real practical value, producing feedback at different phases of the project, thus allowing realignment of work if necessary.