When advanced multimedia features started to get introduced into the mobile phones a few years ago, one of the common feedbacks was: “Yeah, cool, I like it a lot but there’s no way I’ll be able to integrate that feature with my existing design as it would take tens of man-years of effort … “
So where’s the problem? … in order to better understand it, let’s use an example of upgrading video decoding capabilities of a mobile phone Application Processor by adding to it a HD Video co-processor.
Integration of such a video co-processor would cross several system boundaries and would require access to many shared system resources like display rendering, audio decoding and mixing, microphone, camera and file storage. The shared resources have to be accessible remotely from the video co-processor … and that’s one of the key design challenges in the tightly coupled architectures widely in use today!
The system resources are owned by the Application Engine and due to very monolithic design they cannot be simply shared. Therefore, there’s a tendency to duplicate the required resources and add them directly to the video co-processor. By doing this, you not only increase the overall system cost but also the complexity of your video co-processor. Instead of focusing on delivering the best video experience you should also think of implementing all the “extras”.
However, in the Smart Phones segment, just like in the PC, no single company will supply all the components to the Mobile Computers … that makes it impossible to integrate all required shared resources in our co-processor … we have to learn how to talk to modules that come from a different vendor …but how to do it if solutions from different vendors have evolved over time independently and are not free from their own legacy and proprietary solutions.
That’s where NoTA (Network-On-Terminal Architecture) came in … The NoTA approach promises to change all that and implements a distributed, modular system. This style of architecture approach is fully aligned with our research and advanced development programs addressing Interconnect Centric Architecture using MIPI interconnect.
The phone is no longer centered around a single, powerful application engine, but instead is built as a network of so-called subsystems. Each subsystem, or module, using its own resources performs a specific task such as application processing, storage, connectivity, or multimedia. Communication between the subsystems is service based and implemented by means of service nodes that can be based on the open standards.
With this modular approach it’s relatively easy to swap in new hardware and software functions like the video co-processor. We can optimize it for peak performance and can develop it separately, without impacting other portions of the design. By using standard and open interfaces we can also talk to the services implemented by the subsystems from different vendors.
Availability of so called proxies for the multimedia services based on open standards like OpenMAX is another important element to ease the overall system integration of the video co-processor. The proxy mechanism allows the Application Engine to access remotely the video co-processor multimedia services like the OpenMAX High Definition Video decoding. On the video co-processor side you just have to connect OpenMAX components to the proxy without dealing with much more complex SW of the Application Engine.
A modular and service based approach to build a system doesn’t have to be complex when using standard interfaces. We will demonstrate a prototype system with distributed AV playback using NoTA during the 1st International NoTA conference … It will show how you could Just Click-In a video or audio co-processor and enjoy your next experience …
Comments
Understanding NoTA
Marcin
I'm still trying to work out NoTA. Whilst I understand the generic value of standardising interfaces within a context, to understand a particular proposal (such as NoTA) it is first necessary to understand the Context and the Scope, and then to go on to understand the Implementation. After an hour's browsing of notaworld (and another hour registering ;-) most of the material is about Implementation ... But your 'article' is the only content I have found so far which helps me towards my Context and Scope objective. Though it does not take me all the way, it certainly helps ... So thank you for that.
So, if I may question particular points I may yet find the answers I need ...
Par1-5: Nice introduction, I can understand this particular Context and an example of the problem NoTA is 'going' to solve.
Par6: Ok; NoTA concept is Modular and Distributed ... which presumably means that it maps to the sort of distributed 'components' that would be in an Embedded System. HW and SW objects which could be on different 'engines' but are all essentially 'local'. (The bit about MIPI just confuses me at the moment, though I am encouraged to see that this is compatible with MIPI)
Par7: Ok; A NoTA system consists of a collection of *independent* but networked subsystems which enact their function when called to do so by any other subsystem in the network. Being applicable to Embedded Systems, I am guessing that the subsystems are always hardware and software ... such that all components express a SW_Shell, but that their functional content may be any mixture of HW/SW and even Hierarchy. This includes all-software subsystems, which presumably might run as a Process or Thread on an application engine.
... But wait. You also say here that each subsystem uses its own resources. Which at first glance appears to contradict with the shared resource *problem* identified in Par3-4. I must assume that this is not quite as it seems, and that NoTA allows the transparent sharing of Resources, such that each subsystem believes it is the sole owner of those resources. Thus when a Video Processing subsystem is initiated, it finds input data in a block of its memory, and writes the output to a block of its memory. This is ok, but *can* be an inefficient use of resources (eg memory) as blocks remain committed when there is no longer active use. For example the 'Video Processor' has no knowledge of when its output data is taken ... and any data-taker has no knowledge of other takers with access to that data. But I guess this is a detail that NoTA Implementation addresses.
Par8: I can understand that this approach would make it relatively easy to swap new hardware/software. But there will be some performance penalty for this modularity.
Par9: Ok; Its good to hear that it is also compatible with OpenMAX as well as MIPI. I 'read' this as NoTA gives location/implementation independence to the fixed implementation nature of OpenMAX. And presumably this also applies to MIPI. It suggests to me that MIPI/NoTA/OpenMAX (or maybe Khronos) are a *complementary*set* of standards which must be developed together, as it is unreasonable in the longer term that MIPI and OpenMAX will operate in other than a distributed world.
Par10: Now I have a better idea of what the demo was actually doing.
So in addition to "Have I got the above broadly right?" just one additional question ...
Where is Configuration and System Control managed for such a system? (Is there a master somewhere? Or are all subsystems truly independent?)
Thanks.ian
Ian Phillips - Principal Staff Engineer, ARM Ltd
Where is the control?
Ian,
There are at least three different answers to your question.
1. NoTA infrastructure has one centralized element; that is Resource Manager Service, RM. RM is taking care of service registration which includes service authentication, SID allocation, and optional security features. In practice this means that all services need to register them selves to the RM to be available through NoTA interconnect.
2. There are also some system level functions which typically are implemented in centralized manner. Good example of such function is device level power management service. In most cases power management will be hierarchical with independent and intelligent local power management in sub-systems and then high level centralized device level power management service to take care of policy settings and high level guidance of constraints regarding the available power and energy.
Security, system level event handling and high level error management are other typical functions that need, to some extend, centralized entities. In all these cases the way to implement them is through service concept.
Partitioning of the control is a system design decision and NoTA architecture as such do not prevent designer to partition the control as he/she wishes. However, there are some design recommendations to keep sub-systems as independent as possible:
- black box approach
- loose coupling
- the control with thigh timing constraints should be local
- resource sharing only through service interfaces
3. The third control partitioning issue relates to applications. In principle, NoTA gives designer full freedom to do either very centrally controlled traditional applications or build applications using very independent, high level and distributed services. Due to the fact that in NoTA there are no master and slave relation between sub-systems the real control of the functions can reside anywhere in the system depending on the intentions of the system designer. It is also good to note that NoTA architecture do not have concept of host or application host - application nodes (AN) can be implemented in any sub-system in a NoTA network as is the case also with service nodes(SN).
Having said that, it is still good to remind that typical way to start using NoTA with legacy systems is to use it as a extension architecture in relation to existing platforms or application hosts.
I hope these answers gave some insight how NoTA based systems can be controlled and where the control can end up in different cases.
Risto Suoranta
NoTA Team
what I'm more interested in is ...
thanks for your explanation about NoTA, but could you tell me what kind of device or cellphone on the market is made through NoTA? how is the performance ? after all , real is the powerest case!