FIMS, SOA and Media Applications
An interesting white paper by David Austerberry.
Showing posts with label SOA. Show all posts
Showing posts with label SOA. Show all posts
SOA: The Business of Broadcast
Broadcasters have many challenges as the media business evolves, driven by new consumer devices and the increase in mobile viewing. National broadcasters are facing more competition from global operators. New entrants like YouTube and Netflix have changed the VOD landscape. Broadcasters that once aired one channel now air a multiplex of linear channels, as well provide catch-up and mobile services. Summing up, broadcasters must deliver to more platforms, linear and on-demand, in a more competitive business environment.
To meet these challenges, a business must become more agile. Many other sectors have faced similar challenges, and part of the solution for many has been to turn to new software applications, particularly Business Process Management (BPM) and the Service-Oriented Architecture (SOA). Although each can be used stand-alone, BPM and SOA are frequently used in concert as a platform to improve the performance of a business.
Operations that use videotape were constrained by the need for manual handling, but as content migrates from videotape to digital files, the way is open to use IT-based methodologies, including BPM and SOA, to aid broadcast operations.
What is SOA?
SOA is a design methodology for software systems. SOA is not a product, but an architecture to deploy loosely coupled software systems to implement the processes that deliver a business workflow. SOA provides a more viable architecture to build large and complex systems because it is a better fit to the way human activity itself is managed — by delegation. SOA has its roots in object-oriented software and component-based programming.
In the context of the media and entertainment sector, SOA can be used to implement a “media factory,” processing content from the production phase through to multi-platform delivery.
Legacy Broadcast Systems
Traditional broadcast systems comprise processing silos coupled by real-time SDI connections, file transfer and an assortment of control protocols. Such systems are optimized for a specific application and may provide a good price/performance ratio with high efficiency.
The tight coupling of legacy systems makes it difficult to upgrade or replace one or more components. These applications are typically coupled via proprietary APIs, as shown in Figure 1. If the software is upgraded to a new version, the API can change, necessitating changes to other applications that are using the API — work that is usually custom.
It makes it difficult to extend to add new functionality to the system to meet the ever-changing demands of multi-platform delivery and evolving codec standards. Storage architectures change with object-based and cloud storage to become alternates to on-premise NAS and SAN arrays.
When vendors upgrade products, the new versions often do not support legacy operating systems, leading to the need to replace underlying computer hardware platforms.
Traditional systems are just not agile enough to easily support the new demands of the media business. Often new multi-platform systems are tacked on to existing linear playout systems in an ad-hoc manner to support an immediate demand. The system grows in a way that eventually becomes difficult to maintain and operate.
Monitoring — No Overall Visibility
Traditional systems also suffer from a lack of visibility of the internal processes. Individual processes may display the status on a local user interface, but it is difficult to obtain an overall view (dashboard) of the operation of the business.
As broadcast strives for more efficiency, it is vital to have an overall view of technical operations as an aid to manage existing systems and guide future investment. Many broadcasters already have end-to-end alarm monitoring, but resource usage may only be monitored for billing purposes, and not to gain intimate knowledge of hardware and software utilization.
SOA in Media
SOA is not new; it has been in use for a decade or more in other sectors including defense, pharmaceuticals, banking and insurance. It developed from the principles of object-oriented software design and distributed processing.
If SOA is common in other sectors, why not just buy a system from a middleware provider? The problem lies with the special nature of media operations. The media sector has lagged other sectors in the adoption of such systems for a number of reasons. These include the sheer size of media objects and the duration of some processes. A query for an online airline reservation may take a minute at most; a transcode of a movie can take several hours. Conventional SOA implementations are not well suited to handling such long-running processes.
What is a Service?
A service is a mechanism to provide access to a capability. A transcoding application could expose its capability to transcode files as a transform service. Examples of services in the broadcast domain include ingest, transform, playout, file moves and file archiving. The service is defined at the business level rather than the detailed technical level.
It could be said that many broadcasters already operate service-oriented systems; They just don’t extend the methodology to the architecture of technical systems.
Services share a formal contract; now service contracts are commonplace in broadcasting and across the M&E sector, with companies calling on each other for capabilities such as playout, subtitling and effects, as examples. The service-level agreement for playout will include quality aspects such as permitted downtime (99.999 percent). Service contracts operate at the business level, and ultimately may result in monetary exchange.
Abstraction
The business management logic may call for a file to be transcoded from in-house mezzanine to YouTube delivery format, but not define the specifics of a particular make and model of transcoder or the detail of the file formats. This abstracts the business logic from the underlying technical platforms. A generic service interface for file transform can be defined, and then each transcoder is wrapped by a service adaptor that handles the complexity of the transcode process. To the business logic, the transcode is simply a job. The abstraction of the capability is a key principle of the SOA.
In a legacy system, the ingest job is delegated to an operator who configures an encoder, and then starts and stops the encoding at the appropriate times. The operator is functioning autonomously during the processes of the job. These concepts of delegation and autonomy are key to the SOA design philosophy. The encoding may well be automated as a computer process, but the principles remain the same.
Because the service is abstracted, it opens the way for broadcasters to leverage cloud services more easily. As an example, at times of peak transcode demand, a cloud transcode service could be used to supplement in-house resources. With a standard service interface for transcoding, the implementation can be an on-premise or cloud-based service. The operation of the services is orchestrated by a layer of middleware, software that manages business processes according to the needs of the business.
A transform service can be used for different business processes. For example, a transcoder could be used to transform files at ingest to the house codec or used to create multiple versions of content for multi-platform delivery. The transform services can be redeployed to different departments as the needs of the file traffic change from hour to hour.
Planning for a SOA
Migrating from traditional tightly coupled systems to use SOA principles is a big step for a media business. The efficient operation of SOA requires detailed analysis of business needs and definition of services. It also requires rigorous planning of the IT infrastructure, computers and networks for efficient operation of the services. Without the involvement of senior management down to IT services, the benefits of SOA are unlikely to be fully realized.
For broadcasters used to running departmental silos, many with real-time elements, the move to SOA will be a radical change to the way the business operates. However, the advantages of the SOA and allied systems like BPM are proving attractive propositions for the broadcaster — or service provider — running complex file-based operations for multi-platform delivery.
The problems facing a media company looking to embrace SOA and BPM include change management and the sheer problems of keeping on-air through such huge changes in the technical infrastructure supporting broadcast operations.
Many media companies have embraced the architecture, with early adopters using considerable original development of such components as service adapters — the vital link between a service like transcoding and the workflow orchestration middleware.
The use of consultants or internal software services to build a media SOA will achieve the goal, but does it make sense for all media businesses to go their own way?
FIMS
It was this issue on which the Advanced Media Workflow Association (AMWA) and the European Broadcasting Union (EBU) independently agreed when, in 2010, they decided to pool resources and set up the joint Framework for Interoperable Media Services (FIMS) Project, which would develop standards for a framework to implement a media-friendly SOA.
The road will be long, and many obstacles remain to be resolved, but the success of this project will benefit both vendors and media companies in the long run.
The FIMS solution aims to provide a flexible and cost-effective solution that is reliable and future-proof. It should allow best-of-breed content processing products to be integrated with media business systems.
The FIMS team released V1.0 in 2012 as an EBU specification, Tech 3356. Three service interfaces have been specified: transform, transfer and capture.
The FIMS Project has expanded on the conventional SOA with additional features to meet the needs of media operations. Specifically FIMS adds asynchronous operation, resource management, a media bus and security.
Asynchronous operation allows for long-running services. A transcode may take hours; conventional SOA implementations allow for processes that complete in seconds or minutes.
Although services are loosely coupled to the orchestration, jobs can still be run with time constraints. This may be simply to start a job at a certain time, but services can also be real time, like the capture and playout of streams. In these cases, the job requests for the service will also include start and stop times for the process. For playout, this concept is no different from a playlist or schedule.
SOA typically is based on an Enterprise Service Bus (ESB) that carries XML messages between service providers and consumers. A media bus provides a parallel bus to the ESB to carry the large media essence files. Many file-based operations will already have media IP networks that can be adapted to provide the platform for the media bus, as shown in Figure 2.
The Future
Software methodologies like SOA and BPM help media businesses manage file-based operations in more efficient ways and better serve the needs of multi-platform delivery. They provide a holistic approach to running business operations that provide better visibility of operations and simpler ways to leverage cloud services.
They have proved successful in other sectors and are ready to meet the unique needs of the media sector.
By David Austerberry, Broadcast Engineering
Media Services: Services Architecture Enables Functional Flexibility
Unfortunately, the term “media services” contains two overloaded terms — “media” and “services.” In this case, when we talk about media services, we are talking about small (some would say atomic), network-based applications “services” that perform simple, focused tasks.
These tasks are somehow related to either essence or metadata used by professional broadcasters, post-production facilities and the film industry.
An example of a “media service” might be a service sitting out on a network that is available to transcode content from one popular video format to another. One can imagine a host of services, including tape ingest, QC and file movement. Each of these services is available out on the network, and can be used to perform a discrete unit of work. Services perform higher functions by grouping a number of atomic services together in a logical way. But, at their core, media services are small, discrete pieces of software that can be combined in different ways to perform work.
This is a significant departure from traditional media infrastructures, where an ingest station consists of tape machines, routers, monitors and other hardware — all hard-wired together to perform a specific function. In fact, entire broadcast chains are built this way. They are highly optimized and efficient, but they can be very difficult to change. And, if one thing is certain these days, it is that change is a permanent part of our business.
A media services architecture allows discrete blocks of functionality to be combined to build complex workflows. As workflows change, blocks can be recombined into modified workflows. If new functionality is added, new services can be deployed. Additionally, discrete services may be used in multiple workflows. So, a transcoder may be deployed in a post-production scenario for one job and then redeployed in a conversion for web applications next.
Building Workflows
When using media services, it is not enough that the services are available out on the network; something must consume those services in order to perform valuable work for the organization. There are several approaches to using services, but, for this article, we are going to focus on two of them — orchestration and event-driven architecture.
Orchestration
Orchestration systems sit on top of media services and use media services to move work through a defined pipeline from start to finish. For example, an orchestration system might have a workflow that ingests a tape, transcodes the content and then saves the file on a large central server.
The orchestration system tracks the progress of the workflow, calling on various services to work on the job as it moves through the pipeline. The orchestration system is responsible for not only dealing with normal flows, but it is also responsible for dealing with error conditions such as a failed transcode. Orchestration can start out simple, but it can become complicated as engineers consider all of the various states and error conditions possible in the workflow.
Event-driven Architecture
Event-driven architecture is another way to use services to perform work. At a high level, in this architecture, something causes something else to happen (an event) that is of significance to the business. Processing engines can be set up to listen for that event, and when the event happens, they can perform actions based on the event. Other processing engines can be listening down-stream, and when one event engine finishes, others can be triggered. In an event-driven architecture, there is no central system guiding the flow of work through a pipeline. The movement of work through the facility is caused by a sequence of events and processes.
An operator finishing an ingest activity might create an “Ingest Complete” event. Event processing engines subscribe to event channels, such as the “Ingest Complete” event channel. This particular event-processing engine might have two actions: the first is to notify the QC operator that the file is available for quality control checking; and the second is to publish an “Ingest Complete” notification for other systems that might be interested in the event, such as traffic and automation systems. Both of these systems might update the status of the media based on the “Ingest Complete” event.
Note that, in this example, it would be extremely easy to add another event process engine to the event channel. This engine might be responsible for creating a number of different formats from the original ingested file format. Adding this process does not require modifying a workflow in a central system. All one has to do is to subscribe the transcoding engine to that particular event channel.
SOA and EDA
It is important to realize that orchestration and event-driven architecture are complementary, and they are frequently deployed together. For example, in the earlier referenced figure, the tape ingest function might be driven by an orchestration system that precisely controls workflow and error-handling conditions. Event-driven architecture would be used to notify other SOA processes once the ingest is complete.
Common Approach
Common service interface definitions are critical. One can imagine a whole universe of services: a content repository service, a media identification service, a publish content to ISP service, and so on. And, one can imagine that several different vendors would make such services available. If each vendor defined the interface to their service independently, the amount of software integration required to build these systems would be huge. On the other hand, if the industry would agree on the service interface definition for an ingest service, for example, then it would be possible to integrate various ingest services into a workflow with minimal additional development.
Common service interface definitions are critical, but it is also critical that we have a common overall framework within which services can be deployed. How do services communicate with orchestration systems and with each other in event-driven architectures? How do newly commissioned services make their presence known on a network? Again, having a harmonized approach to an overall media service architecture will lower costs and shorten implementation time.
Governance
One last critical element in the discussion of media services is governance. Governance brings logic and structure to media services. Areas typically in governance include: service life cycle (how services are developed, deployed, deprecated and eventually decommissioned); helping to prioritize the deployment of new services; and how the quality of deployed services can be ensured.
FIMS
There is a task force in the industry called the Framework for Interoperable Media Services (FIMS). FIMS is a collaboration between the Advanced Media Workflow Association and the European Broadcasting Union. FIMS is the first industry effort focused on developing services for the professional media industry. The FIMS group consists of a Business Board that develops business priorities for service development, and a Technical Board that oversees the development and deployment of FIMS services.
You can learn more about FIMS at http://www.fims.tv. For technical information, you can visit the FIMS wiki at http://wiki.amwa.tv/ebu. This activity has already yielded an overall framework for media services and several specific media service definitions. Work is ongoing, all the work is public, and anyone can participate.
By Brad Gilmer, Broadcast Engineering
IT Acronyms Explained
As broadcast facilities and post-production houses implement new workflow and media management systems, they deal with an array of new IT-based technologies and are inundated with acronyms associated with those technologies, as well as the best practices concerning their use. Adoption of IT-based technologies continues apace with continued convergence across the media industry.
Despite this rapid shift of operations into the IT realm, however, the industry as a whole has not been as fast in providing engineering staff with an education on the meaning of terms such as XML, WSDL, SOAP, SOA and REST, the technologies underpinning them, and the role they play in present and future media-focused operations. This article will review the most relevant acronyms, their fundamental principles and their typical applications.
XML and XML Schema
Complex control systems depend on a reliable interchange of information in order to make the myriad business decisions that guide material through workflow. This is made simpler if individual systems can swap information in the form of structured data — that is, documents that contain both content and an indication of the content's role in the document.
The concept of structured data is not new; humans have been dealing with information formatted this way for centuries. Examples include published (human-readable) books and magazines, as well as web pages displayed by a browser. In both cases, these systems indicate to the consumer the relevance and importance of any particular item on the page through the use of typographical hints — such as underlining text that hyperlinks to other documents. HTML achieves this by including “tags” in the document, and the browser uses those tags to determine how to emphasize the relative value of the associated data.
As a markup language, HTML represents a fairly limited set of standardized tags intended for the presentation of web content, so its utility beyond that scope likewise is limited.
XML, however, offers flexibility that makes it more suitable for commercial use. This is true because, unlike HTML, XML allows users to define their own tags. Rather than defining the tags themselves (or the semantics of the tags), XML provides a facility for defining tags and using them within a document. As with HTML, data is then bracketed within opening and closing tags, which the receiving device can read and act on as appropriate.
Here is an example of the use of a tag “author” to identify a book's author in a library document:
<author>William Shakespeare</author>
Because XML tags are user-definable, an XML Schema is used to define which tags are valid in a particular document. Such definitions describe, for example, elements that can appear in a document (along with their attributes), whether an element is empty or can include text, and the data types for elements and attributes.
Thus, a document can be compared to an XML Schema — acting much like a blueprint, style guide or template — to ensure the document is valid and contains all relevant information. Again, this idea is not a new concept solely used by XML. Most databases have some sort of schema. Also, textbooks have used schemas for years, generally referring to them as “style guides.”
Here is an example of an XML schema for a memo:
SOAP and SOA
Simple Object Access Protocol (SOAP) is a lightweight protocol for the exchange of information. It describes three main elements: an envelope, the encoding rules, and a convention for the representation of remote procedure calls and responses. SOAP is no longer used; however, the technology remains very much in use.
Though it isn't used, the SOAP acronym is sometimes confused with Service-Oriented Architecture (SOA). Though SOAP standards may be part of an SOA application, the acronyms are not related.
SOA is a design philosophy that separates the core functions of a business into independent modules. These modules are referred to as services — essentially software functions — that are called by one or more other services in the overall workflow. (Media transcoding is an example of a service that could be invoked as part of an overall workflow management system.)
Operating on the principle of loosely coupled systems, SOA abstracts the functions of a service from the low-level substructures of that service, leaving the calling service to use much higher-level language to drive the process. In doing so, SOA can make it easier to choreograph multiple business activities and processes among multiple complex software systems, all working under the control of a central process to achieve the required goal.
Consider the operation of a typical media enterprise: Material comes into a facility on tape, via satellite or as a file transfer, and content from each of these sources must be processed before it can be used. Tapes must be ingested, satellite feeds must be fed through some sort of IRD, and files must be received and error-corrected. Some form of transcoding may then be applied in order to provide the material in the house format. A QC stage may then be applied to ensure technical compliance. In the software world, each of these would be software modules in a SOA-based system.
Without SOA, setting up such systems is a time-consuming process that likely will demand customization so that one vendor's automation/asset management system can talk to another vendor's software module or hardware processor. Any operational or technical changes may require replacement of a module or augmentation with another vendor's module.
For example, in a traditional workflow, the automation system must have deep knowledge of the proprietary API calls required to tell a transcoder to transcode a particular piece of material to another format. Every time a new format is added to the transcoder by its manufacturer, that interface must be modified (or in the worst case, completely rewritten). The custom integration software capable of addressing systems from different vendors thus presents a high-potential investment in time and money.
In a more agile SOA-based architecture, the automation system would simply send an XML message that says, “Transcode file X to format Y, and store the result as Z.” This message remains consistent, even if a new transcoder or format is added. Regardless of the vendor, equally equipped systems will perform the same basic transcode when given the same command. Knowing this, engineers can take a more flexible approach to workflow design. This model depends on the fact that each service describes itself and its capabilities to the rest of the system. Web Services Descriptive Language (WSDL) facilitates this exchange of information.
WSDL is an XML-based language used for describing features and capabilities of a particular service. Provided to the central controlling system, this information ensures all other services and applications are aware of a particular service's capabilities.
Four critical pieces are supplied:
- Interface information describing all publicly available functions;
- Data type information for all message requests and message responses;
- Binding information about the transport protocol to be used in calling the service;
- Address information for locating the specified service.
REST
Representational State Transfer (REST), an architecture for distributed systems such as the World Wide Web, is targeted at HTML rather than XML. It addresses the scalability of component interaction, generality of interfaces and deployment of components to reduce latency and enforce security of transactions. Unlike SOA, which often is considered peer-to-peer architecture, REST is oriented toward client/server interaction where clients initiate requests to servers, and servers process requests and return appropriate responses.
In a RESTful transaction, requests and responses are built around the transfer of representations of resources, which themselves are specific information sources. A representation of a resource is typically a document that captures a resource's current or intended state. Each resource is referenced with a global identifier. To manipulate these resources, components of the network — typically user agents and origin servers — communicate via a standardized interface like HTTP and exchange representations.
In the case of web page information, the raw data on which the page is built is the resource, but the representation of that data could be different for different users. What is representation of a resource? Consider a human using a browser to view a web page in order to retrieve information, and a computer doing the same as part of some larger activity. Both navigate to the same page, but the computer has no interest in the page's layout and styling; it just wants the data. The human does care about layout and styling (which, after all, exists to make the page easier for humans to digest). The two clients (human and computer) want different representations of the resource, both derived from the same raw data source.
While SOA is object-oriented, with the object the service with which other services interact, REST is point-to-point and relies on “GET” commands to a specific URL. It enables code delivery in “applets” as needed, and this capability can reduce basic complexity of any client application. Specialized code (like JavaScript) can be delivered along with data and used to customize how data is presented or obtained.
By Paul Turner, Broadcast Engineering
FIMS and SOA
Broadcasters no longer have the monopoly on the delivery of A/V entertainment to the home. In the fiercely competitive world of media and entertainment, companies have to deliver more versions and formats, but without increasing their costs. A channel is now expected to have a Web presence, as well as mobile and tablet versions of their content.
Organizations like the EBU and Advanced Media Workflow Association (AMWA) are promoting the Service-Oriented Architecture (SOA) as a route to provide the interoperable media services that can serve the new business requirements.
For the seasoned video engineer, the world of SOA introduces terms and concepts that at first encounter seem foreign and more suited to the IT specialist. As video processing migrates to the file domain, there is no option but to become familiar with what at first sight may appear alien.
Broadcast systems have evolved around the imposed workflow of the serial processing steps of videotape operations. Over time, many processes have moved from dedicated hardware boxes with SDI in and out to software applications on a network. A typical broadcast operation is now a hybrid of SDI and IP connections.
In many cases, the workflow remains as the original tape-based flow. Over time, other applications like asset and workflow management are layered over the entire process chain. The system has grown by accident, not by design, and become a web of custom or proprietary interfaces linking the many applications.
Sure it works, it was designed that way, but when the time comes to replace a component part — say the playout automation — the inflexibility of the system rapidly comes apparent. The parts of the system are linked by a web of custom APIs, often restricted to a specific release of a specific software application. It is just not possible to swap out the automation for the latest product without attending to the web of interfaces.
To meet the demands for new services to the public, the broadcaster must add facilities for a mobile news service, a 3-D channel and interaction with a social media website. Along comes CES and some new consumer device to consume content. How do you add support for this new device? Will it mean more custom interfaces or more special workflow applications? The EBU and AMWA are developing a Framework for Interoperable Media Services (FIMS), which aims to provide a new technology platform that leverages current IT practices, like the use of the SOA, to provide business agility and to control costs.
The legacy system architectures suffer from many problems, and these scenarios serve to illustrate just two.
Scenario One: Ingest
Take the example of ingest. A VTR connects to a video noise reducer via HD-SDI. This then connects again via HD-SDI to an encoder card. The card encodes to a suitable codec for editing and also creates a low-resolution proxy.
At some point this is updated so that the encoder card ingests directly from the VTR and encodes to an uncompressed format. The noise reduction is performed in software. The uncompressed file is saved in a watch folder and picked up by transcoder software, which creates files in the wanted resolutions and codecs.
Imagine more control is needed over the clean up, and the file is passed to an operator who will apply craft skills to the clean-up and repair process. Any changes to the process flow involve wiring changes and software reconfiguration.
The system is just not flexible or agile enough for the constant changes needed to serve today's requirements.
Scenario Two: System Upgrades
A broadcaster has a transcoder that is interfaced to a DAM. The broadcaster starts with Transcoder A and DAM A. The transcoder is replaced by a later model from a different vendor, Transcoder B. This requires a new interface to the DAM. The broadcaster migrates to a new DAM B, so the interface to Transcoder B must be rewritten.
The broadcaster liked the compression quality of Transcoder A and decides to move back, but must pay for another interface to the new DAM. Next, the vendor for Transcoder A has to redesign its product to move from a 32-bit to 64-bit operating system. In the process, the API is upgraded, and a new SDK is issued. The custom interface must be similarly upgraded.
Using two transcoder vendors and two DAMs has required the development of five custom interfaces over time. This adds up to costs for professional services, and for the manufacturer, the opportunity cost of not using software resource to develop new products. It is an unsustainable business model.
Integrating an SOA
The implementation of an SOA in a media business must take into account the special needs of the processes that manage A/V files. Off-the-shelf IT products do not cater for these additional requirements without extensive customization. The work to develop FIMS will create a set of standards for vendors to create compliant products and save unnecessary professional services.
This melding of IT with a media-savvy system means that the video engineer and IT specialist can collaborate and share their skills in the creation of products and systems that meet the new demands of the media and entertainment sector.
Services
The linear nature of legacy production is implemented by serial processes acting on the content: ingest, editing, transcoding, distribution, playout and archive being typical. It may be that content is transcoded at ingest — AVC-Intra to DNxHD, for example — and then transcoded after editing to different delivery formats. This could be done by a transcoder in the edit bay and a transcoder in master control.
In a file-based environment, devices all sit on the media network. It makes sense to pool all the transcoders as a central resource and make them available as a transcode service. It could even be that cloud transcoding could be used. This offers much more efficient use of transcoder resources, but for it to work, the service should be reusable at any point in the workflow and should interoperate with all the necessary equipment in the organization, whatever make or model.
This calls for standardization so that a transcode service has a common interface, whatever make or model, and capabilities. A given transcoder may not support all codecs, but the interface can still be common. If demand increases, new transcoders can be added; the architecture scales easily.
The move to an SOA involves a change of mind-set to view processes within the organization as services. Since most post houses have a rate card of services they offer, it is not a great leap to understand this concept. The services include ingest editing, grading, dubbing, encoding, sound mixing, etc. Note that some services require extensive and lengthy use of creative staff; services are not only computer processing operations like transcoding.
Encapsulation
A post facility can take a videotape and return a copy of the tape encoded on a Blu-ray disc. All the detail of the process is hidden. You don't need to know which tape deck was used or what software was used to burn the disc. It is all abstracted from the client. The facility performs a service and bills for that service; it is a business transaction.
In an SOA, this concept of abstracting, or encapsulating, the detail of the process is key. If the post house starts using different software, or a different VTR, you don't care as long as the disc is a faithful copy of the tape.
An SOA uses loose coupling for services, possibly a foreign concept for video engineers used to real-time systems. But many services are already loosely coupled. Captioning is one example. A low-resolution proxy is sent out to the captioning facility, and at some point in the future a caption file is returned. The process is managed at a business level by a scheduling department.
Service Adaptors
Many existing products need the user to control the service via a custom API using an SDK provided by the vendor. In an SOA, services are linked to the middleware via service adaptors. These provide the abstraction from the implementation of the service — how it works — to deliver the service at the business level.
The FIMS project is working to define standard interfaces for common media services. A system with FIMS components contains two broad service categories: workflow services able to realize a given business goal (media workflow services), and infrastructure services that are essential components of the media SOA system (media infrastructure services).
The first three workflow services to be developed are:
- Transfer (moving files)
- Transform (changing essence or metadata)
- Capture (ingest stream to file)
It is envisaged further services will be added to the list as dictated by demand from vendors and users.
Building an SOA
In all but the smallest facility, some middleware is required to manage the services, scheduling resources and the workflow. To link services together to implement broadcast workflows, the SOA middleware orchestrates job requests, calling appropriate services to satisfy a request. There are many SOA middleware systems, but few are “media aware.” It is hoped that FIMS will make it easier for several manufacturers to offer compliant products.
One important term is “web services.” Web services are used for communications between the many components of the SOA implementation. SOA is just an architecture — a methodology; it is not a software application.
Web services have become a commodity, with companies like Amazon and Google offering storage, processing and many other services in the cloud. Web services are not exclusive to the Web, but can equally be used on the company LAN.
Web service standards have two interaction styles: SOAP/WSDL and RESTful. RESTful only operates over HTTP. SOAP is agnostic to transport protocol, so it can use other protocols like TCP. SOAP was designed for distributed computing, whereas RESTful is a lightweight protocol for point-to-point communications. Both can be used to bind a service provider with a service consumer. FIMS uses an object model described by XML schemas, which provides independence from the method used to bind services.
Media-Centric Issues
File-based media systems need to allow for certain issues not found in typical SOA implementations. One is the very large size of media files, terabytes in some cases. Media files cannot travel as attachments on a conventional enterprise service bus, the interconnection bus of an SOA. Instead a separate media service bus is used. This concept is common in broadcast facilities where a high-bandwidth media network carries files separately from the main network, which carries control messages, e-mails etc.
A typical SOA service, like converting a Word document to PDF, happens in seconds. Media processes can take hours or even days. Such operations must happen asynchronously from other processes. Operations such as transcoding must be scheduled to business rules to ensure resources are not used for a low-priority job, holding up a high-priority job.
A resource manager service must manage use of services according to business rules that manage the needs of broadcast operations.
The media bus is a potential bottleneck to scaling of services, so SLA monitoring is vital to ensure efficient running of the media systems. To the video engineer, used to non-blocking systems, with dedicated circuits and crosspoint routers, it is easy to keep track of bandwidth requirements. In a file-based environment, care has to be taken over the use of the media bus though continual monitoring of capacity and performance.
Summary
The AMWA and EBU have now joined with SMPTE to move forward standardization of the framework. After formalization by AMWA and EBU, the work will be submitted to SMPTE.
The standards work is not following a big-bang approach, but progressing in steps, with usable specifications released at regular intervals so that vendors and media companies can start deploying systems immediately, rather than waiting for a long, drawn-out standards process, which has been the case in the past. Businesses have to move now; there is no option to wait for an all-encompassing answer to everything in the media factory. By releasing the framework, and then key adaptors, further work will proceed as the demand arises from the community.
The move to an SOA, and to follow the FIMS route, promises many advantages to the broadcaster. The architecture is more flexible and scalable than legacy systems. The broadcaster can more easily outsource to external services or cloud provision. And finally, the system — through the dashboard — gives management a better view of key performance indicators of the business. The system agility and the better control gives management the ability to invest in new services and to improve the efficiency of existing operations.
By David Austerberry, Broadcast Engineering
Subscribe to:
Posts (Atom)
