Shulou(Shulou.com)06/03 Report--

From: http://www.ibm.com/developerworks/cn/aix/library/au-gstreamer.html

Multimedia, by definition, represents a variety of media types. You can store audio, video, and metadata in a variety of formats. However, this also means using many tools to deal with these different contents.

GStreamer can help you. By hiding all the different tools and libraries in its plug-ins, and using the general concept of media pipelines, GStreamer can represent operations on different types of media in a unified way. This allows you to focus on existing media rather than confused about what kind of pipeline should be used.

The advantages of this unified approach are obvious. Instead of writing a MP3 or AVI/DivX player, you can write a music or video player. When you want to support another format, you don't need to do in-depth research and write code for the new library. Instead, you only need to install plug-ins in this format. It's that simple. It doesn't even need to be recompiled. All GStreamer applications can be run in a new format.

GStreamer can solve many problems, such as "need to store all audio samples from different sources with the same format." Because all formats are handled similarly, you only need to write one tool. This will save time and make the solution more robust and easier to maintain. Moreover, after you understand the relevant concepts of GStreamer, you can apply it almost anywhere. If you want audio information to flow through the network, you only need to think about this network, because the audio API (application programming interface) and all other operations you use remain the same.

Related concepts

Because of its own nature, GStreamer is at a higher level than ordinary libraries. Therefore, it is important to understand what GStreamer really is and what it can do.

GStreamer is a media processing library. This means that it provides you with an abstract model of some kind of transformation process (consisting of inputs, outputs, and different stages), and allows you to build instances of this transformation process to meet a particular end result and a particular media type. Here are some examples of this process:

Convert MP3 audio file to Ogg Vorbis play AVI movie file capture live performance using IEEE1394 digital video (DV) camera and save it as MPEG-2 stream

To achieve these different results, GStreamer works through abstract pipeline concepts. A pipeline is a directed graph in which media flows from input stream to output. Pipes are made up of various elements, and elements are another core concept. An element is an object that can be put into a pipe that wraps some operation on the media. You can link different elements together so that together they form the complete process of converting input to desired output. Typically, the pipe is described using a data flow from left (upstream) to right (downstream). Pipes are written in the same way using gst-launch, which will be covered later in this article on gst-launch.

It is important to note that so far, everything is completely abstract. There is no video or audio involved, and there is a good reason for doing so. The model described above is not limited to any particular media type. As long as you can describe it as an input, output, and transformation process, you can use pipes to operate on it. For example, the desktop can be used as a media source, and you can record on-screen playback of video files. In fact, Istanbul applications do exactly that (see the Resources section).

The core of GStreamer itself does not contain any elements. It only provides knowledge about pipelines. All specific content is provided by the corresponding plug-in. A plug-in is a piece of compiled code that is usually distributed as an object file (.so on UNIX and .dll on Microsoft Windows) and can provide one or more elements. During startup, GStreamer queries all installed plug-ins for a set of elements available to the application. Plug-ins can usually call other libraries to accomplish specific tasks (for example, MPEG-2 × × can use existing libraries to process information in MPEG format), but applications don't need to know this. All it sees are elements that look and work in the same way.

Some plug-ins are distributed as core source packages and compiled into libraries, even conceptually as separate entities. Other basic plug-ins are distributed in the form of gst-based plug-ins. These basic plug-ins are included in most GStreamer installations. Then there are some good, bad, and bad gst plug-ins that collect these different plug-ins according to their level of support and license terms. Finally, there are plug-ins that are distributed or registered by third-party vendors specifically for specific applications.

achieve mastery through a comprehensive study of the subject

Now that you know about the pipeline, you also need to know how to map it to a GStreamer implementation. You will also learn more terminology in the process.

Why switch sources and receivers?

They are not actually exchanged. The receiving unit is where the data flow enters the element, while the source unit is where the data flow is generated. Therefore, an element that contains only the source unit is called the source, and the element that contains only the receiving unit is called the receiver. This is very logical, even if it may seem awkward at first glance.

As I mentioned, the element is the basic unit of the process, represented by the GstElement class. GStreamer is written in C, but it uses the GObject library from the GTK+ toolkit for object-oriented features (see the Resources section). The element contains units that are linked to other elements. There are two types of units:

The receiving unit provides input to the element. The data generated by the element can be accessed through the source unit.

The corresponding functions of these units are called capabilities. These functions indicate what type of data can flow through the unit. For example, if you examine a vorbisdec element (which is the × × of a free Vorbis code), you can see the code shown in listing 1. The dollar sign ($) at the beginning of the line indicates that the line is a regular UNIX Shell command.

Listing 1. Code snippet in vorbisdec element information

$gst-inspect-0.10 vorbisdec [...] Pad Templates: SRC template: 'src' Availability: Always Capabilities: audio/x-raw-float rate: [8000, 50000] channels: [1,6] endianness: 1234 width: 32 SINK template:' sink' Availability: Always Capabilities: audio/x-vorbis [...]

As you can see, there are two types of unit templates: one for the source (src) and the other for the receiver. The availability of the source unit is always (other possible availability values are sometimes and request) and can output raw floating-point audio at a 8kHz-to-50kHz rate, with 1 to 6 channels, small-end byte order, and 32-bit wide sampling. On the other hand, the receiving unit only receives the audio stream encoded by Vorbis.

These templates are critical for the pipes to work properly. When you try to link two elements together to form a pipe, GStreamer checks to see if their unit templates are compatible. This process is called negotiation. During the negotiation process, these elements try to find the best format that they support together. If such a format does not exist, the link will fail. Otherwise, they will agree to use a common format. This format is no longer a template, but a fixed capability, which means that all values are meaningful and explicit. You can then pass data from one element to another.

Now you know what you need before you start working. In order to complete the specific work, I will introduce the "Swiss × ×" sharp tool in GStreamer, the gst-launch tool.

Go back to the top of the page

Use gst-launch

Learn about your tools

In addition to gst-launch, GStreamer comes with a number of other tools, such as gst-inspect and gst-typefind. Please use these tools, they will be your best partners.

You can use gst-inspect when you don't know how to use an element. Specify the name of any element or plug-in, which will display all the information GStreamer knows about the object, which, of course, may be more relevant.

Using gst-typefind, the GStreamer version of the excellent old-fashioned file (1) UNIX utility, you can find out the file type (or exactly what GStreamer thinks it is).

Gst-launch is one of the most versatile tools you've ever encountered. To GStreamer, it is like the Shell of UNIX. With this tool, you can even build replicated pipes with special syntax (correspondingly called gst-launch syntax), as shown in listing 2.

Listing 2. Example of a gst-launch line

$gst-launch-0.10 filesrc location= "concept.mp3"! Decodebin! Alsasink Setting pipeline to PAUSED... Pipeline is PREROLLING... Pipeline is PREROLLED... Setting pipeline to PLAYING... New clock: audioclock0

Listing 2 is the simplest audio player. Now, I am using it to listen to concept.mp3. Table 1 explains how to read the command line from left to right.

Table 1. The element description of the syntax introduced in listing 2

The element describes gst-launch-0.10, which is the name of the command. -0.10 indicates that a specific version of GStreamer 0.10 should be used if the old 0.8 release is also installed. Filesrc location= "concept.mp3" it creates an element of the filesink class and sets its location property to concept.mp3. Because the filesrc element can read the file specified by location, this command creates a reader for the concept.mp3 file in the current directory. ! An exclamation point indicates a link to. Similar to the pipe symbol (|) in Shell, the exclamation point is chosen because it looks similar to (|) and does not need to be escaped in Shell (as long as it has spaces before and after it). Decodebin this is the autoplugger provided by GStreamer. Autoplugger is an element that specifies the data type of its input and output, and it can use other available elements to find subpipes that provide the desired results. Remember that all links in GStreamer should be typed, so the exclamation point (!) Implicitly represents the type information of the element it links. Because filesrc has ANY-type capabilities, decodebin first tries the typefind operation by convection. That is, it looks for feature symbols that represent the type. All of these operations are transparent to the user. Alsasink this is the element suitable for the audio output of my Linux system. It communicates with the sound card and provides it with raw audio samples. It must be in tune with the entire pipe because the sound card has a normal rate of data usage.

When I press Enter, it displays some status information until the pipe reaches the PLAYING state. Then, I start to stream the data, and I can hear the sound, and my sound card (audioclock0) sets its rhythm.

As you can see, GStreamer saves you a lot of work. You don't even need to know the type of media you are trying to decode. Keep in mind that just as Shell cannot replace C programs, gst-launch tools cannot replace complete GStreamer applications. For example, gst-launch does not allow you to control the pipe in any way after it starts, so you cannot skip some parts of the flow. However, it is still very useful, especially for fast tasks, such as recording audio files to another format or experimenting with pipelines only.

Go back to the top of the page

Further research

This article is just a brief introduction to what you can do with GStreamer. Obviously, it's nice to create an audio player with a simple Shell command. However, this is a very simple player without any user interface or controls. To add these items and more, you need to use some code. Even so, GStreamer API is simple and carefully considered. If you don't like C, you can choose from other bindings, including a carefully maintained subset of Python language bindings.

Please read the man page of gst-launch. The full syntax is broader, and you can use it to create more complex and interesting pipes, including those that can be created from code. Yes, you can even create your own gst-launch (see the gst_parse_launch () function documentation to see how to do this).

In addition, please join the mailing list and visit the IRC channel (# gstreamer@irc.freenode.net). GStreamer developers make up a very active group, and there is usually always someone to help you or get your help.

references

Study

You can refer to the English text of this article on the developerWorks global site. GStreamer Application Development Manual: read this manual to learn more about GStreamer concepts and how to write applications using GStreamer. GStreamer plug-in programmer's Guide: this guide is useful when you need to create your own plug-ins. GStreamer 0.10 Core reference Manual: this is a very valuable reference manual. GObject reference manual: read this manual to learn more about the object-oriented libraries used by GStreamer. AIX and UNIX: want to know more? DeveloperWorks's "AIX and UNIX" section offers hundreds of articles about AIX and UNIX, as well as entry-level, intermediate, and advanced tutorials that will be eye-opening. Technical lectures: listen to technical lectures and keep in sync with IBM technical experts.

Access to products and technology

GStreamer home page: visit this site for the latest information and downloads. Istanbul:Istanbul is a desktop session recorder that uses GStreamer. IBM trial software: develop your next project with IBM software, which can be downloaded directly from developerWorks.

Discuss

Participate in "AIX and UNIX" forum, developerWorks blog, and join the developerWorks community. Gstreamer-devel: don't hesitate to post messages on gstreamer-devel, which is a mailing list for GStreamer development and use.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.